The multiplicity problem: When multiple statistical tests are run (and p-values obtained) on the same set of data, the chance of at least one spurious significance (type-I error) is greater than the accepted value (5%). A way of controlling the so called FamilyWise Error Rate (FWER) is needed. Bonferroni method is widely used but considered too conservative (low power to detect real significances). Other methods, like Holm's, Hommel's, Finner's, use a sequential procedure to adjust the p-values to control the FWER, while retaining good power. The following code will compute adjusted p-values with several methods: 1. One-Step Bonferroni 2. One-Step Sidak 3. Step-Down Holm 4. Step-Down Sidak 5. Step-Down Finner 6. Step-Up Hommel 7. Step-Up Hochberg 8. Step-Up Simes A good general reference on p-value adjustment algorithms is: Westfall PH and Young SS (1993), Resampling-Based Multiple Testing: examples and methods for p-value adjustment. New York: John Wiley and Sons. ********************************************************************* BEGINNING OF SYNTAX * Create dataset with p-values. Replace by your own *. DATA LIST list / pvalue(F9.4). BEGIN DATA 0.0728 0.0023 0.3829 0.0041 0.0101 0.4557 END DATA. * SOME AUXILIARY VARIABLES *. COMPUTE id=$CASENUM. FORMAT id (F2.0). SORT CASES BY pvalue (A) . COMPUTE pos=$CASENUM. FORMAT pos (F2.0). * Calculate the number of p values. RANK pvalue /n into N /PRINT=NO. * N contains the number of cases in the file. * ADJUSTED P-VALUES *. * (1) One step methods *. COMPUTE bonferr=pvalue*n. * Replace p-values>1 (absurd) with 1 *. IF bonferr>1 bonferr=1. COMPUTE sidak=1-(1-pvalue)**n. * (2) Step-down methods *. COMPUTE holm=(n-pos+1)*pvalue. * Replace p-values>1 (absurd) with 1 *. IF holm>1 holm=1. * Keep original descending order *. IF (holm1. COMPUTE cn=cn+lag(cn,1). END IF. SORT CASES BY pos(D). IF cn1 hommel=1. * Keep original ascending order *. IF (hommel>LAG(hommel,1)) hommel=LAG(hommel,1). COMPUTE hochberg=(n-pos+1)*pvalue. * Keep original ascending order *. IF (hochberg>LAG(hochberg,1)) hochberg=LAG(hochberg,1). COMPUTE simes=n*pvalue/pos. * Keep original ascending order *. IF (simes>LAG(simes,1)) simes=LAG(simes,1). * FINAL REPORTS *. FORMAT bonferr to simes (F9.4). VARIABLE LABELS id 'Nr.' /pvalue 'Original p-value' /pos 'Rank' /bonferr 'One-step Bonferroni' /sidak 'One-step Sidak' /holm 'Step-down Holm' /downsidk 'Step-down Sidak' /finner 'Step-down Finner' /hommel 'Step-up Hommel' /hochberg 'Step-up Hochberg' /simes 'Step-up Simes'. SORT CASES BY id (A). SUMMARIZE /TABLES=pvalue bonferr TO finner hommel TO simes /FORMAT=LIST NOCASENUM TOTAL /TITLE='Original and adjusted p-values' /CELLS=NONE. ********************************************************************* * END OF SYNTAX.