The MULTTEST Procedure

Example 61.2 Freeman-Tukey and t Tests with Bootstrap Resampling

The data for this example are the same as for Example 61.1, except that a continuous variable T, which indicates the time of death of the animal, has been added.

data a;
   input S1 S2 T Dose @@;
   datalines;
0 1 104 1   1 0  80 1   0 1 104 1
0 1 104 1   0 1 100 1   1 0 104 1
1 0  85 2   1 0  60 2   0 1  89 2
1 0  96 2   0 1  96 2   1 0  99 2
1 0  60 3   1 0  50 3   1 0  80 3
0 1  98 3   0 1  99 3   1 0  50 3
;
proc multtest data=a bootstrap nsample=10000 seed=37081 outsamp=res;
   test ft(S1 S2 / lowertailed) mean(T / lowertailed);
   class Dose;
   contrast 'Linear Trend' 0 1 2;
run;
proc print data=res(obs=36);
run;

The BOOTSTRAP option in the PROC MULTTEST statement requests bootstrap resampling, and NSAMPLE=10000 requests 10,000 bootstrap samples. The SEED=37081 option provides a starting value for the random number generator. The OUTSAMP=res option creates an output SAS data set res containing the 10,000 bootstrap samples.

The TEST statement specifies the Freeman-Tukey test for S1 and S2 and specifies the t test for T. Both tests are lower-tailed. The grouping variable in the CLASS statement is Dose, and the coefficients across the levels of Dose are 0, 1, and 2, as specified in the CONTRAST statement. The PROC PRINT statement displays the first 36 observations of the res data set containing the bootstrap samples.

The results from this analysis are listed in Output 61.2.1 through Output 61.2.5.

The Model Information table in Output 61.2.1 corresponds to the specifications in the invocation of PROC MULTTEST.

Output 61.2.1: FT and t tests with Bootstrap Resampling

The Multtest Procedure

Model Information
Test for discrete variables Freeman-Tukey
Test for continuous variables Mean t-test
Degrees of Freedom Method Pooled
Tails for discrete tests Lower-tailed
Tails for continuous tests Lower-tailed
Strata weights None
P-value adjustment Bootstrap
Center continuous variables Yes
Number of resamples 10000
Seed 37081


The Contrast Coefficients table in Output 61.2.2 shows the coefficients from the CONTRAST statement after centering, and they model a linear trend.

Output 61.2.2: Contrast Coefficients

Contrast Coefficients
Contrast   Dose
1 2 3
Linear Trend Centered -1 0 1


The summary statistics are displayed in Output 61.2.3. The values for the discrete variables S1 and S2 are the same as those from Example 61.1. The mean, standard deviation, and sample size for the continuous variable T at each level of Dose are displayed in the Continuous Variable Tabulations table.

Output 61.2.3: Summary Statistics

Discrete Variable Tabulations
Variable Dose Count NumObs Percent
S1 1 2 6 33.33
S1 2 4 6 66.67
S1 3 4 6 66.67
S2 1 4 6 66.67
S2 2 2 6 33.33
S2 3 2 6 33.33

Continuous Variable Tabulations
Variable Dose NumObs Mean Standard Deviation
T 1 6 99.3333 9.6056
T 2 6 87.5000 14.4326
T 3 6 72.8333 22.7017


The p-values, displayed in Output 61.2.4, are from the Freeman-Tukey test for S1 and S2, and are from the t test for T.

Output 61.2.4: p-Values

p-Values
Variable Contrast Raw Bootstrap
S1 Linear Trend 0.8547 1.0000
S2 Linear Trend 0.1453 0.4605
T Linear Trend 0.0070 0.0281


The Raw column in Output 61.2.4 contains the results from the tests on the original data, while the Bootstrap column contains the bootstrap resampled adjustment to raw_p. Note that the adjusted p-values are larger than the raw p-values for all three variables. The adjusted p-values more accurately reflect the correlation of the raw p-values, the small size of the data, and the multiple testing.

Output 61.2.5 displays the first 36 observations of the SAS data set resulting from the OUTSAMP=RES option in the PROC MULTTEST statement. The entire data set has 180,000 observations, which is 10,000 times the number of observations in the data set.

Output 61.2.5: Resampling Data Set

Obs _sample_ _class_ _obs_ S1 S2 T
1 1 1 17 0 1 26.1667
2 1 1 8 1 0 -27.5000
3 1 1 5 0 1 0.6667
4 1 1 9 0 1 1.5000
5 1 1 7 1 0 -2.5000
6 1 1 3 0 1 4.6667
7 1 2 12 1 0 11.5000
8 1 2 12 1 0 11.5000
9 1 2 14 1 0 -22.8333
10 1 2 17 0 1 26.1667
11 1 2 1 0 1 4.6667
12 1 2 15 1 0 7.1667
13 1 3 4 0 1 4.6667
14 1 3 17 0 1 26.1667
15 1 3 14 1 0 -22.8333
16 1 3 15 1 0 7.1667
17 1 3 15 1 0 7.1667
18 1 3 6 1 0 4.6667
19 2 1 6 1 0 4.6667
20 2 1 17 0 1 26.1667
21 2 1 8 1 0 -27.5000
22 2 1 13 1 0 -12.8333
23 2 1 9 0 1 1.5000
24 2 1 8 1 0 -27.5000
25 2 2 9 0 1 1.5000
26 2 2 18 1 0 -22.8333
27 2 2 15 1 0 7.1667
28 2 2 14 1 0 -22.8333
29 2 2 9 0 1 1.5000
30 2 2 17 0 1 26.1667
31 2 3 16 0 1 25.1667
32 2 3 11 0 1 8.5000
33 2 3 14 1 0 -22.8333
34 2 3 18 1 0 -22.8333
35 2 3 18 1 0 -22.8333
36 2 3 10 1 0 8.5000


The _sample_ variable is the sample indicator and _class_ indicates the resampling group—that is, the level of the CLASS variable Dose assigned to the new observation. The number of the observation in the original data set is represented by _obs_. Also listed are the values of the original test variables, S1 and S2, and the mean-centered values of T.