The SEQDESIGN Procedure

Example 89.3 Creating Two-Sided Pocock and O’Brien-Fleming Designs

This example requests two 4-stage group sequential designs for normally distributed statistics with equally spaced information levels at all stages. One design uses Pocock’s method and the other uses the O’Brien-Fleming method. The following statements invoke the SEQDESIGN procedure and request these two designs:

proc seqdesign altref=0.4
               pss
               stopprob
               errspend
               ;
   TwoSidedPocock:        design nstages=4 method=poc;
   TwoSidedOBrienFleming: design nstages=4 method=obf;
   samplesize model=twosamplemean(stddev=0.8 weight=2);
run;

By default (or equivalently if you specify ALT=TWOSIDED and STOP=REJECT in the DESIGN statement), each design has a null hypothesis $H_{0}$ with a two-sided alternative with early stopping to reject $H_{0}$ .

The "Design Information" table in Output 89.3.1 displays design specifications and derived statistics for the Pocock’s design. With the specified ALTREF= option, the maximum information $I_ X= 77.6984$ is also derived.

Output 89.3.1: Pocock Design Information

The SEQDESIGN Procedure

Design: TwoSidedPocock

Design Information
Statistic Distribution	Normal
Boundary Scale	Standardized Z
Alternative Hypothesis	Two-Sided
Early Stop	Reject Null
Method	Pocock
Boundary Key	Both
Alternative Reference	0.4
Number of Stages	4
Alpha	0.05
Beta	0.1
Power	0.9
Max Information (Percent of Fixed Sample)	118.3143
Max Information	77.69844
Null Ref ASN (Percent of Fixed Sample)	115.6074
Alt Ref ASN (Percent of Fixed Sample)	69.74805

With the corresponding fixed-sample information

$I_{0} = \frac{ ( \Phi ^{-1}(1-\alpha /2) + \Phi ^{-1}(1-\beta ) )^{2} }{ {0.4}^{2} } = \frac{ (1.96 + 1.28155)^{2} }{ 0.16 } = 65.6728$

the fixed-sample information ratio is $77.6984 / 65.6728 = 1.1831$ .

For a two-sided design with early stopping to reject the null hypothesis, lower and upper $\alpha$ boundaries are created. The "Method Information" table in Output 89.3.2 displays the $\alpha$ and $\beta$ errors, alternative references, and derived drift parameters, which are the standardized alternative references at the final stage.

Output 89.3.2: Method Information

Method Information
Boundary	Method	Alpha	Beta	Unified Family			Alternative Reference	Drift
Boundary	Method	Alpha	Beta	Rho	Tau	C	Alternative Reference	Drift
Upper Alpha	Pocock	0.02500	0.10000	0	0	2.36129	0.4	3.525869
Lower Alpha	Pocock	0.02500	0.10000	0	0	2.36129	-0.4	-3.52587

With the METHOD=POC option, the Pocock method is used for each boundary. The Pocock method is one of the unified family methods, and the table also displays its corresponding parameters $\rho =0$ as a unified family method and the derived parameters $C=2.3613$ for the boundary values.

With the PSS option, the "Power and Expected Sample Sizes" table in Output 89.3.3 displays powers and expected sample sizes under various hypothetical references $\theta = c_{i} \theta _{1}$ , where $\theta _{1}$ is the alternative reference and $c_{i}$ are values specified in the CREF= option. By default, $c_{i}= 0, 0.5, 1.0, 1.5$ .

Output 89.3.3: Power and Expected Sample Size Information

Powers and Expected Sample Sizes Reference = CRef * (Alt Reference)
CRef	Power	Sample Size
CRef	Power	Percent Fixed-Sample
0.0000	0.02500	115.6074
0.5000	0.34252	104.0615
1.0000	0.90000	69.7480
1.5000	0.99869	43.6600

Note that at $c_{i}=0$ , the null reference $\theta =0$ , and the power 0.025 corresponds to the one-sided Type I error probability 0.025. At $c_{i}=1$ , $\theta =\theta _{1}$ , the power 0.9 is the power of the design. The expected sample sizes are displayed in a percentage scale to its corresponding fixed-sample size design. With the specified SAMPLESIZE statement, the expected sample sizes for the specified model in the SAMPLESIZE statement are also displayed.

With the STOPPROB option, the "Expected Cumulative Stopping Probabilities" table in Output 89.3.4 displays the expected cumulative stopping stage and cumulative stopping probability to reject the null hypothesis $H_{0}$ at each stage under various hypothetical references $\theta = c_{i} \theta _{1}$ , where $\theta _{1}$ is the alternative reference and $c_{i}$ are values specified in the CREF= option. By default, $c_{i}= 0, 0.5, 1.0, 1.5$ .

Output 89.3.4: Stopping Probabilities

Expected Cumulative Stopping Probabilities Reference = CRef * (Alt Reference)
CRef	Expected Stopping Stage	Source	Stopping Probabilities
CRef	Expected Stopping Stage	Source	Stage_1	Stage_2	Stage_3	Stage_4
0.0000	3.908	Reject Null	0.01821	0.03155	0.04176	0.05000
0.5000	3.518	Reject Null	0.07005	0.15939	0.25242	0.34327
1.0000	2.358	Reject Null	0.27482	0.58074	0.78638	0.90002
1.5000	1.476	Reject Null	0.61145	0.92348	0.98900	0.99869

Note that at $c_{i}=0$ , the cumulative stopping probability to reject $H_{0}$ at the final stage is the overall Type I error probability 0.05. At $c_{i}=1$ , the alternative hypothesis $H_{1}: \theta =\theta _{1}$ , the cumulative stopping probability to reject $H_{0}$ includes both the probability in the lower rejection region and the probability in the upper rejection region. This stopping probability to reject $H_{0}$ at the final stage, 0.90002, is slightly greater than the power $1-\beta = 0.90$ , which corresponds to the cumulative stopping probability in the upper rejection region only. See the section Type I and Type II Errors for a detailed description of the Type II error probability $\beta$ .

The "Boundary Information" table in Output 89.3.5 displays the information level, alternative references, and boundary values at each stage. By default (or equivalently if you specify BOUNDARYSCALE=STDZ), the standardized Z scale is used to display the alternative references and boundary values. The resulting standardized alternative reference at stage k is given by $\theta _1 \sqrt {I_ k}$ , where $\theta _1$ is the alternative reference and $I_ k$ is the information level at stage k, $k= 1, 2, 3, 4$ .

Output 89.3.5: Boundary Information

Boundary Information (Standardized Z Scale) Null Reference = 0
_Stage_				Alternative		Boundary Values
	Information Level			Reference		Lower	Upper
	Proportion	Actual	N	Lower	Upper	Alpha	Alpha
1	0.2500	19.42461	55.94288	-1.76293	1.76293	-2.36129	2.36129
2	0.5000	38.84922	111.8858	-2.49317	2.49317	-2.36129	2.36129
3	0.7500	58.27383	167.8286	-3.05349	3.05349	-2.36129	2.36129
4	1.0000	77.69844	223.7715	-3.52587	3.52587	-2.36129	2.36129

By default (or equivalently if you specify INFO=EQUAL in the DESIGN statement), equally spaced information levels are used. With the SAMPLESIZE statement, the required sample size N is also displayed under the heading "Information Level." With the Pocock method, the standardized Z boundary values are identical at all stages for each $\alpha$ boundary.

At each interim stage, the hypothesis of $H_{0}: \theta = 0$ is rejected if the standardized normal test statistic $z \leq -2.36129$ , the lower $\alpha$ boundary, or $z \geq 2.36129$ , the upper $\alpha$ boundary. Otherwise, the trial continues to the next stage. At the final stage, stage 4, the trial stops and the hypothesis is rejected if the test statistic $|z_{4}| \geq 2.36129$ . Otherwise, the hypothesis is accepted.

The "Error Spending Information" in Output 89.3.6 displays cumulative error spending at each stage for each boundary. It shows that more $\alpha$ errors are used in early stages than in later stages.

Output 89.3.6: Error Spending Information

Error Spending Information
_Stage_	Information Level	Cumulative Error Spending
	Information Level	Lower		Upper
	Proportion	Alpha	Beta	Beta	Alpha
1	0.2500	0.00911	0.00002	0.00002	0.00911
2	0.5000	0.01577	0.00002	0.00002	0.01577
3	0.7500	0.02088	0.00002	0.00002	0.02088
4	1.0000	0.02500	0.10000	0.10000	0.02500

The "Sample Size Summary" table in Output 89.3.7 displays the specified parameters for the sample size computation of the two-sample test for mean difference.

Output 89.3.7: Sample Size Summary

Sample Size Summary
Test	Two-Sample Means
Mean Difference	0.4
Standard Deviation	0.8
Max Sample Size	223.7715
Expected Sample Size (Null Ref)	218.652
Expected Sample Size (Alt Ref)	131.9167
Weight (Group A)	2
Weight (Group B)	1

The "Sample Sizes (N)" table in Output 89.3.8 displays the derived sample sizes at each stage, in both fractional and integer numbers. With the WEIGHT=2 option, the allocation ratio is 2 for the first group and 1 for the second group. See the section Test for the Difference between Two Normal Means for the derivation of these sample sizes. With the fixed-sample information ratio 1.1831, the derived sample sizes in fractional numbers are derived by multiplying 1.1831 by the corresponding sample sizes in the fixed-sample design.

Output 89.3.8: Sample Sizes

Sample Sizes (N) Two-Sample Z Test for Mean Difference
_Stage_	Fractional N				Ceiling N
_Stage_	N	N(Grp 1)	N(Grp 2)	Information	N	N(Grp 1)	N(Grp 2)	Information
1	55.94	37.30	18.65	19.4246	57	38	19	19.7917
2	111.89	74.59	37.30	38.8492	113	75	38	39.4082
3	167.83	111.89	55.94	58.2738	168	112	56	58.3333
4	223.77	149.18	74.59	77.6984	225	150	75	78.1250

These fractional sample sizes are rounded up to integers under the heading "Ceiling N." When the resulting integer sample sizes are used, the corresponding information levels are slightly larger than the levels from the design. This can increase the power slightly if a trial uses these integer sample sizes.

Note that compared with other designs, a Pocock design can stop the trial early with a larger p-value. However, this might not be persuasive enough to make a new treatment widely accepted (Pocock and White, 1999).

The "Design Information" table in Output 89.3.9 displays design specifications and the derived statistics for the O’Brien-Fleming design. With the specified ALTREF= option, the maximum information $I_{X}= 67.1268$ is derived.

Output 89.3.9: O’Brien-Fleming Design Information

The SEQDESIGN Procedure

Design: TwoSidedOBrienFleming

Design Information
Statistic Distribution	Normal
Boundary Scale	Standardized Z
Alternative Hypothesis	Two-Sided
Early Stop	Reject Null
Method	O'Brien-Fleming
Boundary Key	Both
Alternative Reference	0.4
Number of Stages	4
Alpha	0.05
Beta	0.1
Power	0.9
Max Information (Percent of Fixed Sample)	102.2163
Max Information	67.12682
Null Ref ASN (Percent of Fixed Sample)	101.5728
Alt Ref ASN (Percent of Fixed Sample)	76.7397

With the corresponding fixed-sample information

$I_{0} = \frac{ ( \Phi ^{-1}(1-\alpha /2) + \Phi ^{-1}(1-\beta ) )^{2} }{ {0.4}^{2} } = \frac{ (1.96 + 1.28155)^{2} }{ 0.16 } = 65.6728$

the fixed-sample information ratio is $67.1268 / 65.6728 = 1.022$ . That is, the maximum information for the O’Brien-Fleming design is only 2.2% more than for the corresponding fixed-sample design.

The "Method Information" table in Output 89.3.10 displays the Type I $\alpha$ level and Type II $\beta$ level. It also displays the derived drift parameter $\theta _{1} \sqrt {I_{X}}$ , which is the standardized alternative reference at the final stage.

Output 89.3.10: Method Information

Method Information
Boundary	Method	Alpha	Beta	Unified Family			Alternative Reference	Drift
Boundary	Method	Alpha	Beta	Rho	Tau	C	Alternative Reference	Drift
Upper Alpha	O'Brien-Fleming	0.02500	0.10000	0.5	0	2.02429	0.4	3.277238
Lower Alpha	O'Brien-Fleming	0.02500	0.10000	0.5	0	2.02429	-0.4	-3.27724

With the METHOD=OBF option, the O’Brien-Fleming method is used for each boundary. The O’Brien-Fleming method is one of the unified family methods, and the table also displays its corresponding parameters $\rho =0.5$ as a unified family method and the derived parameter $C=2.0243$ for the boundary values.

With the PSS option, the "Power and Expected Sample Sizes" table in Output 89.3.11 displays powers and expected sample sizes under various hypothetical references $\theta = c_{i} \theta _{1}$ , where $\theta _{1}$ is the alternative reference and $c_{i}$ are values specified in the CREF= option.

Output 89.3.11: Power and Expected Sample Size Information

Powers and Expected Sample Sizes Reference = CRef * (Alt Reference)
CRef	Power	Sample Size
CRef	Power	Percent Fixed-Sample
0.0000	0.02500	101.5728
0.5000	0.36495	96.3684
1.0000	0.90000	76.7397
1.5000	0.99821	57.2590

Compared with the corresponding Pocock design, the O’Brien-Fleming design has a smaller maximum sample size, and smaller expected sample sizes under hypothetical references $\theta =0$ and $\theta =0.5 \, \theta _{1}$ , but larger expected sample sizes under hypothetical references $\theta =\theta _{1}$ and $\theta =1.5 \, \theta _{1}$ .

With the STOPPROB option, the "Expected Cumulative Stopping Probabilities" table in Output 89.3.12 displays the expected stopping stage and cumulative stopping probability to reject the null hypothesis at each stage under various hypothetical references $\theta = c_{i} \theta _{1}$ , where $\theta _{1}$ is the alternative reference and $c_{i}$ are values specified in the CREF= option.

Output 89.3.12: Stopping Probabilities

Expected Cumulative Stopping Probabilities Reference = CRef * (Alt Reference)
CRef	Expected Stopping Stage	Source	Stopping Probabilities
CRef	Expected Stopping Stage	Source	Stage_1	Stage_2	Stage_3	Stage_4
0.0000	3.975	Reject Null	0.00005	0.00422	0.02091	0.05000
0.5000	3.771	Reject Null	0.00062	0.04430	0.18392	0.36515
1.0000	3.003	Reject Null	0.00798	0.29296	0.69603	0.90000
1.5000	2.241	Reject Null	0.05584	0.73031	0.97315	0.99821

Compared with the corresponding Pocock design, the O’Brien-Fleming design has smaller stopping probabilities in early stages under each hypothetical reference.

The "Boundary Information" table in Output 89.3.13 displays the boundary values for the design that uses the O’Brien-Fleming method. Compared with the Pocock method, the standardized statistics $\alpha$ boundary values derived from the O’Brien-Fleming method in absolute values are larger in early stages and smaller in later stages. This makes the O’Brien-Fleming design less likely to reject the null hypothesis in early stages than the Pocock design. With the derived parameter $C=2.0243$ for the $\alpha$ boundary, the $\alpha$ boundaries at stage j are computed as $C \sqrt {4/j}$ , $j=1, \ldots , 4$ .

Output 89.3.13: Boundary Information

Boundary Information (Standardized Z Scale) Null Reference = 0
_Stage_				Alternative		Boundary Values
	Information Level			Reference		Lower	Upper
	Proportion	Actual	N	Lower	Upper	Alpha	Alpha
1	0.2500	16.7817	48.33131	-1.63862	1.63862	-4.04859	4.04859
2	0.5000	33.56341	96.66262	-2.31736	2.31736	-2.86278	2.86278
3	0.7500	50.34511	144.9939	-2.83817	2.83817	-2.33745	2.33745
4	1.0000	67.12682	193.3252	-3.27724	3.27724	-2.02429	2.02429

The "Error Spending Information" in Output 89.3.14 displays cumulative error spending at each stage for each boundary. With smaller $\alpha$ spending in early stages for the O’Brien-Fleming method, it also indicates that the O’Brien-Fleming design is less likely to reject the null hypothesis in early stages than the Pocock design.

Output 89.3.14: Error Spending Information

Error Spending Information
_Stage_	Information Level	Cumulative Error Spending
	Information Level	Lower		Upper
	Proportion	Alpha	Beta	Beta	Alpha
1	0.2500	0.00003	0.00000	0.00000	0.00003
2	0.5000	0.00211	0.00000	0.00000	0.00211
3	0.7500	0.01046	0.00000	0.00000	0.01046
4	1.0000	0.02500	0.10000	0.10000	0.02500

The "Sample Size Summary" table in Output 89.3.15 displays the specified parameters for the sample size computation of the two-sample test for mean difference.

Output 89.3.15: Sample Size Summary

Sample Size Summary
Test	Two-Sample Means
Mean Difference	0.4
Standard Deviation	0.8
Max Sample Size	193.3252
Expected Sample Size (Null Ref)	192.1081
Expected Sample Size (Alt Ref)	145.1404
Weight (Group A)	2
Weight (Group B)	1

The "Sample Sizes (N)" table in Output 89.3.16 displays the derived sample sizes at each stage, in both fractional and integer numbers. With the fixed-sample information ratio 1.0222, the required sample sizes in fractional numbers are derived by multiplying 1.0222 by the corresponding sample sizes in the fixed-sample design.

Output 89.3.16: Derived Sample Sizes

Sample Sizes (N) Two-Sample Z Test for Mean Difference
_Stage_	Fractional N				Ceiling N
_Stage_	N	N(Grp 1)	N(Grp 2)	Information	N	N(Grp 1)	N(Grp 2)	Information
1	48.33	32.22	16.11	16.7817	50	33	17	17.5313
2	96.66	64.44	32.22	33.5634	98	65	33	34.1996
3	144.99	96.66	48.33	50.3451	146	97	49	50.8669
4	193.33	128.88	64.44	67.1268	194	129	65	67.5338