The FREQ Procedure

Binomial Proportion

If you specify the BINOMIAL option in the TABLES statement, PROC FREQ computes the binomial proportion for one-way tables. By default, this is the proportion of observations in the first variable level that appears in the output. (You can use the LEVEL= option to specify a different level for the proportion.) The binomial proportion is computed as

$\hat{p} = n_{1} ~ / ~ n$

where $n_{1}$ is the frequency of the first (or designated) level and n is the total frequency of the one-way table. The standard error of the binomial proportion is computed as

$\mr {se}(\hat{p}) = \sqrt { \hat{p} ~ (1 - \hat{p} ) ~ / ~ n }$

Binomial Confidence Limits

By default, PROC FREQ provides Wald and exact (Clopper-Pearson) confidence limits for the binomial proportion. If you do not specify any confidence limit requests with the CL= binomial-option, PROC FREQ computes Wald asymptotic confidence limits. You can also request Agresti-Coull, Jeffreys, and Wilson (score) confidence limits for the binomial proportion. For details about these binomial confidence limits, including comparisons of their performance, see Brown, Cai, and DasGupta (2001); Agresti and Coull (1998); Newcombe (1998b).

Wald Confidence Limits

Wald asymptotic confidence limits are based on the normal approximation to the binomial distribution. PROC FREQ computes the Wald confidence limits for the binomial proportion as

$\hat{p} ~ \pm ~ ( ~ z_{\alpha /2} \times \mr {se}(\hat{p}) ~ )$

where $z_{\alpha /2}$ is the $100(1-\alpha /2)$ percentile of the standard normal distribution. The confidence level $\alpha$ is determined by the ALPHA= option, which, by default, equals 0.05 and produces 95% confidence limits.

If you specify CL=WALD(CORRECT) or the CORRECT binomial-option, PROC FREQ includes a continuity correction of in the Wald asymptotic confidence limits. The purpose of this correction is to adjust for the difference between the normal approximation and the discrete binomial distribution. See Fleiss, Levin, and Paik (2003) for more information. The continuity-corrected Wald confidence limits for the binomial proportion are computed as

$\hat{p} ~ \pm ~ ( ~ z_{\alpha /2} \times \mr {se}(\hat{p}) ~ + ~ (1/2n) ~ )$

Agresti-Coull Confidence Limits

If you specify the CL=AGRESTICOULL binomial-option, PROC FREQ computes Agresti-Coull confidence limits for the binomial proportion as

$\tilde{p} ~ \pm ~ ( ~ z_{\alpha /2} \times \sqrt { \tilde{p} ~ (1-\tilde{p}) ~ / ~ \tilde{n} } ~ )$

where

$\displaystyle \tilde{n}_1$	$\displaystyle =$	$\displaystyle n_1 + z_{\alpha /2}^2/2$
$\displaystyle \tilde{n}$	$\displaystyle =$	$\displaystyle n + z_{\alpha /2}^2$
$\displaystyle \tilde{p}$	$\displaystyle =$	$\displaystyle \tilde{n}_1 ~ / ~ \tilde{n}$

The Agresti-Coull confidence interval has the same general form as the standard Wald interval but uses $\tilde{p}$ in place of $\hat{p}$ . For $\alpha =0.05$ , the value of $z_{\alpha /2}$ is close to 2, and this interval is the “add 2 successes and 2 failures” adjusted Wald interval of Agresti and Coull (1998).

Jeffreys Confidence Limits

If you specify the CL=JEFFREYS binomial-option, PROC FREQ computes Jeffreys confidence limits for the binomial proportion as

$\bigl ( ~ \beta ( \alpha /2, ~ n_1 + 1/2, ~ n - n_1 + 1/2 ), ~ ~ \beta ( 1-\alpha /2, ~ n_1 + 1/2, ~ n - n_1 + 1/2 ) ~ \bigr )$

where $\beta ( \alpha , b, c)$ is the $\alpha$ percentile of the beta distribution with shape parameters b and c. The lower confidence limit is set to 0 when , and the upper confidence limit is set to 1 when . This is an equal-tailed interval based on the noninformative Jeffreys prior for a binomial proportion. See Brown, Cai, and DasGupta (2001) for details. See Berger (1985) for information about using beta priors for inference on the binomial proportion.

Wilson (Score) Confidence Limits

If you specify the CL=WILSON binomial-option, PROC FREQ computes Wilson confidence limits for the binomial proportion. These are also known as score confidence limits (Wilson, 1927). The confidence limits are based on inverting the normal test that uses the null proportion in the variance (the score test). Wilson confidence limits are the roots of

$| p - \hat{p} | = z_{\alpha /2} \sqrt { p (1-p)/n }$

and are computed as

$\left( \hat{p} ~ + ~ z_{\alpha /2}^2/2n ~ \pm ~ z_{\alpha /2} \sqrt { \left( \hat{p} (1-\hat{p}) + z_{\alpha /2}^2 / 4n \right) / n } ~ \right) ~ / ~ \left( 1 + z_{\alpha /2}^2 / n \right)$

If you specify CL=WILSON(CORRECT) or the CORRECT binomial-option, PROC FREQ provides continuity-corrected Wilson confidence limits, which are computed as the roots of

$| p - \hat{p} | - 1/2n = z_{\alpha /2} \sqrt { p (1-p)/n }$

The Wilson interval has been shown to have better performance than the Wald interval and the exact (Clopper-Pearson) interval. For more information, see Agresti and Coull (1998); Brown, Cai, and DasGupta (2001); Newcombe (1998b).

Exact (Clopper-Pearson) Confidence Limits

Exact (Clopper-Pearson) confidence limits for the binomial proportion are constructed by inverting the equal-tailed test based on the binomial distribution. This method is attributed to Clopper and Pearson (1934). The exact confidence limits $p_{\mi {L}}$ and $p_{\mi {U}}$ satisfy the following equations, for $n_1 = 1, 2, \ldots n-1$ :

$\displaystyle \sum _{x=n_1}^{n} \binom {n}{x} p_{\mi {L}}^{~ x} (1 - p_{\mi {L}})^{~ n-x}$	$\displaystyle =$	$\displaystyle \alpha /2$
$\displaystyle \sum _{x=0}^{n_1} \binom {n}{x} p_{\mi {U}}^{~ x} (1 - p_{\mi {U}})^{~ n-x}$	$\displaystyle =$	$\displaystyle \alpha /2$

The lower confidence limit equals 0 when , and the upper confidence limit equals 1 when .

PROC FREQ computes the exact (Clopper-Pearson) confidence limits by using the F distribution as

$\displaystyle p_{\mi {L}}$	$\displaystyle =$	$\displaystyle \left( 1 + \frac{n - n_1 + 1}{n_1 ~ F(~ \alpha /2, ~ 2 n_1,~ 2(n - n_1 + 1) ~ )} \right)^{-1}$
$\displaystyle p_{\mi {U}}$	$\displaystyle =$	$\displaystyle \left( 1 + \frac{n - n_1}{(n_1 + 1) ~ F(~ 1-\alpha /2, ~ 2(n_1 + 1), ~ 2(n - n_1) ~ )} \right)^{-1}$

where $F(\alpha /2, b, c)$ is the $\alpha /2$ percentile of the F distribution with b and c degrees of freedom. See Leemis and Trivedi (1996) for a derivation of this expression. Also see Collett (1991) for more information about exact binomial confidence limits.

Because this is a discrete problem, the confidence coefficient (coverage probability) of the exact (Clopper-Pearson) interval is not exactly $(1-\alpha )$ but is at least $(1-\alpha )$ . Thus, this confidence interval is conservative. Unless the sample size is large, the actual coverage probability can be much larger than the target value. For more information about the performance of these confidence limits, see Agresti and Coull (1998); Brown, Cai, and DasGupta (2001); Leemis and Trivedi (1996).

Binomial Tests

The BINOMIAL option provides an asymptotic equality test for the binomial proportion by default. You can also specify binomial-options to request tests of noninferiority, superiority, and equivalence for the binomial proportion. If you specify the BINOMIAL option in the EXACT statement, PROC FREQ also computes exact p-values for the tests that you request with the binomial-options.

Equality Test

PROC FREQ computes an asymptotic test of the hypothesis that the binomial proportion equals , where you can specify the value of with the P= binomial-option. If you do not specify a null value with P=, PROC FREQ uses by default. The binomial test statistic is computed as

$z = ( \hat{p} - p_0 ) / \mr {se}$

By default, the standard error is based on the null hypothesis proportion as

$\mr {se} = \sqrt { p_0 ( 1 - p_0 ) / n}$

If you specify the VAR=SAMPLE binomial-option, the standard error is computed from the sample proportion as

$\mr {se} = \sqrt { \hat{p} ( 1 - \hat{p} ) / n }$

If you specify the CORRECT binomial-option, PROC FREQ includes a continuity correction in the asymptotic test statistic, towards adjusting for the difference between the normal approximation and the discrete binomial distribution. See Fleiss, Levin, and Paik (2003) for details. The continuity correction of is subtracted from the numerator of the test statistic if $(\hat{p} - p_0)$ is positive; otherwise, the continuity correction is added to the numerator.

PROC FREQ computes one-sided and two-sided p-values for this test. When the test statistic z is greater than zero (its expected value under the null hypothesis), PROC FREQ computes the right-sided p-value, which is the probability of a larger value of the statistic occurring under the null hypothesis. A small right-sided p-value supports the alternative hypothesis that the true value of the proportion is greater than . When the test statistic is less than or equal to zero, PROC FREQ computes the left-sided p-value, which is the probability of a smaller value of the statistic occurring under the null hypothesis. A small left-sided p-value supports the alternative hypothesis that the true value of the proportion is less than . The one-sided p-value can be expressed as

$\begin{equation*} P_1 = \begin{cases} \mr {Prob} (Z > z) \quad \mr {if} \hspace{.1in} z > 0 \\ \mr {Prob} (Z < z) \quad \mr {if} \hspace{.1in} z \leq 0 \\ \end{cases}\end{equation*}$

where Z has a standard normal distribution. The two-sided p-value is computed as $P_2 = 2 \times P_1$ .

If you specify the BINOMIAL option in the EXACT statement, PROC FREQ also computes an exact test of the null hypothesis $H_0\colon p = p_0$ . To compute the exact test, PROC FREQ uses the binomial probability function,

$\mr {Prob} (X = x ~ | ~ p_0 ) = \binom {n}{x} p_0^{~ x} ~ (1-p_0)^{~ (n-x)} \hspace{.2in} \mr {for} \hspace{.1in} x = 0,1,2, \ldots ,n$

where the variable X has a binomial distribution with parameters n and . To compute the left-sided p-value, $\mr {Prob} (X \leq n_1)$ , PROC FREQ sums the binomial probabilities over x from zero to . To compute the right-sided p-value, $\mr {Prob} (X \geq n_1)$ , PROC FREQ sums the binomial probabilities over x from to n. The exact one-sided p-value is the minimum of the left-sided and right-sided p-values,

$P_{1} = \min ~ ( ~ \mr {Prob}(X \leq n_1 ~ | ~ p_0 ), ~ \mr {Prob}(X \geq n_1 ~ | ~ p_0 ) ~ )$

and the exact two-sided p-value is computed as $P_{2} = 2 \times P_{1}$ .

Noninferiority Test

If you specify the NONINF binomial-option, PROC FREQ provides a noninferiority test for the binomial proportion. The null hypothesis for the noninferiority test is

$H_0\colon p - p_0 \leq -\delta$

versus the alternative

$H_ a\colon p - p_0 > -\delta$

where $\delta$ is the noninferiority margin and is the null proportion. Rejection of the null hypothesis indicates that the binomial proportion is not inferior to the null value. See Chow, Shao, and Wang (2003) for more information.

You can specify the value of $\delta$ with the MARGIN= binomial-option, and you can specify with the P= binomial-option. By default, $\delta = 0.2$ and .

PROC FREQ provides an asymptotic Wald test for noninferiority. The test statistic is computed as

$z = ( \hat{p} - p_0^\ast ) ~ / ~ \mr {se}$

where $p_0^\ast$ is the noninferiority limit,

$p_0^\ast = p_0 - \delta$

By default, the standard error is computed from the sample proportion as

$\mr {se} = \sqrt { \hat{p} (1 - \hat{p}) / n}$

If you specify the VAR=NULL binomial-option, the standard error is based on the noninferiority limit (determined by the null proportion and the margin) as

$\mr {se} = \sqrt { p_0^\ast (1 - p_0^\ast ) / n}$

If you specify the CORRECT binomial-option, PROC FREQ includes a continuity correction in the asymptotic test statistic z. The continuity correction of is subtracted from the numerator of the test statistic if $(\hat{p} - p_0^\ast )$ is positive; otherwise, the continuity correction is added to the numerator.

The p-value for the noninferiority test is

$P_ z = \mr {Prob} ( Z > z )$

where Z has a standard normal distribution.

As part of the noninferiority analysis, PROC FREQ provides asymptotic Wald confidence limits for the binomial proportion. These confidence limits are computed as described in the section Wald Confidence Limits but use the same standard error (VAR=NULL or VAR=SAMPLE) as the noninferiority test statistic z. The confidence coefficient is $100(1-2\alpha )$ % (Schuirmann, 1999). By default, if you do not specify the ALPHA= option, the noninferiority confidence limits are 90% confidence limits. You can compare the confidence limits to the noninferiority limit, $p_0^\ast = p_0 - \delta$ .

If you specify the BINOMIAL option in the EXACT statement, PROC FREQ provides an exact noninferiority test for the binomial proportion. The exact p-value is computed by using the binomial probability function with parameters $p_0^\ast$ and n,

$P_ x = \sum _{k=n_1}^{k=n} \binom {n}{k} (p_0^\ast )^{~ k} ~ (1-p_0^\ast )^{~ (n-k)}$

See Chow, Shao, and Wang (2003, p. 116) for details. If you request exact binomial statistics, PROC FREQ also includes exact (Clopper-Pearson) confidence limits for the binomial proportion in the equivalence analysis display. See the section Exact (Clopper-Pearson) Confidence Limits for details.

Superiority Test

If you specify the SUP binomial-option, PROC FREQ provides a superiority test for the binomial proportion. The null hypothesis for the superiority test is

$H_0\colon p - p_0 \leq \delta$

versus the alternative

$H_ a\colon p - p_0 > \delta$

where $\delta$ is the superiority margin and is the null proportion. Rejection of the null hypothesis indicates that the binomial proportion is superior to the null value. You can specify the value of $\delta$ with the MARGIN= binomial-option, and you can specify the value of with the P= binomial-option. By default, $\delta = 0.2$ and .

The superiority analysis is identical to the noninferiority analysis but uses a positive value of the margin $\delta$ in the null hypothesis. The superiority limit equals $p_0 + \delta$ . The superiority computations follow those in the section Noninferiority Test but replace – $\delta$ with $\delta$ . See Chow, Shao, and Wang (2003) for more information.

Equivalence Test

If you specify the EQUIV binomial-option, PROC FREQ provides an equivalence test for the binomial proportion. The null hypothesis for the equivalence test is

$H_0\colon p - p_0 \leq \delta _{\mi {L}} \hspace{.15in} \mr {or} \hspace{.15in} p - p_0 \geq \delta _{\mi {U}}$

versus the alternative

$H_ a\colon \delta _{\mi {L}} < p - p_0 < \delta _{\mi {U}}$

where $\delta _{\mi {L}}$ is the lower margin, $\delta _{\mi {U}}$ is the upper margin, and is the null proportion. Rejection of the null hypothesis indicates that the binomial proportion is equivalent to the null value. See Chow, Shao, and Wang (2003) for more information.

You can specify the value of the margins $\delta _ L$ and $\delta _ U$ with the MARGIN= binomial-option. If you do not specify MARGIN=, PROC FREQ uses lower and upper margins of –0.2 and 0.2 by default. If you specify a single margin value $\delta$ , PROC FREQ uses lower and upper margins of – $\delta$ and $\delta$ . You can specify the null proportion with the P= binomial-option. By default, .

PROC FREQ computes two one-sided tests (TOST) for equivalence analysis (Schuirmann, 1987). The TOST approach includes a right-sided test for the lower margin and a left-sided test for the upper margin. The overall p-value is taken to be the larger of the two p-values from the lower and upper tests.

For the lower margin, the asymptotic Wald test statistic is computed as

$z_{\mi {L}} = ( \hat{p} - p_{\mi {L}}^\ast ) ~ / ~ \mr {se}$

where the lower equivalence limit is

$p_{\mi {L}}^\ast = p_0 + \delta _{\mi {L}}$

By default, the standard error is computed from the sample proportion as

$\mr {se} = \sqrt { \hat{p} (1 - \hat{p}) / n}$

If you specify the VAR=NULL binomial-option, the standard error is based on the lower equivalence limit (determined by the null proportion and the lower margin) as

$\mr {se} = \sqrt { p_{\mi {L}}^\ast (1 - p_{\mi {L}}^\ast ) / n}$

If you specify the CORRECT binomial-option, PROC FREQ includes a continuity correction in the asymptotic test statistic $z_{\mi {L}}$ . The continuity correction of is subtracted from the numerator of the test statistic $(\hat{p} - p_{\mi {L}}^\ast )$ if the numerator is positive; otherwise, the continuity correction is added to the numerator.

The p-value for the lower margin test is

$P_{z,\mi {L}} = \mr {Prob} ( Z > z_{\mi {L}} )$

The asymptotic test for the upper margin is computed similarly. The Wald test statistic is

$z_{\mi {U}} = ( \hat{p} - p_{\mi {U}}^\ast ) ~ / ~ \mr {se}$

where the upper equivalence limit is

$p_{\mi {U}}^\ast = p_0 + \delta _{\mi {U}}$

By default, the standard error is computed from the sample proportion. If you specify the VAR=NULL binomial-option, the standard error is based on the upper equivalence limit as

$\mr {se} = \sqrt { p_{\mi {U}}^\ast (1 - p_{\mi {U}}^\ast ) / n}$

If you specify the CORRECT binomial-option, PROC FREQ includes a continuity correction of in the asymptotic test statistic $z_{\mi {U}}$ .

The p-value for the upper margin test is

$P_{z,\mi {U}} = \mr {Prob} ( Z < z_{\mi {U}} )$

Based on the two one-sided tests (TOST), the overall p-value for the test of equivalence equals the larger p-value from the lower and upper margin tests, which can be expressed as

$P_ z = \max ( P_{z,\mi {L}} ~ , ~ P_{z,\mi {U}} )$

As part of the equivalence analysis, PROC FREQ provides asymptotic Wald confidence limits for the binomial proportion. These confidence limits are computed as described in the section Wald Confidence Limits, but use the same standard error (VAR=NULL or VAR=SAMPLE) as the equivalence test statistics and have a confidence coefficient of $100(1-2\alpha )$ % (Schuirmann, 1999). By default, if you do not specify the ALPHA= option, the equivalence confidence limits are 90% limits. If you specify VAR=NULL, separate standard errors are computed for the lower and upper margin tests, each based on the null proportion and the corresponding (lower or upper) margin. The confidence limits are computed by using the maximum of these two standard errors. You can compare the confidence limits to the equivalence limits, $( p_0 + \delta _{\mi {L}}, ~ p_0 + \delta _{\mi {U}} )$ .

If you specify the BINOMIAL option in the EXACT statement, PROC FREQ also provides an exact equivalence test by using two one-sided exact tests (TOST). The procedure computes lower and upper margin exact tests by using the binomial probability function as described in the section Noninferiority Test. The overall exact p-value for the equivalence test is taken to be the larger p-value from the lower and upper margin exact tests. If you request exact statistics, PROC FREQ also includes exact (Clopper-Pearson) confidence limits in the equivalence analysis display. The confidence coefficient is $100(1-2\alpha )$ % (Schuirmann, 1999). See the section Exact (Clopper-Pearson) Confidence Limits for details.