The REG Procedure
- Overview
-
Getting Started
-
Syntax
-
DetailsMissing ValuesInput Data SetsOutput Data SetsInteractive AnalysisModel-Selection MethodsCriteria Used in Model-Selection MethodsLimitations in Model-Selection MethodsParameter Estimates and Associated StatisticsPredicted and Residual ValuesModels of Less Than Full RankCollinearity DiagnosticsModel Fit and Diagnostic StatisticsInfluence StatisticsReweighting Observations in an AnalysisTesting for HeteroscedasticityTesting for Lack of FitMultivariate TestsAutocorrelation in Time Series DataComputations for Ridge Regression and IPC AnalysisConstruction of Q-Q and P-P PlotsComputational MethodsComputer Resources in Regression AnalysisDisplayed OutputODS Table NamesODS Graphics
-
Examples
- References
Criteria Used in Model-Selection Methods
When many significance tests are performed, each at a level of, for example, 5%, the overall probability of rejecting at least one true null hypothesis is much larger than 5%. If you want to guard against including any variables that do not contribute to the predictive power of the model in the population, you should specify a very small SLE= significance level for the FORWARD and STEPWISE methods and a very small SLS= significance level for the BACKWARD and STEPWISE methods.
In most applications, many of the variables considered have some predictive power, however small. If you want to choose the model that provides the best prediction computed using the sample estimates, you need only to guard against estimating more parameters than can be reliably estimated with the given sample size, so you should use a moderate significance level, perhaps in the range of 10% to 25%.
In addition to R square, the 
statistic is displayed for each model generated in the model-selection methods. The statistic is proposed by Mallows (1973) as a criterion for selecting a model. It is a measure of total squared error defined as
![\[ C_ p = \frac{\mbox{SSE}_ p}{s^2} - (N - 2p) \]](images/statug_reg0071.png){kind=small} |
where 
is the MSE for the full model, and 
is the sum-of-squares error for a model with p parameters including the intercept, if any. If is plotted against p, Mallows recommends the model where first approaches p. When the right model is chosen, the parameter estimates are unbiased, and this is reflected in near p. For further discussion, see Daniel and Wood (1980).
The adjusted R square statistic is an alternative to R square that is adjusted for the number of parameters in the model. The adjusted R square statistic is calculated as
![\[ \mbox{ADJRSQ} = 1 - \frac{(n - i)(1 - R^2)}{n - p} \]](images/statug_reg0073.png){kind=small} |
where n is the number of observations used in fitting the model, and i is an indicator variable that is 1 if the model includes an intercept, and 0 otherwise.