Return to the Develop Models window (Figure 49.9) and notice the Root Mean Square Error button at the right side of the table banner. This is the model selection criterion—the
statistic used by the system to select the best fitting model. So far in this example you have fit two models and have left
the default criterion, root mean square error (RMSE), in effect. Because the Airline Model has the smaller value of this criterion
and because smaller values of the RMSE indicate better fit, the system has chosen this model as the forecasting model, indicated
by the check box in the Forecast Model
column.
The statistics available as model selection criteria are a subset of the statistics available for informational purposes.
To access the entire set, select Options
from the menu bar, and then select Statistics of Fit.
The Statistics of Fit Selection
window appears, as shown in Figure 49.12.
By default, five of the more well known statistics are selected. You can select and deselect statistics by clicking the check
boxes in the left column. For this exercise, select All
, and notice that all the check boxes become checked. Select the OK
button to close the window. Now if you choose Statistics of Fit
in the Model Viewer
window, all of the statistics will be shown for the selected model.
To change the model selection criterion, click the Root Mean Square Error
button or select Options
from the menu bar and then select Model Selection Criterion.
Notice that most of the statistics of fit are shown, but those which are not relevant to model selection, such as number
of observations, are not shown. Select Schwarz Bayesian Information Criterion
and click OK
. Since this statistic puts a high penalty on models with larger numbers of parameters, the ARIMA(0,1,0)(0,1,0)s model comes
out with the better fit.
Notice that changing the selection criterion does not automatically select the model that is best according to that criterion.
You can always choose the model you want to use for forecasts by selecting its check box in the Forecast Model
column.
Now bring up the Model Selection Criterion window again and select Akaike Information Criterion
. This statistic puts a lesser penalty on number of parameters, and the Airline Model comes out as the better fitting model.