Forecasting Process Details

Statistics of Fit

This section explains the goodness-of-fit statistics reported to measure how well different models fit the data. The statistics of fit for the various forecasting models can be viewed or stored in a data set by using the Model Viewer window.

Statistics of fit are computed by using the actual and forecasted values for observations in the period of evaluation. One-step forecasted values are used whenever possible, including the case when a hold-out sample contains no missing values. If a one-step forecast for an observation cannot be computed due to missing values for previous series observations, a multi-step forecast is computed, using the minimum number of steps as the previous nonmissing values in the data range permit.

The various statistics of fit reported are as follows. In these formulas, n is the number of nonmissing observations and k is the number of fitted parameters in the model.

Number of Nonmissing Observations. The number of nonmissing observations used to fit the model.

Number of Observations. The total number of observations used to fit the model, including both missing and nonmissing observations.

Number of Missing Actuals. The number of missing actual values.

Number of Missing Predicted Values. The number of missing predicted values.

Number of Model Parameters. The number of parameters fit to the data. For combined forecast, this is the number of forecast components.

Total Sum of Squares (Uncorrected). The total sum of squares for the series, SST, uncorrected for the mean: ${\sum _{t=1}^{n}{y_{t}^{2}}}$.

Total Sum of Squares (Corrected). The total sum of squares for the series, SST, corrected for the mean: ${\sum _{t=1}^{n}{( y_{t} - {\overline y} )^{2}}}$, where ${{\overline y}}$ is the series mean.

Sum of Square Errors. The sum of the squared prediction errors, SSE. ${\mi{SSE} = \sum _{t=1}^{n}{( y_{t} - \hat{y}_{t} )^{2}}}$, where ${\hat{y}}$ is the one-step predicted value.

Mean Squared Error. The mean squared prediction error, MSE, calculated from the one-step-ahead forecasts. ${\mi{MSE} = \frac{1}{n} \mi{SSE} }$. This formula enables you to evaluate small hold-out samples.

Root Mean Squared Error. The root mean square error (RMSE), ${\sqrt {\mi{MSE} }}$.

Mean Absolute Percent Error. The mean absolute percent prediction error (MAPE), ${\frac{100}{n} \sum _{t=1}^{n}{{|( y_{t} - \hat{y}_{t} ) / y_{t} |}}}$.The summation ignores observations where $y_ t = 0$.

Mean Absolute Error. The mean absolute prediction error, ${\frac{1}{n} \sum _{t=1}^{n}{{|y_{t} - \hat{y}_{t} |}}}$.

R-Square. The ${\mi{R} ^{2}}$ statistic, ${ \mi{R} ^{2}=1-\mi{SSE} / \mi{SST} }$. If the model fits the series badly, the model error sum of squares, SSE, can be larger than SST and the ${\mi{R} ^{2}}$ statistic will be negative.

Adjusted R-Square. The adjusted ${\mi{R} ^{2}}$ statistic, ${1 - (\frac{n-1}{n-k}) (1- \mi{R} ^{2}) }$.

Amemiya’s Adjusted R-Square. Amemiya’s adjusted ${\mi{R} ^{2}}$, ${1 - (\frac{n+k}{n-k}) (1 - \mi{R} ^{2}) }$.

Random Walk R-Square. The random walk ${\mi{R} ^{2}}$ statistic (Harvey’s ${ \mi{R} ^{2}}$ statistic by using the random walk model for comparison), ${1 - (\frac{n-1}{n}) \mi{SSE} / \mi{RWSSE} }$, where ${\mi{RWSSE} = \sum _{t=2}^{n}{( y_{t} - y_{t-1} - {\mu } )^{2}}}$, and ${{\mu } = \frac{1}{n-1} \sum _{t=2}^{n}{( y_{t} - y_{t-1} )}}$.

Akaike’s Information Criterion. Akaike’s information criterion (AIC), ${n ~ {\ln }( \mi{MSE} ) + 2 k}$.

Schwarz Bayesian Information Criterion. Schwarz Bayesian information criterion (SBC or BIC),${n ~ {\ln }( \mi{MSE} ) + k ~ {\ln }( n )}$.

Amemiya’s Prediction Criterion. Amemiya’s prediction criterion, ${\frac{1}{n} \mi{SST} (\frac{n+k}{n-k})(1- \mi{R} ^{2}) = (\frac{n+k}{n-k }) \frac{1}{n} \mi{SSE} }$.

Maximum Error. The largest prediction error.

Minimum Error. The smallest prediction error.

Maximum Percent Error. The largest percent prediction error, ${100 ~ \mr{max} ( ( y_{t} - \hat{y}_{t} ) / y_{t} )}$. The summation ignores observations where $y_ t = 0$.

Minimum Percent Error. The smallest percent prediction error, ${100 ~ \mr{min} ( ( y_{t} - \hat{y}_{t} ) / y_{t} )}$. The summation ignores observations where $y_ t = 0$.

Mean Error. The mean prediction error, ${\frac{1}{n} \sum _{t=1}^{n}{( y_{t} - \hat{y}_{t} )}}$.

Mean Percent Error. The mean percent prediction error, ${\frac{100}{n} \sum _{t=1}^{n}{\frac{(y_{t} - \hat{y}_{t})}{y_{t}}}}$. The summation ignores observations where $y_ t = 0$.