The UCM Procedure

Statistics of Fit

This section explains the goodness-of-fit statistics reported to measure how well the specified model fits the data.

First the various statistics of fit that are computed using the prediction errors, $y_{t} - \hat{y}_{t}$ , are considered. In these formulas, n is the number of nonmissing prediction errors and k is the number of fitted parameters in the model. Moreover, the sum of squared errors, ${\mi {SSE} = \sum {( y_{t} - \hat{y}_{t} )^{2}}}$ , and the total sum of squares for the series corrected for the mean, $SST = {\sum {( y_{t} - {\overline y} )^{2}}}$ , where ${{\overline y}}$ is the series mean, and the sums are over all the nonmissing prediction errors.

: Mean Squared Error The mean squared prediction error, ${\mi {MSE} = \frac{1}{n} \mi {SSE} }$
: Root Mean Squared Error The root mean square error, RMSE = ${\sqrt {\mi {MSE} }}$
: Mean Absolute Percent Error The mean absolute percent prediction error, MAPE = ${\frac{100}{n} \sum _{t=1}^{n}{{|( y_{t} - \hat{y}_{t} ) / y_{t} |}}}$ .The summation ignores observations where .
: R-square The R-square statistic, ${ \mi {R}^{2}=1-\mi {SSE} / \mi {SST} }$ .If the model fits the series badly, the model error sum of squares, SSE, might be larger than SST and the R-square statistic will be negative.
: Adjusted R-square The adjusted R-square statistic, ${1 - (\frac{n-1}{n-k}) (1- \mi {R} ^{2}) }$
: Amemiya’s Adjusted R-square Amemiya’s adjusted R-square, ${1 - (\frac{n+k}{n-k}) (1 - \mi {R} ^{2}) }$
: Random Walk R-square The random walk R-square statistic (Harvey’s R-square statistic that uses the random walk model for comparison), ${1 - (\frac{n-1}{n}) \mi {SSE} / \mi {RWSSE} }$ , where ${\mi {RWSSE} = \sum _{t=2}^{n}{( y_{t} - y_{t-1} - {\mu } )^{2}}}$ , and ${{\mu } = \frac{1}{n-1} \sum _{t=2}^{n}{( y_{t} - y_{t-1} )}}$
: Maximum Percent Error The largest percent prediction error, ${100 ~ \mr {max} ( ( y_{t} - \hat{y}_{t} ) / y_{t} )}$ . In this computation the observations where are ignored.

The likelihood-based fit statistics are reported separately (see the section The UCMs as State Space Models). They include the full log likelihood ( $L_{\infty }$ ), the diffuse part of the log likelihood, the normalized residual sum of squares, and several information criteria: AIC, AICC, HQIC, BIC, and CAIC. Let denote the number of estimated parameters, be the number of nonmissing measurements in the estimation span, and be the number of diffuse elements in the initial state vector that are successfully initialized during the Kalman filtering process. Moreover, let . The reported information criteria, all in smaller-is-better form, are described in Table 35.4:

Table 35.4: Information Criteria

Criterion	Formula	Reference
AIC	$-2 L_{\infty } + 2q$	Akaike (1974)
AICC	$-2 L_{\infty } + 2q n^/(n^-q-1)$	Hurvich and Tsai (1989)
		Burnham and Anderson (1998)
HQIC	$-2 L_{\infty } + 2q \log \log ( n^* )$	Hannan and Quinn (1979)
BIC	$-2 L_{\infty } + q \log ( n^* )$	Schwarz (1978)
CAIC	$-2 L_{\infty } + q(\log ( n^* ) + 1)$	Bozdogan (1987)