The IRT Procedure (Experimental)

Model and Item Fit

The IRT procedure includes five model fit statistics: log likelihood, Akaike’s information criterion (AIC), Bayesian information criterion (BIC), and likelihood ratio $G^2$ .

The following equation computes the likelihood ratio $G^2$ ,

$G^2 = 2\left[\sum _{l=1}^{2^ J} r_ l \log \frac{r_ l}{N P_ l}\right]$

where N is the number of subjects, $P_ l$ is the estimated probability of observing response pattern l, and $r_ l$ is the number of subjects who have response pattern l. When $2^ J$ , the number of possible response patterns, is much greater than N, the frequency table is sparse; this invalidates the use of chi-square distribution as the asymptotic distribution for these two statistics, and as a result they should not be used to evaluate overall model fit.

For item fit, PROC IRT computes the likelihood ratio $G^2$ and Pearson’s chi-square. The Pearson’s chi-square statistic, proposed by Yen (1981), has the form

$Q_{1j} = \sum _{k=1}^{10} N_ k\frac{(O_{jk} - E_{jk})^2}{E_{jk}(1-E_{jk})}$

The likelihood ratio $G^2$ , proposed by McKinley and Mills (1985), uses the following equation:

$G^2 = 2 \sum _{k=1}^{10} N_ k \left[ O_{jk} \log \frac{O_{jk}}{E_{ik}} + (1-O_{jk})\log \frac{1-O_{jk}}{1-E_{ik}}\right]$

To calculate these two statistics, first order all the subjects based on their estimated factor scores, and then partition them into 10 intervals such that the numbers of subjects in each interval are approximately equal. $O_{jk}$ and $E_{jk}$ are the observed and expected proportions of subjects in interval k who have a correct response on item j. The expected proportions $E_{jk}$ are computed as the mean predicted probability of a correct response in interval k.