The FMM Procedure

Metropolis-Hastings Algorithm

If Metropolis-Hastings is the only sampler available for the specified model (see Table 39.8) or if the METROPOLIS option is specified in the BAYES statement, PROC FMM uses the Metropolis-Hastings approach of Gamerman (1997). See the section Metropolis and Metropolis-Hastings Algorithms in Chapter 7: Introduction to Bayesian Analysis Procedures, for a general discussion of the Metropolis-Hastings algorithm.

The Gamerman (1997) algorithm derives a specific density that is used to generate proposals for the component-specific parameters $\bbeta _ j$ . The form of this proposal density is multivariate normal, with mean $\mb {m}_ j$ and covariance matrix $\bC _ j$ derived as follows.

Suppose $\bbeta _ j$ is the vector of model coefficients in the jth component and suppose that $\bbeta _ j$ has prior distribution $N(\mb {a}, \mb {R})$ . Consider a generalized linear model (GLM) with link function $g(\mu ) = \eta = \mb {x}’\bbeta$ and variance function $a(\mu )$ . The pseudo-response and weight in the GLM for a weighted least squares step are

$\begin{align*} y^* =& \eta + (y-\mu )/ \frac{\partial \mu }{\partial \eta } \\ w =& \frac{\partial \mu }{\partial \eta } / a(\mu ) \end{align*}$

If the model contains offsets or FREQ or WEIGHT statements, or if a trials variable is involved, suitable adjustments are made to these quantities.

In each component, $j=1, \cdots , k$ , form an adjusted cross-product matrix with a “pseudo” border

$\left[ \begin{array}{ll} \bX _ j’\bW _ j\bX _ j + \bR ^{-1} & \bX _ j’\bW _ j\mb {y}_ j^* + \bR ^{-1}\mb {a} \cr \mb {y}_ j^{*}\bW _ j\bX _ j + \mb {a}’\bR ^{-1} & c \end{array}\right]$

where $\bW _ j$ is a diagonal matrix formed from the pseudo-weights w, $\mb {y}^*$ is a vector of pseudo-responses, and c is arbitrary. This is basically a system of normal equations with ridging, and the degree of ridging is governed by the precision and mean of the normal prior distribution of the coefficients. Sweeping on the leading partition leads to

$\begin{align*} \bC _ j =& \left(\bX _ j’\bW _ j\bX _ j + \bR ^{-1} \right)^- \\ \mb {m}_ j =& \bC _ j \left(\bX _ j’\bW _ j\mb {y}_ j^* + \bR ^{-1}\mb {a}\right) \end{align*}$

where the generalized inverse is a reflexive, $g_2$ -inverse (see the section Linear Model Theory in Chapter 3: Introduction to Statistical Modeling with SAS/STAT Software, for details).

PROC FMM then generates a proposed parameter vector from the resulting multivariate normal distribution, and then accepts or rejects this proposal according to the appropriate Metropolis-Hastings thresholds.