The GAM Procedure

Additive Models and Generalized Additive Models

This section describes the methodology and the fitting procedure behind generalized additive models.

Let Y be a response random variable and $X_1, X_2, \cdots , X_ p$ be a set of predictor variables. A regression procedure can be viewed as a method for estimating the expected value of Y given the values of $X_1, X_2,\cdots , X_ p$ . The standard linear regression model assumes a linear form for the dependency of Y on X:

$Y = \beta _0 + \beta _1 X_1 + \beta _2 X_2 + \cdots + \beta _ p X_ p + \epsilon$

where $E(\epsilon ) = 0$ and $\mr {Var}(\epsilon )=\sigma ^2$ . Given a sample, estimates of $\beta _0, \beta _1, \cdots , \beta _ p$ are usually obtained by the least squares method.

The additive model generalizes the linear model by modeling the dependency as

$Y = s_0 + s_1(X_1) + s_2(X_2) + \cdots + s_ p(X_ p) + \epsilon$

where $s_ j(X), j=1,2,\ldots , p,$ are smooth functions, $E(\epsilon ) = 0$ and $\mr {Var}(\epsilon )=\sigma ^2$ .

In order to be estimable, the smooth functions have to satisfy standardized conditions such as $E\left(s_ j(X_ j)\right) = 0$ . These functions are not given a parametric form but instead are estimated in a nonparametric fashion.

While traditional linear models and additive models can be used in most statistical data analysis, there are types of problems for which they are not appropriate. For example, the normal distribution might not be adequate for modeling discrete responses such as counts or bounded responses such as proportions.

Generalized additive models address these difficulties, extending additive models to many other distributions besides just the normal. Thus, generalized additive models can be applied to a much wider range of data analysis problems.

Like generalized linear models, generalized additive models consist of a random component, an additive component, and a link function relating the two components. The response Y, the random component, is assumed to have exponential family density

$f_ Y(y;\theta , \phi ) = \exp \left\{ \frac{y\theta - b(\theta )}{a(\phi )} +c(y, \phi )\right\}$

where $\theta$ is called the natural parameter and $\phi$ is the scale parameter. The mean of the response variable $\mu$ is related to the set of covariates $X_1, X_2, \cdots , X_ p$ by a link function g. The quantity

$\eta = s_0 + \sum _{j=1}^ p s_ j(X_ j)$

defines the additive component, where $s_1(~ ), \cdots , s_ p(~ )$ are smooth functions, and the relationship between $\mu$ and $\eta$ is defined by $g(\mu ) = \eta$ . The most commonly used link function is the canonical link, for which $\eta = \theta$ .

Generalized additive models and generalized linear models can be applied in similar situations, but they serve different analytic purposes. Generalized linear models emphasize estimation and inference for the parameters of the model, while generalized additive models focus on exploring data nonparametrically. Generalized additive models are more suitable for exploring the data and visualizing the relationship between the dependent variable and the independent variables.