The NLMIXED Procedure

Hessian Scaling

The rows and columns of the Hessian matrix can be scaled when you use the trust region, Newton-Raphson, and double-dogleg optimization techniques. Each element $H_{i,j}$, $i,j=1,\ldots ,n$ is divided by the scaling factor $d_ i d_ j$, where the scaling vector $d=(d_1,\ldots ,d_ n)$ is iteratively updated in a way specified by the HESCAL= i option, as follows:

$i = 0:$

No scaling is done (equivalent to $d_ i=1$).

$i \neq 0:$

First iteration and each restart iteration sets:

\[ d_ i^{(0)} = \sqrt {\max (|H^{(0)}_{i,i}|,\epsilon )} \]
$i = 1:$

See Moré (1978):

\[ d_ i^{(k+1)} = \max \left[ d_ i^{(k)},\sqrt {\max (|H^{(k)}_{i,i}|, \epsilon )} \right] \]
$i = 2:$

See Dennis, Gay, and Welsch (1981):

\[ d_ i^{(k+1)} = \mr{max} \left[\, 0.6 d_ i^{(k)}, \sqrt {\max (|H^{(k)}_{i,i}|,\epsilon )} \right] \]
$i = 3:$

$d_ i$ is reset in each iteration:

\[ d_ i^{(k+1)} = \sqrt {\max (|H^{(k)}_{i,i}|,\epsilon )} \]

In the preceding equations, $\epsilon $ is the relative machine precision or, equivalently, the largest double-precision value that, when added to 1, results in 1.