Quasi-Newton Hessian Update Methods

Four update formulas are available via HessianUpdateMethod, selected with the Update keyword in the OPT section:

OPT
  Update  bfgs | psb | ms | bofill | sr1 | dfp | bfgs_powell | ts-bfgs
END

MSP (Mixed Symmetric Powell — default for minimisations)

Adaptively blends BFGS and Murtagh–Sargent updates via a mixing parameter $\varphi =1-(\Delta \mathbf{y}\cdot \mathbf{\Xi }{)}^2/((\Delta \mathbf{y}\cdot \Delta \mathbf{y})(\mathbf{\Xi }\cdot \mathbf{\Xi }))$ where $\mathbf{\Xi }=\mathbf{H}\cdot \mathbf{s}-\mathbf{y}$:

$${\mathbf{H}}_{\text{new}}=\mathbf{H}+\frac{\mathbf{y}{\mathbf{y}}^{†}}{\mathbf{y}\cdot \mathbf{s}}-\frac{(\mathbf{Hs})(\mathbf{Hs}{)}^{†}}{{\mathbf{s}}^{†}\mathbf{Hs}}\text{(BFGS part)}$$ $$+\varphi \cdot \frac{\mathbf{\Xi }{\mathbf{\Xi }}^{†}}{\mathbf{\Xi }\cdot \mathbf{s}}\text{(SR1 correction)}$$

BFGS (Broyden–Fletcher–Goldfarb–Shanno)

The classic rank-2 update:

$$\Delta \mathbf{H}=\frac{\mathbf{y}{\mathbf{y}}^{†}}{\mathbf{y}\cdot \mathbf{s}}-\frac{(\mathbf{Hs})(\mathbf{Hs}{)}^{†}}{{\mathbf{s}}^{†}\mathbf{Hs}}$$

Preserves positive-definiteness when $\mathbf{y}\cdot \mathbf{s}>0$; unsuitable for TS searches.

TS-BFGS (Transition-State BFGS — default for saddle-point searches)

Standard BFGS drives the Hessian positive-definite, which destroys the negative curvature direction needed for TS optimization. TS-BFGS replaces the curvature metric ${\mathbf{s}}^{†}\mathbf{Hs}$ with ${\mathbf{s}}^{†}|\mathbf{H}|\mathbf{s}$ (using the absolute-value Hessian $|\mathbf{H}|$, obtained by diagonalizing $\mathbf{H}$ and flipping negative eigenvalues to their absolute values) so that the negative eigenvalue directions are maintained across steps.

The rank-2 single-step formula:

$$\mathbf{u}=\frac{(\mathbf{s}\cdot \mathbf{y})\mathbf{y}+(\mathbf{s}\cdot |\mathbf{H}|\mathbf{s})|\mathbf{H}|\mathbf{s}}{(\mathbf{s}\cdot \mathbf{y}{)}^2+(\mathbf{s}\cdot |\mathbf{H}|\mathbf{s}{)}^2}$$ $$\mathbf{J}=\mathbf{y}-\mathbf{Hs}$$ $$\Delta \mathbf{H}=\mathbf{u}{\mathbf{J}}^{†}+\mathbf{J}{\mathbf{u}}^{†}-(\mathbf{J}\cdot \mathbf{s})\mathbf{u}{\mathbf{u}}^{†}$$ $${\mathbf{H}}_{\text{new}}=\mathbf{H}+\Delta \mathbf{H}$$

where $\mathbf{s}={\mathbf{q}}_{n+1}-{\mathbf{q}}_n$ is the IC-space displacement and $\mathbf{y}={\mathbf{g}}_{n+1}-{\mathbf{g}}_n$ is the gradient change.

Properties:

• The quasi-Newton secant condition ${\mathbf{H}}_{\text{new}}\mathbf{s}\approx \mathbf{y}$ is approximately satisfied
• Negative eigenvalues are preserved, not destroyed
• Reduces exactly to standard BFGS when $\mathbf{H}$ is positive-definite

SR1 (Symmetric Rank-1)

$$\mathbf{\xi }=\mathbf{y}-\mathbf{Hs}$$ $$\Delta \mathbf{H}=\frac{\mathbf{\xi }{\mathbf{\xi }}^{†}}{\mathbf{\xi }\cdot \mathbf{s}}[\text{skipped when }|\mathbf{\xi }\cdot \mathbf{s}|<10^{-8}]$$

SR1 can build indefinite Hessian approximations naturally (no positive-definite constraint). Can be unstable if $\mathbf{\xi }\cdot \mathbf{s}$ is small, so a skip condition is applied.

DFP (Davidon–Fletcher–Powell)

$$\Delta \mathbf{H}=\frac{\mathbf{y}{\mathbf{y}}^{†}}{\mathbf{y}\cdot \mathbf{s}}-\frac{(\mathbf{Hs})(\mathbf{Hs}{)}^{†}}{{\mathbf{s}}^{†}\mathbf{Hs}}$$

DFP is the inverse-Hessian analogue of BFGS. It preserves positive-definiteness but can be slow to converge on ill-conditioned surfaces.

Bofill (Bofill Weighted Update — default for TS searches)

The Bofill update blends the Powell-Symmetric-Broyden (PSB) and Murtagh-Sargent (MS) updates with a weight $\varphi$ that measures how well the secant condition is satisfied:

$$\varphi =1-\frac{({\mathbf{s}}^{†}\mathbf{y}{)}^2}{{\mathbf{s}}^{†}\mathbf{s}\cdot {\mathbf{y}}^{†}\mathbf{y}}$$ $${\mathbf{H}}_{\text{new}}=\varphi \cdot \text{PSB}(\mathbf{H},\mathbf{s},\mathbf{y})+(1-\varphi )\cdot \text{MS}(\mathbf{H},\mathbf{s},\mathbf{y})$$

Properties:

• When $\varphi \to 1$: pure PSB (good for TS, preserves curvature information)
• When $\varphi \to 0$: pure MS (good for minima, rank-1 correction)
• Automatically adapts between PSB and MS based on the local surface curvature

PSB (Powell-Symmetric-Broyden)

$$\mathbf{\xi }=\mathbf{y}-\mathbf{Hs}$$ $${\mathbf{H}}_{\text{new}}=\mathbf{H}+\frac{\mathbf{\xi }{\mathbf{s}}^{†}+\mathbf{s}{\mathbf{\xi }}^{†}}{{\mathbf{s}}^{†}\mathbf{s}}-\frac{({\mathbf{\xi }}^{†}\mathbf{s})\mathbf{s}{\mathbf{s}}^{†}}{({\mathbf{s}}^{†}\mathbf{s}{)}^2}$$

PSB is a symmetric rank-2 update that does not preserve positive-definiteness. It is used as part of the Bofill update and is suitable for TS searches where the Hessian must remain indefinite.

MS (Murtagh-Sargent)

$$\mathbf{\xi }=\mathbf{y}-\mathbf{Hs}$$ $${\mathbf{H}}_{\text{new}}=\mathbf{H}+\frac{\mathbf{\xi }{\mathbf{\xi }}^{†}}{{\mathbf{\xi }}^{†}\mathbf{s}}$$

MS is a rank-1 update that can produce indefinite Hessians naturally. It is used as part of the Bofill update and is the simplest update that allows negative eigenvalues.

BFGS/Powell Mixed

if is_ts:
    H_new = Bofill(H,s,y)    # φ·PSB + (1-φ)·MS
else:
    H_new = BFGS(H,s,y)      # Standard BFGS

This method automatically selects BFGS for minima and Bofill for TS searches, providing a seamless transition between the two regimes.

Summary