Hessian

Summary

• If \(f\) is a function of \(2\) variables then the \(2×2\) matrix

\[H=\begin{pmatrix} f''_{xx}(x,y) & f''_{xy}(x,y) \\ f''_{yx}(x,y) & f''_{yy}(x,y) \end{pmatrix}\]

• is called the Hessian of \(f\) .
• If \(f\) is a smooth function then \(f_{xy}^{''}\left( x,y \right)=f_{yx}^{''}\left( x,y \right)\) and the Hessian is a symmetric matrix.
• The determinant of the Hessian , denoted by \(|H|\) is

\[|H|=f_{xx}^{''}f_{yy}^{''}-f_{yx}^{''}f_{xy}^{''}\]

• which, in general, is a function of \(x\) and \(y\) .
• We say that the Hessian is:
• Positive semidefinite if and only if \(f_{xx}^{''}≥0\) and \(|H|≥0\)
• Negative semidefinite if and only if \(f_{xx}^{''}≤0\) and \(|H|≥0\)
• Positive definite if and only if \(f_{xx}^{''}>0\) and \(|H|>0\)
• Negative definite if and only if \(f_{xx}^{''}<0\) and \(|H|>0\)
• for all \(x,y\) in the domain of \(f\) .

Convex and concave functions

• We say that \(A⊆R^2\) is a convex set if the following condition holds: Pick any two points in \(A\) . If all points along a straight line between these points are in \(A\) then we say that \(A\) is a convex set.
• \(f:A→R^2\) is a function of \(2\) variables where \(A\) is a convex subset of \(R^2\) and \(f\) is twice differentiable. \(H\) is the Hessian of \(f\) . Then
• \(H\) is positive semidefinite on \(A\) \( \iff \) \(f\) is convex on \(A\)
• \(H\) is negative semidefinite on \(A\) \( \iff \) \(f\) is concave on \(A\)
• \(H\) is positive definite on \(A\) \(⇒\) \(f\) is strictly convex on \(A\) (opposite not necessarily true)
• \(H\) is negative definite on \(A\) \(⇒\) \(f\) is strictly concave on \(A\) (opposite not necessarily true)
• Example: \(f(x,y)=x^2+y^2\) on \(R^2\) has a Hessian which is positive definite for all \((x,y)\) and \(f\) is strictly convex on \(R^2\) .