Second order derivatives in matrix form (Hessian)

Summary

• \(f\) is a real valued function of \(m\) variables \(x={\left( x_1, \ldots ,x_m \right)}'\) , \(f:R^m→R\) .
• The second order derivative of \(f\) with respect to the vector \(x\) , denoted by \( \frac{∂^2f}{∂x∂x'}\) is defined as the \(m×m\) matrix

\[ \frac{∂^2f}{∂x∂x'}=\begin{bmatrix} \frac{∂^2f}{∂x_1^2} & \ldots & \frac{∂^2f}{∂x_1∂x_m} \\ ⋮ & ⋱ & ⋮ \\ \frac{∂^2f}{∂x_m∂x_1} & \ldots & \frac{∂^2f}{∂x_m^2}\end{bmatrix}\]

• This matrix is called the Hessian of \(f\) .
• Result: If \(f\left( x \right)=a'x\) then

\[ \frac{∂^2f}{∂x∂x'}=0\]

• 0 on the right hand side is the \(m×m\) zero matrix.
• Result: If \(f\left( x \right)=x'Ax\) and \(A\) is symmetric then

\[ \frac{∂^2f}{∂x∂x'}=2A\]