Second order derivatives in matrix form (Hessian)

Summary

• $f$ is a real valued function of $m$ variables $x={\left( x_1, \ldots ,x_m \right)}'$ , $f:R^m→R$ .
• The second order derivative of $f$ with respect to the vector $x$ , denoted by $\frac{∂^2f}{∂x∂x'}$ is defined as the $m×m$ matrix

$$\frac{∂^2f}{∂x∂x'}=\begin{bmatrix} \frac{∂^2f}{∂x_1^2} & \ldots & \frac{∂^2f}{∂x_1∂x_m} \\ ⋮ & ⋱ & ⋮ \\ \frac{∂^2f}{∂x_m∂x_1} & \ldots & \frac{∂^2f}{∂x_m^2}\end{bmatrix}$$

• This matrix is called the Hessian of $f$ .
• Result: If $f\left( x \right)=a'x$ then

$$\frac{∂^2f}{∂x∂x'}=0$$

• 0 on the right hand side is the $m×m$ zero matrix.
• Result: If $f\left( x \right)=x'Ax$ and $A$ is symmetric then

$$\frac{∂^2f}{∂x∂x'}=2A$$