The variance of the OLS estimator

Problem

  • Setup:
    • a linear regression model \(y=Xβ+ε\) with a random sample
    • exogeneity \( E\left( ε \right|X)=0\)
    • homoscedasticity, \(Var\left( ε \right|X)=σ^2I\)
  • Show that \(Var\left( b \mid X \right)=σ^2{\left( X'X \right)}^{-1}\) .
  • Hint: Begin with The OLS statistical formula.

Solution

\(b=β+{\left( X'X \right)}^{-1}X'ε\) . Thus,

\[Var\left( b \mid X \right)=Var\left( β+{\left( X'X \right)}^{-1}X'ε \mid X \right)=Var\left( {\left( X'X \right)}^{-1}X'ε \mid X \right)\]

Since we are conditioning on \(X\) , we can take \({\left( X'X \right)}^{-1}X'\) “on the left” and the transpose of this “on the right”. Since the transpose of \({\left( X'X \right)}^{-1}X'\) is \(X{\left( X'X \right)}^{-1}\) we have

\(Var\left( \left( X'X \right)X'ε \mid X \right)={\left( X'X \right)}^{-1}X'Var\left( ε \mid X \right)X{\left( X'X \right)}^{-1}={\left( X'X \right)}^{-1}X'σ^2IX{\left( X'X \right)}^{-1}\)

\(σ^2\) is a scalar so we can put it anywhere we like and \(IX=X\) so \(I\) will go away and

\(Var\left( \left( X'X \right)X'ε \mid X \right)=σ^2{\left( X'X \right)}^{-1}\left( X'X \right){\left( X'X \right)}^{-1}\)

\({\left( X'X \right)}^{-1}\left( X'X \right)=I\) and

\[Var\left( \left( X'X \right)X'ε \mid X \right)=σ^2{\left( X'X \right)}^{-1}\]