The variance of the OLS estimator
Problem
- Setup:
- a linear regression model \(y=Xβ+ε\) with a random sample
- exogeneity \( E\left( ε \right|X)=0\)
- homoscedasticity, \(Var\left( ε \right|X)=σ^2I\)
- Show that \(Var\left( b \mid X \right)=σ^2{\left( X'X \right)}^{-1}\) .
- Hint: Begin with The OLS statistical formula.
Solution
\(b=β+{\left( X'X \right)}^{-1}X'ε\) . Thus,
\[Var\left( b \mid X \right)=Var\left( β+{\left( X'X \right)}^{-1}X'ε \mid X \right)=Var\left( {\left( X'X \right)}^{-1}X'ε \mid X \right)\]
Since we are conditioning on \(X\) , we can take \({\left( X'X \right)}^{-1}X'\) “on the left” and the transpose of this “on the right”. Since the transpose of \({\left( X'X \right)}^{-1}X'\) is \(X{\left( X'X \right)}^{-1}\) we have
\(Var\left( \left( X'X \right)X'ε \mid X \right)={\left( X'X \right)}^{-1}X'Var\left( ε \mid X \right)X{\left( X'X \right)}^{-1}={\left( X'X \right)}^{-1}X'σ^2IX{\left( X'X \right)}^{-1}\)
\(σ^2\) is a scalar so we can put it anywhere we like and \(IX=X\) so \(I\) will go away and
\(Var\left( \left( X'X \right)X'ε \mid X \right)=σ^2{\left( X'X \right)}^{-1}\left( X'X \right){\left( X'X \right)}^{-1}\)
\({\left( X'X \right)}^{-1}\left( X'X \right)=I\) and
\[Var\left( \left( X'X \right)X'ε \mid X \right)=σ^2{\left( X'X \right)}^{-1}\]