OLS is consistent under heteroscedasticity

Problem

  • Setup
    • Random sample \(\left( y_i,x_i \right)\) for \(i=1, \ldots ,n\)
    • Correctly specified linear regression model, \(y_i=x'_iβ+ε_i\) or \(y=Xβ+ε\) where \(E\left( ε_i \mid x_i \right)=0\) .
    • \(Var\left( ε_i \mid x_i \right)=σ^2σ_i^2\) where \(σ^2\) is unknown and \(σ_i^2\) is known or in matrix form, \(Var\left( ε \mid X \right)=σ^2Ψ\) where \(Ψ=diag\left( σ_1^2, \ldots ,σ_n^2 \right)\) .
    • \(b_{OLS}={\left( X'X \right)}^{-1}X'y\)

\(Σ_{xx'}=E\left( x_ix'_i \right)\) . Assume that \(Σ_{xx'}\) is invertible. Show that the OLS estimator is still consistent,

\[\textrm{plim } b_{OLS}=β\]

Solution

The proof is basically identical to the proof that when we have homoscedasticity.

Start with \(b_{OLS}=β+{\left( X'X \right)}^{-1}X'ε\) so

\[plim b_{OLS}=β+plim {\left( X'X \right)}^{-1}X'ε\]

\[=β+plim {\left( n^{-1}X'X \right)}^{-1}n^{-1}X'ε=\]

\[=β+{\left( plim n^{-1}X'X \right)}^{-1}⋅plim n^{-1}X'ε\]

\(plim n^{-1}X'X=Σ_{xx'}\) and \(plim n^{-1}X'ε=0\) (the expected value is zero and the variance goes to zero due to random sampling).

\(E\left( n^{-1}X'ε \right)=E\left( E\left( n^{-1}X'ε \mid X \right) \right)=0\) as before but there is a small adjustment in the details of the variance

\[Var\left( n^{-1}X'ε \right)=E\left( Var\left( n^{-1}X'ε \mid X \right) \right)=n^{-2}E\left( X'Var\left( ε \mid X \right)X \right)=n^{-2}σ^2E\left( X'ΨX \right)\]

This must go to zero since (random sample)

\[E\left( X'ΨX \right)=E\left( \sum_{i=1}^{n}{ σ_i^2x_ix'_i } \right)=nE\left( σ_i^2x_ix'_i \right)\]