The WLS estimator is consistent

Problem

Random sample \(\left( y_i,x_i \right)\) for \(i=1, \ldots ,n\)
Correctly specified linear regression model, \(y=Xβ+ε\) where \(E\left( ε \mid X \right)=0\) .
\(Var\left( ε_i \mid x_i \right)=σ^2σ_i^2\) where \(σ^2\) is unknown and \(σ_i^2\) is known or in matrix form, \(Var\left( ε \mid X \right)=σ^2Ψ\) where \(Ψ=diag\left( σ_1^2, \ldots ,σ_n^2 \right)\) .
\(b_{WLS}={\left( X'Ψ^{-1}X \right)}^{-1}X'Ψ^{-1}y\)

Define

\[Σ=E\left( \frac{x_ix'_i}{σ_i^2} \right)\]

Assume that \(Σ\) is invertible. Show that \(b_{WLS}\) is consistent.

Solution

The solution is very similar to proving consistency of the OLS estimator.

Start from \(b_{WLS}=β+{\left( X'Ψ^{-1}X \right)}^{-1}X'Ψ^{-1}ε\) .

\[plim b_{WLS}=β+plim {\left( X'Ψ^{-1}X \right)}^{-1}X'Ψ^{-1}ε=\]

\[β+plim {\left( n^{-1}X'Ψ^{-1}X \right)}^{-1}⋅plim n^{-1}X'Ψ^{-1}ε\]

We have

\[n^{-1}X'Ψ^{-1}X=n^{-1}\sum_{i=1}^{n}{ \frac{x_ix'_i}{σ_i^2} }\]

and

\[plim {\left( n^{-1}X'Ψ^{-1}X \right)}^{-1}=Σ\]

For the next term,

\[E\left( n^{-1}X'Ψ^{-1}ε \right)=E\left( E\left( n^{-1}X'Ψ^{-1}ε \mid X \right) \right)=0\]

and

\[Var\left( n^{-1}X'Ψ^{-1}ε \right)=E\left( Var\left( n^{-1}X'Ψ^{-1}ε \mid X \right) \right)=n^{-2}σ^2E\left( X'Ψ^{-1}X \right)=n^{-1}σ^2Σ→0\]

\[plim n^{-1}X'Ψ^{-1}ε=0\]

Combining,

\[plim b_{WLS}=β+Σ^{-1}⋅0=β\]