The WLS estimator

Problem

Random sample \(\left( y_i,x_i \right)\) for \(i=1, \ldots ,n\)
Correctly specified linear regression model, \(y_i=x'_iβ+ε_i\) , \(y=Xβ+ε\) where \(E\left( ε_i \mid x_i \right)=0\) .
\(Var\left( ε_i \mid x_i \right)=σ^2σ_i^2\) where \(σ^2\) is unknown and \(σ_i^2\) is known or in matrix form, \(Var\left( ε \mid X \right)=σ^2Ψ\) where \(Ψ=diag\left( σ_1^2, \ldots ,σ_n^2 \right)\) .
Transformed variables, vector form: \(y^*=Ψ^{-1/2}y\) , \(X^*=Ψ^{-1/2}X\) , \(ε^*=Ψ^{-1/2}ε\)
Transformed model: \(y^*=X^*β+ε^*\)

a. Show that

\[{\left( {X^*}'X^* \right)}^{-1}{X^*}'y^*={\left( X'Ψ^{-1}X \right)}^{-1}X'Ψ^{-1}y\]

b. Show that

\[{\left( X'Ψ^{-1}X \right)}^{-1}X'Ψ^{-1}y=β+{\left( X'Ψ^{-1}X \right)}^{-1}X'Ψ^{-1}ε\]

Solution

a. \({X^*}'=X'Ψ^{-1/2}\) and \({X^*}'X^*=X'Ψ^{-1/2}Ψ^{-1/2}X=X'Ψ^{-1}X\) . Similarly, \({X^*}'y^*=X'Ψ^{-1}y\)

\[{\left( X'Ψ^{-1}X \right)}^{-1}X'Ψ^{-1}y={\left( X'Ψ^{-1}X \right)}^{-1}X'Ψ^{-1}\left( Xβ+ε \right)=\]

\[{\left( X'Ψ^{-1}X \right)}^{-1}X'Ψ^{-1}Xβ+{\left( X'Ψ^{-1}X \right)}^{-1}X'Ψ^{-1}ε=\]

\[β+{\left( X'Ψ^{-1}X \right)}^{-1}X'Ψ^{-1}ε\]