The variance of the WLS estimator

Problem

  • Setup
    • Random sample \(\left( y_i,x_i \right)\) for \(i=1, \ldots ,n\)
    • Correctly specified linear regression model, \(y=Xβ+ε\) where \(E\left( ε \mid X \right)=0\) .
    • \(Var\left( ε_i \mid x_i \right)=σ^2σ_i^2\) where \(σ^2\) is unknown and \(σ_i^2\) is known or in matrix form, \(Var\left( ε \mid X \right)=σ^2Ψ\) where \(Ψ=diag\left( σ_1^2, \ldots ,σ_n^2 \right)\) .
    • \(b_{WLS}={\left( X'Ψ^{-1}X \right)}^{-1}X'Ψ^{-1}y\)

Show that

\[Var\left( b_{WLS}|X^* \right)=σ^2{\left( X'Ψ^{-1}X \right)}^{-1}\]

Solution

Note: It makes no difference if we condition on \(X\) or \(X^*\) since one is determined deterministically by the other. Start from \(b_{WLS}=β+{\left( X'Ψ^{-1}X \right)}^{-1}X'Ψ^{-1}ε\)

\[Var\left( b_{WLS}|X^* \right)=Var\left( {\left( X'Ψ^{-1}X \right)}^{-1}X'Ψ^{-1}ε|X^* \right)\]

\(Ψ\) is known conditionally on \(X^*\) (or \(X)\) and the transpose of \({\left( X'Ψ^{-1}X \right)}^{-1}X'Ψ^{-1}\) is \(Ψ^{-1}X{\left( X'Ψ^{-1}X \right)}^{-1}\) .

\[Var\left( {\left( X'Ψ^{-1}X \right)}^{-1}X'Ψ^{-1}ε|X^* \right)={\left( X'Ψ^{-1}X \right)}^{-1}X'Ψ^{-1}Var\left( ε|X^* \right)Ψ^{-1}X{\left( X'Ψ^{-1}X \right)}^{-1}=\]

\[{\left( X'Ψ^{-1}X \right)}^{-1}X'Ψ^{-1}σ^2Ψ Ψ^{-1}X{\left( X'Ψ^{-1}X \right)}^{-1}=\]

\[σ^2{\left( X'Ψ^{-1}X \right)}^{-1}X'Ψ^{-1}Ψ Ψ^{-1}X{\left( X'Ψ^{-1}X \right)}^{-1}=\]

\[σ^2{\left( X'Ψ^{-1}X \right)}^{-1}X' Ψ^{-1}X{\left( X'Ψ^{-1}X \right)}^{-1}=\]

\[σ^2{\left( X'Ψ^{-1}X \right)}^{-1}\]