Matrix square root

Problem

• Setup
• Random sample \(\left( y_i,x_i \right)\) for \(i=1, \ldots ,n\)
• Correctly specified linear regression model, \(y_i=x'_iβ+ε_i\) or \(y=Xβ+ε\) where \(E\left( ε_i \mid x_i \right)=0\) .
• \(Var\left( ε_i \mid x_i \right)=σ^2σ_i^2\) where \(σ^2\) is unknown and \(σ_i^2\) is known or in matrix form, \(Var\left( ε \mid X \right)=σ^2Ψ\) where \(Ψ=diag\left( σ_1^2, \ldots ,σ_n^2 \right)\) .

Suppose that \(n=3\) ,

\[Ψ=\begin{bmatrix}σ_1^2 & 0 & 0 \\ 0 & σ_2^2 & 0 \\ 0 & 0 & σ_3^2\end{bmatrix}\]

where \(σ_i^2>0\) for \(i=1 \ldots ,n=3\)

a. Show that

\[Ψ^{1/2}=\begin{bmatrix}σ_1 & 0 & 0 \\ 0 & σ_2 & 0 \\ 0 & 0 & σ_3\end{bmatrix}\]

Hint: You need to show that \(Ψ^{1/2}⋅Ψ^{1/2}=Ψ\)

b. Show that

\[Ψ^{-1/2}=\begin{bmatrix}1/σ_1 & 0 & 0 \\ 0 & 1/σ_2 & 0 \\ 0 & 0 & 1/σ_3\end{bmatrix}\]

Hint: You need to show that \(Ψ^{-1/2}⋅Ψ^{1/2}=I\)

c. \(y={\left( y_1,y_2,y_3 \right)}'\) . Show that the transformed \(y\) ,

\[y^*=Ψ^{-1/2 }y=\begin{bmatrix}y_1/σ_1 \\ y_2/σ_2 \\ y_3/σ_3\end{bmatrix}\]

d. Given one explanatory variable,

\[X=\begin{bmatrix}1 & x_{1,2} \\ 1 & x_{2,2} \\ 1 & x_{3,2}\end{bmatrix}\]

show that

\[X^*=Ψ^{-1/2 }X=\begin{bmatrix}1/σ_1 & x_{1,2}/σ_1 \\ 1/σ_2 & x_{2,2}/σ_2 \\ 1/σ_3 & x_{3,2}/σ_3\end{bmatrix}\]

“Heteroscedasticity is easy” – we just put a weight on every observation. Observations with a high error–variance will get a smaller weight and less influence.