Regressor moments

Summary

• Given: a random sample \(\left( y_i,x_i \right)\) of size \(n\) where \(y_i\) is a scalar and \(x_i={\left( x_{i,1},x_{i,2}, \ldots ,x_{i,k} \right)}'\) is \(k×1\) .
• Definitions of first and second order moments of \(x\) ( \(j=1, \ldots ,k\) and \(l=1, \ldots ,k\) ):
• \(μ_{x_j}=E\left( x_{i,j} \right)\)
• \(ρ_{x_jx_l}=E\left( x_{i,j}x_{i,l} \right)\)
• \(ρ_{x_j}^2=E\left( x_{i,j}^2 \right)=ρ_{x_jx_j}\)
• \(σ_{x_jx_l}^2=cov\left( x_{i,j},x_{i,l} \right)=ρ_{x_jx_l}-μ_{x_j}μ_{x_l}\)
• \(σ_{x_j}^2=var\left( x_{i,j} \right)=σ_{x_jx_j}^2=ρ_{x_j}^2-μ_{x_j}^2\)
• Definition \(μ_x\) :

\[μ_x=E\left( x_i \right)=\begin{bmatrix}μ_{x_1} \\ ⋮ \\ μ_{x_k}\end{bmatrix}\]

• \(μ_x\) is \(k×1\) .
• Definition \(Σ_{xx'}\) (second order moment around zero)

\[Σ_{xx'}=E\left( x_ix'_i \right)=\begin{bmatrix}E\left( x_{i,1}^2 \right) & ⋯ & E\left( x_{i,1}x_{i,k} \right) \\ ⋮ & ⋱ & ⋮ \\ E\left( x_{i,k}x_{i,1} \right) & ⋯ & E\left( x_{i,k}^2 \right)\end{bmatrix}=\begin{bmatrix}ρ_{x_1}^2 & ⋯ & ρ_{x_1x_k} \\ ⋮ & ⋱ & ⋮ \\ ρ_{x_kx_1} & ⋯ & ρ_{x_k}^2\end{bmatrix}\]

• \(Σ_{xx'}\) is a \(k×k\) matrix.
• Result for \(Var\left( x_i \right)\) (second order moment around the mean)

\[Var\left( x_i \right)=E\left( \left( x_i-μ_x \right){\left( x_i-μ_x \right)}' \right)=Σ_{xx'}-μ_xμ'_x\]

• where

\[Var\left( x_i \right)=\begin{bmatrix}Var\left( x_{i,1} \right) & ⋯ & Cov\left( x_{i,1},x_{i,k} \right) \\ ⋮ & ⋱ & ⋮ \\ Cov\left( x_{i,1}x_{i,j} \right) & ⋯ & Var\left( x_{i,k} \right)\end{bmatrix}=\begin{bmatrix}σ_{x_1}^2 & ⋯ & σ_{x_jx_k} \\ ⋮ & ⋱ & ⋮ \\ σ_{x_kx_1} & ⋯ & σ_{x_k}^2\end{bmatrix}\]