More on the distribution of the OLS estimator, the t-distribution

Summary

• Setup:
• a linear regression model \(y=Xβ+ε\) with a random sample
• \(b={\left( X'X \right)}^{-1}X'y\) is the OLS estimator of \(β\) (both are \(k×1\) )
• \(e=y-Xb\) are the OLS residuals.
• \(s^2=e'e/(n-k)\)
• \(ε_i|x_i \sim N(0,σ^2)\)
• \(b|X \sim N\left( β,σ^2{\left( X'X \right)}^{-1} \right)\)
• Result (conditionally on \(X\) ): for \(j=1, \ldots ,k\) :

\[b_j \sim N\left( β_j,Var\left( b_j \right) \right)\]

• where

\[Var\left( b_j \right)=σ^2c_{j,j}\]

• where \(c_{j,j}\) is element \(j,j\) of \({\left( X'X \right)}^{-1}\) .
• Result (conditionally on \(X\) ): for \(j=1, \ldots ,k\) :

\[ \frac{b_j-β_j}{SD\left( b_j \right)} \sim N\left( 0,1 \right)\]

• \(SD\left( b_j \right)=σ\sqrt{c_{j,j}}\) is the standard deviation of \(b_j\)
• Result (conditionally on \(X\) ): for \(j=1, \ldots ,k\) :

\[ \frac{b_j-β_j}{SE\left( b_j \right)} \sim t_{n-k}\]

• \(SE\left( b_j \right)=s\sqrt{c_{j,j}}\) is the standard error of \(b_j\) (estimated standard deviation).