A t-distribution

Problem

a linear regression model \(y=Xβ+ε\) with a random sample
\(b={\left( X'X \right)}^{-1}X'y\) is the OLS estimator of \(β\) (both are \(k×1\) )
\(e=y-Xb\) are the OLS residuals.
\(s^2=e'e/(n-k)\)
\(ε_i|x_i \sim N(0,σ^2)\)

Show that

\[ \frac{b_j-β_j}{SE\left( b_j \right)} \sim t_{n-k}\]

Solution

We know that

\[ \frac{b_j-β_j}{SD\left( b_j \right)} \sim N\left( 0,1 \right)\]

and

\[ \frac{\left( n-k \right)s^2}{σ^2} \sim χ_{n-k}^2\]

Further, these are independent random variables. Therefore (see The t-distribution)

\[ \frac{ \frac{b_j-β_j}{SD\left( b_j \right)}}{\sqrt{ \frac{\left( n-k \right)s^2}{σ^2}/(n-k)}} \sim t_{n-k}\]

\(SD\left( b_j \right)=σ\sqrt{c_{j,j}}\) where \(c_{j,j}\) is element \(j,j\) of \({\left( X'X \right)}^{-1}\) . Thus, the left-hand side is

\[ \frac{b_j-β_j}{σ\sqrt{c_{j,j}}}⋅ \frac{σ}{s}= \frac{b_j-β_j}{s\sqrt{c_{j,j}}}= \frac{b_j-β_j}{SE\left( b_j \right)}\]