A t-distribution

Problem

  • Setup:
    • a linear regression model \(y=Xβ+ε\) with a random sample
    • \(b={\left( X'X \right)}^{-1}X'y\) is the OLS estimator of \(β\) (both are \(k×1\) )
    • \(e=y-Xb\) are the OLS residuals.
    • \(s^2=e'e/(n-k)\)
    • \(ε_i|x_i \sim N(0,σ^2)\)

if \(r={\left( r_1, \ldots ,r_k \right)}'\) is a \(k×1\) vector of parameters. Show that

\[ \frac{r'b-r'β}{SE\left( r'b \right)} \sim t_{n-k}\]

where

\[SE\left( r'b \right)=s\sqrt{r'{\left( X'X \right)}^{-1}r}\]

Solution

Conditionally on \(X\) , \(b \sim N\left( β,σ^2{\left( X'X \right)}^{-1} \right)\) . \(r'b\) is a linear combination of normally distributed random variables so \(r'b\) must be normal. We have

\[E\left( r'b \right)=r'E\left( b \right)=r'β\]

and

\[Var\left( r'b \right)=r'Var\left( b \right)r=σ^2r'{\left( X'X \right)}^{-1}r\]

We have

\[r'b \sim N\left( r'β,Var\left( r'b \right) \right)\]

or

\[r'b \sim N\left( r'β,r'Var\left( b \right)r \right)\]

or

\[r'b \sim N\left( r'β,σ^2r'{\left( X'X \right)}^{-1}r \right)\]

We also have

\[ \frac{r'b-r'β}{SD\left( r'b \right)} \sim N\left( 0,1 \right)\]

where

\[SD\left( r'b \right)=\sqrt{Var\left( r'b \right)}=\sqrt{r'Var\left( b \right)r}=\sqrt{σ^2r'{\left( X'X \right)}^{-1}r}=σ\sqrt{r'{\left( X'X \right)}^{-1}r}\]

Since

\[ \frac{\left( n-k \right)s^2}{σ^2} \sim χ_{n-k}^2\]

we have

\[ \frac{ \frac{r'b-r'β}{SD\left( r'b \right)}}{\sqrt{ \frac{\left( n-k \right)s^2}{σ^2}/(n-k)}} \sim t_{n-k}\]

Simplifying,

\[ \frac{r'b-r'β}{σ\sqrt{r'{\left( X'X \right)}^{-1}r}} \frac{σ}{s}= \frac{r'b-r'β}{s\sqrt{r'{\left( X'X \right)}^{-1}r}}= \frac{r'b-r'β}{SE\left( r'b \right)}\]