A t-distribution
Problem
- Setup:
- a linear regression model \(y=Xβ+ε\) with a random sample
- \(b={\left( X'X \right)}^{-1}X'y\) is the OLS estimator of \(β\) (both are \(k×1\) )
- \(e=y-Xb\) are the OLS residuals.
- \(s^2=e'e/(n-k)\)
- \(ε_i|x_i \sim N(0,σ^2)\)
if \(r={\left( r_1, \ldots ,r_k \right)}'\) is a \(k×1\) vector of parameters. Show that
\[ \frac{r'b-r'β}{SE\left( r'b \right)} \sim t_{n-k}\]
where
\[SE\left( r'b \right)=s\sqrt{r'{\left( X'X \right)}^{-1}r}\]
Solution
Conditionally on \(X\) , \(b \sim N\left( β,σ^2{\left( X'X \right)}^{-1} \right)\) . \(r'b\) is a linear combination of normally distributed random variables so \(r'b\) must be normal. We have
\[E\left( r'b \right)=r'E\left( b \right)=r'β\]
and
\[Var\left( r'b \right)=r'Var\left( b \right)r=σ^2r'{\left( X'X \right)}^{-1}r\]
We have
\[r'b \sim N\left( r'β,Var\left( r'b \right) \right)\]
or
\[r'b \sim N\left( r'β,r'Var\left( b \right)r \right)\]
or
\[r'b \sim N\left( r'β,σ^2r'{\left( X'X \right)}^{-1}r \right)\]
We also have
\[ \frac{r'b-r'β}{SD\left( r'b \right)} \sim N\left( 0,1 \right)\]
where
\[SD\left( r'b \right)=\sqrt{Var\left( r'b \right)}=\sqrt{r'Var\left( b \right)r}=\sqrt{σ^2r'{\left( X'X \right)}^{-1}r}=σ\sqrt{r'{\left( X'X \right)}^{-1}r}\]
Since
\[ \frac{\left( n-k \right)s^2}{σ^2} \sim χ_{n-k}^2\]
we have
\[ \frac{ \frac{r'b-r'β}{SD\left( r'b \right)}}{\sqrt{ \frac{\left( n-k \right)s^2}{σ^2}/(n-k)}} \sim t_{n-k}\]
Simplifying,
\[ \frac{r'b-r'β}{σ\sqrt{r'{\left( X'X \right)}^{-1}r}} \frac{σ}{s}= \frac{r'b-r'β}{s\sqrt{r'{\left( X'X \right)}^{-1}r}}= \frac{r'b-r'β}{SE\left( r'b \right)}\]