The properties of the OLS estimator

Summary

Setup

The linear regression model with random sampling,

\[y_i=β_1+β_2x_{i,2}+β_3x_{i,3}+…+β_kx_{i,k}+ε_i, i=1,…,n\]

\(b_1,b_2,…,b_k\) are the OLS estimators of \(β_1,β_2,…,β_k\) .

Gauss-Markov assumptions

The following set of assumptions qualify as Gauss-Markov assumptions:

\[E\left( ε_i|x_i \right)=0, i=1,…,n\]

\[Var\left( ε_i|x_i \right)=σ^2, i=1,…,n\]

OLS unbiased

The OLS estimators \(b_1,b_2,…,b_k\) are unbiased if all the \(x\) -variables are exogenous

OLS consistent

The OLS estimators \(b_1,b_2,…,b_k\) are consistent under mild conditions if all the \(x\) -variables are exogenous.

Under GM assumptions,

It is possible to derive the variance of the of the OLS estimators (the OLS GM variance formulas)

\[Var\left( b_j|x \right) , j=1,…,k\]

This variance will depend on all the \(x\) -data as well as \(σ^2\) but the formulas are messy when \(k>2\) (without using matrices).
We define

\[s^2= \frac{RSS}{n-k}= \frac{1}{n-k}\sum_{i=1}^{n}{ e_i^2 }\]

Under GM, \(s^2\) (the OLS estimator of \(σ^2\) ) is an unbiased and consistent (under mild conditions) estimator of \(σ^2\) .
\(s\) , the square root of \(s^2\) , is called standard error of the regression.
If we replace \(σ^2\) with \(s^2\) in the OLS GM variance formulas, we get the estimated variance of \(b_1,b_2,…,b_k\) :

\[\widehat{Var}\left( b_j \right) , j=1,…,k\]