Proof that the OLS estimator is unbiased
Problem
- Setup:
- a linear regression model \(y=Xβ+ε\) with a random sample
- exogeneity, \(E\left( ε \right|X)=0\)
- Show that the OLS estimator \(b={\left( X'X \right)}^{-1}X'y\) is unbiased.
- Hint: Begin with The OLS statistical formula. Then show that \(E\left( b \mid X \right)=β\) . Finally, use the law of iterated expectations.
Solution
\(b=β+{\left( X'X \right)}^{-1}X'ε\) . Thus,
\[E\left( b \mid X \right)=E\left( β+{\left( X'X \right)}^{-1}X'ε \mid X \right)=E\left( β \mid X \right)+E\left( {\left( X'X \right)}^{-1}X'ε \mid X \right)\]
The first term is just \(β\) . For the second term, we can take \({\left( X'X \right)}^{-1}X'\) outside since we are conditioning on \(X\) :
\[E\left( b \mid X \right)=β+{\left( X'X \right)}^{-1}X'E\left( ε \mid X \right)=β+{\left( X'X \right)}^{-1}X'0=β\]