Proof that the OLS estimator is unbiased

Problem

  • Setup:
    • a linear regression model \(y=Xβ+ε\) with a random sample
    • exogeneity, \(E\left( ε \right|X)=0\)
  • Show that the OLS estimator \(b={\left( X'X \right)}^{-1}X'y\) is unbiased.
  • Hint: Begin with The OLS statistical formula. Then show that \(E\left( b \mid X \right)=β\) . Finally, use the law of iterated expectations.

Solution

\(b=β+{\left( X'X \right)}^{-1}X'ε\) . Thus,

\[E\left( b \mid X \right)=E\left( β+{\left( X'X \right)}^{-1}X'ε \mid X \right)=E\left( β \mid X \right)+E\left( {\left( X'X \right)}^{-1}X'ε \mid X \right)\]

The first term is just \(β\) . For the second term, we can take \({\left( X'X \right)}^{-1}X'\) outside since we are conditioning on \(X\) :

\[E\left( b \mid X \right)=β+{\left( X'X \right)}^{-1}X'E\left( ε \mid X \right)=β+{\left( X'X \right)}^{-1}X'0=β\]