OLS as a method of moments estimator
Problem
- Setup
- Random sample \(\left( y_i,x_i \right)\) for \(i=1, \ldots ,n\)
- Correctly specified linear regression model, \(y_i=x'_iβ+ε_i\) where \(E\left( ε_i \mid x_i \right)=0\) , \(E\left( y_i \mid x_i \right)=x'_iβ\) .
a. Show that \(E\left( {x_iε}_i \right)=E\left( x_i\left( y_i-x'_iβ \right) \right)=0\)
b. Show that the solution to
\[ \frac{1}{n}\sum_{i=1}^{n}{ x_i\left( y_i-x'_ib_{MM} \right) }=0\]
is given by
\[b_{MM}={\left( \sum_{i=1}^{n}{ x_ix'_i } \right)}^{-1}\sum_{i=1}^{n}{ x_iy_i }={\left( X'X \right)}^{-1}X'y=b_{OLS}\]
Solution
a. \(E\left( {x_iε}_i \right)=E\left( E\left( {x_iε}_i|x_i \right) \right)=E\left( x_iE\left( ε_i|x_i \right) \right)=E\left( x_i⋅0 \right)=0\) . The first equality by LIE
b.
\[\sum_{i=1}^{n}{ x_i\left( y_i-x'_ib_{MM} \right) }=\sum_{i=1}^{n}{ x_iy_i }-\sum_{i=1}^{n}{ x_ix'_ib_{MM} }=\sum_{i=1}^{n}{ x_iy_i }-\left( \sum_{i=1}^{n}{ x_ix'_i } \right)b_{MM}=0\]
We have
\[\left( \sum_{i=1}^{n}{ x_ix'_i } \right)b_{MM}=\sum_{i=1}^{n}{ x_iy_i }\]
or
\[b_{MM}={\left( \sum_{i=1}^{n}{ x_ix'_i } \right)}^{-1}\sum_{i=1}^{n}{ x_iy_i }\]
In matrix notation, this is \({\left( X'X \right)}^{-1}X'y=b_{OLS}\) .