OLS as a method of moments estimator

Problem

  • Setup
    • Random sample \(\left( y_i,x_i \right)\) for \(i=1, \ldots ,n\)
    • Correctly specified linear regression model, \(y_i=x'_iβ+ε_i\) where \(E\left( ε_i \mid x_i \right)=0\) , \(E\left( y_i \mid x_i \right)=x'_iβ\) .

a. Show that \(E\left( {x_iε}_i \right)=E\left( x_i\left( y_i-x'_iβ \right) \right)=0\)

b. Show that the solution to

\[ \frac{1}{n}\sum_{i=1}^{n}{ x_i\left( y_i-x'_ib_{MM} \right) }=0\]

is given by

\[b_{MM}={\left( \sum_{i=1}^{n}{ x_ix'_i } \right)}^{-1}\sum_{i=1}^{n}{ x_iy_i }={\left( X'X \right)}^{-1}X'y=b_{OLS}\]

Solution

a. \(E\left( {x_iε}_i \right)=E\left( E\left( {x_iε}_i|x_i \right) \right)=E\left( x_iE\left( ε_i|x_i \right) \right)=E\left( x_i⋅0 \right)=0\) . The first equality by LIE

b.

\[\sum_{i=1}^{n}{ x_i\left( y_i-x'_ib_{MM} \right) }=\sum_{i=1}^{n}{ x_iy_i }-\sum_{i=1}^{n}{ x_ix'_ib_{MM} }=\sum_{i=1}^{n}{ x_iy_i }-\left( \sum_{i=1}^{n}{ x_ix'_i } \right)b_{MM}=0\]

We have

\[\left( \sum_{i=1}^{n}{ x_ix'_i } \right)b_{MM}=\sum_{i=1}^{n}{ x_iy_i }\]

or

\[b_{MM}={\left( \sum_{i=1}^{n}{ x_ix'_i } \right)}^{-1}\sum_{i=1}^{n}{ x_iy_i }\]

In matrix notation, this is \({\left( X'X \right)}^{-1}X'y=b_{OLS}\) .