The IV estimator, asymptotics

Problem

  • Setup
    • Random sample \(\left( y_i,x_i,z_i \right)\) for \(i=1, \ldots ,n\) where \(x_i\) is \(k×1\) and \(z_i\) is \(k×1\)
    • A linear regression model, \(y_i=x'_iβ+ε_i\)
    • Exogeneity fails, \(E\left( ε_i \mid x_i \right)≠0\)
    • z-variables are instruments, \(E\left( ε_i|z_i \right)=0\) and \(E\left( z_ix'_i \right)=Σ_{zx'}\) is \(k×k\) and invertible
    • \(Var\left( ε_i|z_i \right)=σ^2\) .

Show that

a. \(b_{IV}=β+{\left( Z'X \right)}^{-1}Z'ε\)

b. \(plim\left( n^{-1}Z'X \right)=Σ_{zx'}\)

c. \(plim\left( n^{-1}Z'ε \right)=0\)

d. \(plim\left( b_{IV} \right)=β\)

e. \(\sqrt{n}\left( b_{IV}-β \right)→N\left( 0,σ^2{\left( Σ_{xz'}Σ_{zz'}^{-1}Σ_{zx'} \right)}^{-1} \right)\)

f. Explain why \(s^2{\left( X'P_ZX \right)}^{-1}\) makes sense as an estimator of \(Var\left( b_{IV} \right)\)

Solution

a.

\[b_{IV}={\left( Z'X \right)}^{-1}Z'y={\left( Z'X \right)}^{-1}Z'\left( Xβ+ε \right)={\left( Z'X \right)}^{-1}Z'Xβ+{\left( Z'X \right)}^{-1}Z'ε=β+{\left( Z'X \right)}^{-1}Z'ε\]

b.

\[E\left( n^{-1}Z'X \right)=E\left( \frac{1}{n}\sum_{i=1}^{n}{ z_ix'_i } \right)= \frac{1}{n}\sum_{i=1}^{n}{ E\left( z_ix'_i \right) }= \frac{1}{n}\sum_{i=1}^{n}{ Σ_{zx'} }=Σ_{zx'}\]

Element at position \(j,k\) in \(n^{-1}Z'X\) is

\[ \frac{1}{n}\sum_{i=1}^{n}{ z_{i,j}x_{i,k} }\]

which has variance

\[ \frac{1}{n^2}\sum_{i=1}^{n}{ Var\left( z_{i,j}x_{i,k} \right) }= \frac{1}{n}Var\left( z_{i,j}x_{i,k} \right)→0\]

c.

\[E\left( n^{-1}Z'ε \right)=E\left( E\left( n^{-1}Z'ε|Z \right) \right)=E\left( n^{-1}Z'E\left( ε|Z \right) \right)=E\left( n^{-1}Z'⋅0 \right)=0\]

By the law of iterated variance,

\[Var\left( n^{-1}Z'ε \right)=E\left( Var\left( n^{-1}Z'ε|Z \right) \right)+Var\left( E\left( n^{-1}Z'ε|Z \right) \right)=\]

\[E\left( n^{-2}Z'Var\left( ε|Z \right)Z \right)+Var\left( 0 \right)=n^{-2}σ^2E\left( Z'Z \right)=n^{-1}σ^2Σ_{zz'}→0\]

  1. d.

\[plim\left( b_{IV} \right)=plim\left( β+{\left( Z'X \right)}^{-1}Z'ε \right)=β+plim\left( {\left( Z'X \right)}^{-1}Z'ε \right)=\]

\[β+plim\left( {\left( n^{-1}Z'X \right)}^{-1} \right)plim\left( n^{-1}Z'ε \right)=\]

\[β+{\left( plim\left( n^{-1}Z'X \right) \right)}^{-1}plim\left( n^{-1}Z'ε \right)=β+Σ_{zx'}^{-1}⋅0=β\]

e.

\[\sqrt{n}\left( b_{IV}-β \right)=\sqrt{n}{\left( Z'X \right)}^{-1}Z'ε={\left( n^{-1}Z'X \right)}^{-1}\left( \frac{1}{\sqrt{n}}Z'ε \right)\]

As \(n→∞\) , \({\left( n^{-1}Z'X \right)}^{-1}→Σ_{zx'}^{-1}\) . Since

\[Var\left( \frac{1}{\sqrt{n}}Z'ε \right)=E\left( Var\left( n^{-1/2}Z'ε|Z \right) \right)=E\left( n^{-1}Z'Var\left( ε|Z \right)Z \right)=\]

\[=n^{-1}σ^2E\left( Z'Z \right)=σ^2Σ_{zz'}\]

neither goes to zero, nor explodes,

\[ \frac{1}{\sqrt{n}}Z'ε→N\left( 0,σ^2Σ_{zz'} \right)\]

As \(n→∞\) , \(\sqrt{n}\left( b_{IV}-β \right)\) becomes

\[Σ_{zx'}^{-1}⋅N\left( 0,σ^2Σ_{zz'} \right)\]

The transpose of \(Σ_{zx'}^{-1}\) is

\[{\left( Σ_{zx'}^{-1} \right)}'={\left( Σ'_{zx'} \right)}^{-1}={\left( E{\left( z_ix'_i \right)}' \right)}^{-1}={\left( E\left( \left( z_ix'_i \right)' \right) \right)}^{-1}={\left( E\left( x_iz'_i \right) \right)}^{-1}=Σ_{xz'}^{-1}\]

if we define \(Σ_{xz'}=E\left( x_iz'_i \right)\) . Therefore,

\[\sqrt{n}\left( b_{IV}-β \right)→N\left( 0,σ^2Σ_{zx'}^{-1}Σ_{zz'}Σ_{xz'}^{-1} \right)\]

We have (see matrix problem)

\[Σ_{zx'}^{-1}Σ_{zz'}Σ_{xz'}^{-1}={\left( Σ_{xz'}Σ_{zz'}^{-1}Σ_{zx'} \right)}^{-1}\]

and

\[\sqrt{n}\left( b_{IV}-β \right)→N\left( 0,σ^2{\left( Σ_{xz'}Σ_{zz'}^{-1}Σ_{zx'} \right)}^{-1} \right)\]

f. Since \(Σ_{xz'}=E\left( x_iz'_i \right)\) , we can consistently estimate \(Σ_{xz'}\) using

\[ \frac{1}{n}\sum_{i=1}^{n}{ x_iz'_i }= \frac{1}{n}X'Z\]

Similarly, \(Σ_{zz'}^{-1}\) can be consistently estimated using \({\left( n^{-1}Z'Z \right)}^{-1}\) and \(Σ_{zx'}\) can be consistently estimated using \(n^{-1}Z'X\) . Therefore, \(Σ_{xz'}Σ_{zz'}^{-1}Σ_{zx'}\) can be consistently estimated using

\[n^{-1}X'Z{\left( n^{-1}Z'Z \right)}^{-1}n^{-1}Z'X=n^{-1}X'Z{\left( Z'Z \right)}^{-1}Z'X=n^{-1}X'P_ZX\]

Therefore,

\[s^2{\left( n^{-1}X'P_ZX \right)}^{-1}=ns^2{\left( X'P_ZX \right)}^{-1}\]

Is a consistent estimator of \(Var\left( \sqrt{n}\left( b_{IV}-β \right) \right)=nVar\left( b_{IV} \right)\) which is why \(s^2{\left( X'P_ZX \right)}^{-1}\) makes sense as an estimator of \(Var\left( b_{IV} \right)\) .