The IV estimator, asymptotics
Problem
- Setup
- Random sample \(\left( y_i,x_i,z_i \right)\) for \(i=1, \ldots ,n\) where \(x_i\) is \(k×1\) and \(z_i\) is \(k×1\)
- A linear regression model, \(y_i=x'_iβ+ε_i\)
- Exogeneity fails, \(E\left( ε_i \mid x_i \right)≠0\)
- z-variables are instruments, \(E\left( ε_i|z_i \right)=0\) and \(E\left( z_ix'_i \right)=Σ_{zx'}\) is \(k×k\) and invertible
- \(Var\left( ε_i|z_i \right)=σ^2\) .
Show that
a. \(b_{IV}=β+{\left( Z'X \right)}^{-1}Z'ε\)
b. \(plim\left( n^{-1}Z'X \right)=Σ_{zx'}\)
c. \(plim\left( n^{-1}Z'ε \right)=0\)
d. \(plim\left( b_{IV} \right)=β\)
e. \(\sqrt{n}\left( b_{IV}-β \right)→N\left( 0,σ^2{\left( Σ_{xz'}Σ_{zz'}^{-1}Σ_{zx'} \right)}^{-1} \right)\)
f. Explain why \(s^2{\left( X'P_ZX \right)}^{-1}\) makes sense as an estimator of \(Var\left( b_{IV} \right)\)
Solution
a.
\[b_{IV}={\left( Z'X \right)}^{-1}Z'y={\left( Z'X \right)}^{-1}Z'\left( Xβ+ε \right)={\left( Z'X \right)}^{-1}Z'Xβ+{\left( Z'X \right)}^{-1}Z'ε=β+{\left( Z'X \right)}^{-1}Z'ε\]
b.
\[E\left( n^{-1}Z'X \right)=E\left( \frac{1}{n}\sum_{i=1}^{n}{ z_ix'_i } \right)= \frac{1}{n}\sum_{i=1}^{n}{ E\left( z_ix'_i \right) }= \frac{1}{n}\sum_{i=1}^{n}{ Σ_{zx'} }=Σ_{zx'}\]
Element at position \(j,k\) in \(n^{-1}Z'X\) is
\[ \frac{1}{n}\sum_{i=1}^{n}{ z_{i,j}x_{i,k} }\]
which has variance
\[ \frac{1}{n^2}\sum_{i=1}^{n}{ Var\left( z_{i,j}x_{i,k} \right) }= \frac{1}{n}Var\left( z_{i,j}x_{i,k} \right)→0\]
c.
\[E\left( n^{-1}Z'ε \right)=E\left( E\left( n^{-1}Z'ε|Z \right) \right)=E\left( n^{-1}Z'E\left( ε|Z \right) \right)=E\left( n^{-1}Z'⋅0 \right)=0\]
By the law of iterated variance,
\[Var\left( n^{-1}Z'ε \right)=E\left( Var\left( n^{-1}Z'ε|Z \right) \right)+Var\left( E\left( n^{-1}Z'ε|Z \right) \right)=\]
\[E\left( n^{-2}Z'Var\left( ε|Z \right)Z \right)+Var\left( 0 \right)=n^{-2}σ^2E\left( Z'Z \right)=n^{-1}σ^2Σ_{zz'}→0\]
- d.
\[plim\left( b_{IV} \right)=plim\left( β+{\left( Z'X \right)}^{-1}Z'ε \right)=β+plim\left( {\left( Z'X \right)}^{-1}Z'ε \right)=\]
\[β+plim\left( {\left( n^{-1}Z'X \right)}^{-1} \right)plim\left( n^{-1}Z'ε \right)=\]
\[β+{\left( plim\left( n^{-1}Z'X \right) \right)}^{-1}plim\left( n^{-1}Z'ε \right)=β+Σ_{zx'}^{-1}⋅0=β\]
e.
\[\sqrt{n}\left( b_{IV}-β \right)=\sqrt{n}{\left( Z'X \right)}^{-1}Z'ε={\left( n^{-1}Z'X \right)}^{-1}\left( \frac{1}{\sqrt{n}}Z'ε \right)\]
As \(n→∞\) , \({\left( n^{-1}Z'X \right)}^{-1}→Σ_{zx'}^{-1}\) . Since
\[Var\left( \frac{1}{\sqrt{n}}Z'ε \right)=E\left( Var\left( n^{-1/2}Z'ε|Z \right) \right)=E\left( n^{-1}Z'Var\left( ε|Z \right)Z \right)=\]
\[=n^{-1}σ^2E\left( Z'Z \right)=σ^2Σ_{zz'}\]
neither goes to zero, nor explodes,
\[ \frac{1}{\sqrt{n}}Z'ε→N\left( 0,σ^2Σ_{zz'} \right)\]
As \(n→∞\) , \(\sqrt{n}\left( b_{IV}-β \right)\) becomes
\[Σ_{zx'}^{-1}⋅N\left( 0,σ^2Σ_{zz'} \right)\]
The transpose of \(Σ_{zx'}^{-1}\) is
\[{\left( Σ_{zx'}^{-1} \right)}'={\left( Σ'_{zx'} \right)}^{-1}={\left( E{\left( z_ix'_i \right)}' \right)}^{-1}={\left( E\left( \left( z_ix'_i \right)' \right) \right)}^{-1}={\left( E\left( x_iz'_i \right) \right)}^{-1}=Σ_{xz'}^{-1}\]
if we define \(Σ_{xz'}=E\left( x_iz'_i \right)\) . Therefore,
\[\sqrt{n}\left( b_{IV}-β \right)→N\left( 0,σ^2Σ_{zx'}^{-1}Σ_{zz'}Σ_{xz'}^{-1} \right)\]
We have (see matrix problem)
\[Σ_{zx'}^{-1}Σ_{zz'}Σ_{xz'}^{-1}={\left( Σ_{xz'}Σ_{zz'}^{-1}Σ_{zx'} \right)}^{-1}\]
and
\[\sqrt{n}\left( b_{IV}-β \right)→N\left( 0,σ^2{\left( Σ_{xz'}Σ_{zz'}^{-1}Σ_{zx'} \right)}^{-1} \right)\]
f. Since \(Σ_{xz'}=E\left( x_iz'_i \right)\) , we can consistently estimate \(Σ_{xz'}\) using
\[ \frac{1}{n}\sum_{i=1}^{n}{ x_iz'_i }= \frac{1}{n}X'Z\]
Similarly, \(Σ_{zz'}^{-1}\) can be consistently estimated using \({\left( n^{-1}Z'Z \right)}^{-1}\) and \(Σ_{zx'}\) can be consistently estimated using \(n^{-1}Z'X\) . Therefore, \(Σ_{xz'}Σ_{zz'}^{-1}Σ_{zx'}\) can be consistently estimated using
\[n^{-1}X'Z{\left( n^{-1}Z'Z \right)}^{-1}n^{-1}Z'X=n^{-1}X'Z{\left( Z'Z \right)}^{-1}Z'X=n^{-1}X'P_ZX\]
Therefore,
\[s^2{\left( n^{-1}X'P_ZX \right)}^{-1}=ns^2{\left( X'P_ZX \right)}^{-1}\]
Is a consistent estimator of \(Var\left( \sqrt{n}\left( b_{IV}-β \right) \right)=nVar\left( b_{IV} \right)\) which is why \(s^2{\left( X'P_ZX \right)}^{-1}\) makes sense as an estimator of \(Var\left( b_{IV} \right)\) .