Conditional expected value and variance of linear functions

Problem

\(ε_1, \ldots ,ε_n\) are \(n\) IID random variables. \(ε=\left( ε_1, \ldots ,ε_n \right)\) is an \(n×1\) random vector.

\(x_1, \ldots ,x_n\) are \(n\) \(k×1\) IID random vectors and \(X={\left( x_1, \ldots ,x_n \right)}'\) is \(n×k\) .

Further, \(E\left( ε|X \right)=0\) and \(Var\left( ε|X \right)=σ^2I\) .

  1. Find \(E\left( X'ε|X \right)\)
  2. Find \(E\left( X'ε \right)\) (Hint: law of iterated expectations)
  3. Find \(Var\left( X'ε|X \right)\)
  4. Is it true that \(Var\left( X'ε \right)=σ^2X'X\) ?

Solution

  1. Since we are conditioning on \(X\) , we may treat \(X\) as a matrix of constants and “take it outside”, \(E\left( X'ε|X \right)=X'E\left( ε|X \right)=X'0=0\) where the second “ \(0\) ” is a \(k×1\) vector of zeros ( \(X'ε\) is \(k×1\) ). Note that it is not true that \(E\left( X'ε \right)=X'E\left( ε \right)\) since \(X\) is stochastic.
  2. \(E\left( X'ε \right)=E\left( E\left( X'ε|X \right) \right)=E\left( 0 \right)=0\)
  3. Same point as in a. \(Var\left( X'ε|X \right)=X'Var\left( ε|X \right)X=X'σ^2IX=σ^2X'X\)
  4. No. We cannot do anything with \(Var\left( X'ε \right)\) since both \(X\) and \(ε\) are stochastic. Further, “ \(Var\left( X'ε \right)=σ^2X'X\) ” would make no sense . \(X'ε\) is a \(k×1\) random vector so \(Var\left( X'ε \right)\) must be a \(k×k\) matrix of constants . However, \(σ^2X'X\) is a \(k×k\) matrix of random variables . Because “ \(Var\left( X'ε \right)=σ^2X'X\) ” makes no sense, it is actually used. If you see this expression, it really means “ \(Var\left( X'ε|X \right)=σ^2X'X\) ”. We sometimes get tired of writing “conditionally on \(X\) ” and drop it.