Understanding conditional expectations

Problem

Students at an unknown department in Scandinavia took two courses. First, they did a course in mathematics and, following this course, a course in econometrics. Denote by \(x_i\) the result on the mathematics courses for student \(i\) and \(y_i\) the result on the econometrics course for student \(i\) where \(i=1, \ldots ,n\) ( \(n\) students).

  1. Would you expect \(x_i\) to be independent of \(y_i\) ?
  2. Would you expect \(x_i\) to be independent of \(x_j\) if \(i≠j\) ?
  3. Explain why it makes sense to treat \(\left( x_1,y_1 \right), \ldots ,\left( x_n,y_n \right)\) as a random sample.
  4. Suppose that \(E\left( y_i \mid x_i \right)=a+bx_i\) where \(a\) and \(b\) are constants. Explain in words what this means.
  5. If \(E\left( y_i \mid x_i \right)=a+bx_i\) , which sign would you expect for \(b\) ?
  6. If you are told that \(a=10\) and \(b=0.9\) , what is the expected result for a student that scored 70 points in the math course?
  7. If it turned out to be the case that \(x_i\) was independent of \(y_i\) then what would be the values of \(a\) and \(b\) ?

Solution

  1. I would not. I would expect students who did well on the math course to do better on the econometrics course. That is, I would expect \(x_i\) and \(y_i\) to be positively correlated.
  2. I would. These are results on the math course for two different random students and there is no reason for them to be related.
  3. I would expect \(\left( x_i,y_i \right)\) to be independent of \(\left( x_j,y_j \right)\) if \(y≠j\) for the same reason as in b., these are results for two different students. Also, I would expect \(\left( x_i,y_i \right)\) and \(\left( x_j,y_j \right)\) to be drawn from the same distribution. Why? Well, the opposite would be to assume that \(\left( x_i,y_i \right)\) is drawn from one distribution and \(\left( x_j,y_j \right)\) from another, different distribution. But that would not make much sense. Student \(i\) and student \(j\) are two different students and I have no additional information about them. It would be very strange, for example, to expect student 1 to do better on both exams than student 2. Thus, it makes sense to treat \(\left( x_1,y_1 \right), \ldots ,\left( x_n,y_n \right)\) as independent and identically distributed, that is, as a random sample.
  4. First, the unconditional expectation, \(E\left( y_i \right)\) is the same for all students (random sample). This is simply the expected result on the econometrics exam of a random student without knowing anything about the student. However, the conditional expectation \(E\left( y_i \mid x_i \right)\) is different. \(E\left( y_i \mid x_i \right)\) is the result we expect on the econometrics final for a student that scored \(x_i\) points on the math exam. If \(E\left( y_i \mid x_i \right)=a+bx_i\) then this conditional expectation is simply a linear function of the math score.
  5. I would expect \(b\) to be positive. That is, I would expect \(E\left( y_i \mid x_i \right)\) to increase with \(x_i\) , the better a student did on the math exam, the better I expect her to do on the econometrics exam.
  6. \(E\left( y_i \mid x_i \right)=10+0.9⋅70=73\) . The expected value of \(y_i\) given \(x_i\) is 73. This, of course, does not mean that the student will get precisely 73 points.
  7. If \(x_i\) and \(y_i\) are independent, then knowing the value of \(x_i\) will not change what we expect for \(y_i\) . That is, if \(x_i\) and \(y_i\) are independent, then \(E\left( y_i \mid x_i \right)=E\left( y_i \right)\) . Therefore, \(b=0\) and \(a=E(y_i)\) .