Problem: Prove that ESS + RSS = TSS
Problem
This problem is more difficult. You may want to skip this problem for now. We will show that
\[\sum_{i=1}^{n}{ {\left( {\hat{y}}_i-\bar{y} \right)}^2 }+\sum_{i=1}^{n}{ e_i^2 }=\sum_{i=1}^{n}{ {\left( y_i-\bar{y} \right)}^2 }\]
a) Show that
\[\sum_{i=1}^{n}{ {\left( {\hat{y}}_i-\bar{y} \right)}^2 }=\sum_{i=1}^{n}{ {\hat{y}}_i^2 }-n{\bar{y}}^2\]
b) Show that
\[\sum_{i=1}^{n}{ {\hat{y}}_i^2 }=\sum_{i=1}^{n}{ {\hat{y}}_i(y_i-e_i) }=\sum_{i=1}^{n}{ {\hat{y}}_iy_i }\]
c) Show that
\[\sum_{i=1}^{n}{ {\hat{y}}_iy_i }=\sum_{i=1}^{n}{ y_i^2 }-\sum_{i=1}^{n}{ e_iy_i }=\sum_{i=1}^{n}{ y_i^2 }-\sum_{i=1}^{n}{ e_i^2 }\]
d) Now combine everything and show that \(ESS+RSS=TSS\) .
Solution
a)
See Problem: Important sum rules for sample moments
b)
\[\sum_{i=1}^{n}{ {\hat{y}}_i(y_i-e_i) }=\sum_{i=1}^{n}{ {\hat{y}}_iy_i }-\sum_{i=1}^{n}{ {\hat{y}}_ie_i }\]
The last sum is zero, see Some OLS results
c) \({\hat{y}}_i=y_i-e_i\) and
\[\sum_{i=1}^{n}{ {\hat{y}}_iy_i }=\sum_{i=1}^{n}{ \left( y_i-e_i \right)y_i }=\sum_{i=1}^{n}{ y_i^2 }-\sum_{i=1}^{n}{ e_iy_i }\]
\(y_i=b_1+b_2x_i+e_i\) so
\[\sum_{i=1}^{n}{ e_iy_i }=\sum_{i=1}^{n}{ e_i\left( b_1+b_2x_i+e_i \right) }=\sum_{i=1}^{n}{ e_ib_1 }+\sum_{i=1}^{n}{ e_ib_2x_i }+\sum_{i=1}^{n}{ e_i^2 }\]
We have
\[\sum_{i=1}^{n}{ e_ib_1 }=b_1\sum_{i=1}^{n}{ e_i }=b_1×0=0\]
and
\[\sum_{i=1}^{n}{ e_ib_2x_i }=b_2\sum_{i=1}^{n}{ e_ix_i }=b_2×0=0\]
Therefore,
\[\sum_{i=1}^{n}{ {\hat{y}}_iy_i }=\sum_{i=1}^{n}{ y_i^2 }-\sum_{i=1}^{n}{ e_iy_i }=\sum_{i=1}^{n}{ y_i^2 }-\sum_{i=1}^{n}{ e_i^2 }\]
Yep, we get \(ESS+RSS=TSS\) .