Fitted values and residuals

Summary

• We try to find parameters \(b_1, \ldots ,b_k\) such that \(b_1x_{i,1}+b_2x_{i,2}+ \ldots +b_kx_{i,k}\) is “close to” \(y_i\) for all \(i=1, \ldots ,n\) .
• Definition of fitted values \({\hat{y}}_i\) for \(i=1, \ldots ,n\) :

\[{\hat{y}}_i=b_1x_{i,1}+b_2x_{i,2}+ \ldots +b_kx_{i,k}\]

• We try to find \(b_1, \ldots ,b_k\) such that \({\hat{y}}_i\) is “close to” \(y_i\) for all \(i=1, \ldots ,n\) . (a more formal definition of fitted values later)
• Definition: The \(b\) vector:

\[b=\begin{bmatrix}b_1 \\ ⋮ \\ b_k\end{bmatrix}\]

• is an arbitrary \(k×1\) vector of parameters that determine the fitted values.
• fitted values \({\hat{y}}_i\) in vector form for \(i=1, \ldots ,n\) :

\[{\hat{y}}_i=x'_ib\]

• Definition of the vector of fitted values \(\hat{y}\) :

\[\hat{y}=\begin{bmatrix}{\hat{y}}_1 \\ ⋮ \\ {\hat{y}}_n\end{bmatrix}\]

• \(\hat{y}\) is \(n×1\) and we want this to be close to the \(n×1\) vector \(y\) .
• Result: vector of fitted

\[\hat{y}=Xb\]

• Definition of residuals \(e_i\) for \(i=1, \ldots ,n\) :

\[e_i=y_i-{\hat{y}}_i\]

• We want to pick \(b\) such that \(e_i\) are as small as possible. Alternatively:

\[e_i=y_i-b_1-b_2x_{i,2}- \ldots -b_kx_{i,k}\]

• Alternatively:

\[e_i=y_i-x'_ib\]

• Definition of the vector of residuals:

\[e=\begin{bmatrix}e_1 \\ ⋮ \\ e_n\end{bmatrix}\]

• \(e\) is \(n×1\) .
• Vector of residuals in vector form:

\[e=y-\hat{y}=y-Xb\]