Prove the OLS statistical formula

Problem

Prove The OLS statistical formula. \(b={\left( X'X \right)}^{-1}X'y\) and \(y=Xβ+ε\) . Show that

\[b=β+{\left( X'X \right)}^{-1}X'ε\]

Solution

\[b={\left( X'X \right)}^{-1}X'y={\left( X'X \right)}^{-1}X'\left( Xβ+ε \right)={\left( X'X \right)}^{-1}X'Xβ+{\left( X'X \right)}^{-1}X'ε\]

We can write the first term more suggestively as \({\left( X'X \right)}^{-1}\left( X'X \right)β\) . Since for any square invertible matrix \(A\) we have \(A^{-1}A=I\) it follows that

\[{\left( X'X \right)}^{-1}\left( X'X \right)β=Iβ=β\]