Problem: Deriving the OLS formula for a model with no intercept
Problem
Given an OLS trendline with no intercept \(y=b_2x\) . Let
\[f\left( b_2 \right)=\sum_{i=1}^{n}{ e_i^2 }=\sum_{i=1}^{n}{ {\left( y_i-b_2x_i \right)}^2 }\]
a) Show that
\[ \frac{df}{db_2}=-2\sum_{i=1}^{n}{ x_i\left( y_i-b_2x_i \right) }\]
b) Show that the stationary point is the no-intercept OLS formula.
c) Show that
\[ \frac{d^2f}{db_2^2}=2\sum_{i=1}^{n}{ x_i^2 }>0\]
d) Explain why the stationary point in b is a global minimum point.
Solution
a)
\[ \frac{d}{db_2}{\left( y_i-b_2x_i \right)}^2=-2x_i\left( y_i-b_2x_i \right)\]
b)
\[ \frac{df}{db_2}=0 \textrm{ or } \sum_{i=1}^{n}{ x_i\left( y_i-b_2x_i \right) }=0\]
We have
\[\sum_{i=1}^{n}{ x_iy_i }-b_2\sum_{i=1}^{n}{ x_i^2 }=0\]
or
\[b_2= \frac{\sum_{i=1}^{n}{ x_iy_i }}{\sum_{i=1}^{n}{ x_i^2 }}\]
c)
\[ \frac{d^2}{db_2^2}{\left( y_i-b_2x_i \right)}^2= \frac{d}{db_2}\left( -2x_i\left( y_i-b_2x_i \right) \right)=2x_i^2\]
d)
Second derivative is strictly positive, so RSS is strictly convex. Stationary point must be a global minimum.