Single-variable optimization, results

Summary

First derivative test (necessary conditions for extreme values)

  • \(f:A→B\) and \(x_0∈A\) . If \(f\) is differentiable at \(x_0\) and \(x_0\) is a local interior extreme point then \(x_0\) must be a stationary point.
  • Examples:
    • If \(f(x)=x^2\) (natural domain \(R\) ) then \(f\) is differentiable everywhere and all points are interior points. The extreme point \(x_0=0\) must be a stationary point, \(f'(0)=0\) .
    • If \(f(x)=x\) on \([0,1]\) then \(x_0=0\) is an extreme point. Since \(x_0\) is not an interior point, it need not be a stationary point (and it is not).
    • If \(f(x)=|x|\) then \(x_0=0\) is an interior extreme point but since \(f\) is not differentiable at \(x=0\) it need not be a critical point (and it is not).
  • The opposite is not true. If \(f(x)=x^3\) then \(f\) is differentiable but the interior stationary point \(x=0\) is not a local extreme point (it is a saddle point).

Second derivative test

  • \(f:A→B\) and \(x_0∈A\) . \(f\) is twice differentiable at \(x_0\) and \(x_0\) is a stationary point.
    • If \(f''(x_0)<0\) then \(x_0\) is a strict local maximum point
    • If \(f''(x_0)>0\) then \(x_0\) is a strict local minimum point
    • If \(f''(x_0)=0\) then no conclusion may be drawn.
  • Examples:
    • If \(f(x)=x^2\) then \(x_0=0\) is a stationary point. Since \(f^{''}\left( 0 \right)=2>0\) , it is a strict local minimum point.
    • If \(f(x)=-x^2\) then \(x_0=0\) is a stationary point. Since \(f^{''}\left( 0 \right)=-2<0\) , it is a strict local maximum point.
    • If \(f(x)=x^3\) then \(x_0=0\) is a stationary point. Since \(f^{''}\left( 0 \right)=0\) , the second derivative test is inconclusive. Since \(f\) is strictly increasing, \(x_0=0\) is not a local extreme point. An interior stationary point which is not a local extreme point is called a saddle point .
    • If \(f(x)=x^4\) then \(x_0=0\) is a stationary point. Since \(f^{''}\left( 0 \right)=0\) , the second derivative test is inconclusive. Since \(f\left( 0 \right)=0\) and \(f\left( x \right)>0\) if \(x≠0\) , \(x_0=0\) is a a strict local minimum point.
    • If \(f\left( x \right)=-x^4\) then \(x_0=0\) is a stationary point. Since \(f^{''}\left( 0 \right)=0\) , the second derivative test is inconclusive. Since \( f\left( 0 \right)=0\) and \(f\left( x \right)<0\) if \(x≠0\) , \(x_0=0\) is a a strict local maximum point.
    • If \(f(x)=1\) then all points are stationary points. Since \(f''(0)=0\) for all \(x\) the second derivative test is inconclusive. In this case, every point is a local minimum and a local maximum point, although not strict.

Extreme value theorem

  • If \(f\) is continuous function and the domain of \(f\) is a closed and bounded interval, \(I=[a,b]\) then there exists a global maximum point \(x_{max}\) in \(I\) and a global minimum point \(x_{min}\) in \(I\) .
  • If \(y_{max}=f\left( x_{max} \right)\) and \(y_{min}=f\left( x_{min} \right)\) then \(y_{min}≤f(x)≤y_{max}\) for all \(x∈I\) .
  • \(x_{max}\) and \(x_{min}\) need not be unique. If \(f(x)=2\) on \(I=\left[ 0,1 \right]\) then every point in \([0,1]\) is a maximum point and a minimum point.
  • \(f\) need not be differentiable for the extreme value theorem to hold.

Concave and convex functions

  • If \(f\) is convex function defined on an interval \(I\) and \(x_0\) is a stationary point, then \(x_0\) is a global minimum point.
  • If \(f\) is concave function defined on an interval \(I\) and \(x_0\) is a stationary point, then \(x_0\) is a global maximum point.
  • If \(f\) is strictly convex / concave then \(x_0\) is a strict global minimum / maximum point.