Cheatsheet 05 - Single Variable Optimization
Mathematics for Master Students
The Optimization Recipe
To find the optima of a smooth function \(f\), follow four steps:
- Differentiate: compute \(f'(x)\).
- Solve the FOC \(f'(x) = 0\): the roots are the stationary points (the candidates). Factor \(f'\) where possible; its roots hand them to you.
- Classify each candidate with the SOC \(f''(x^*)\) (table below).
- Decide global: on a closed interval, also test the endpoints; on an unbounded domain, check the behaviour as \(x \to \pm\infty\).
First-order condition (FOC): every smooth interior extremum satisfies \[f'(x^*) = 0.\] It is necessary but not sufficient: a flat tangent flags a candidate, nothing more.
Classifying: the Second-Order Condition
At a stationary point \(x^*\) (where \(f'(x^*) = 0\)), curvature decides the type:
| \(f''(x^*)\) | Curvature | \(x^*\) is a |
|---|---|---|
| \(> 0\) | convex (bends up) | local minimum |
| \(< 0\) | concave (bends down) | local maximum |
| \(= 0\) | flat | inconclusive: test fails |
- Mnemonic: positive \(f''\) holds water → minimum; negative \(f''\) spills it → maximum.
- When \(f'' = 0\), fall back to the first-derivative test: if \(f'\) changes \(+ \to -\) across \(x^*\) it is a maximum; \(- \to +\) a minimum; same sign on both sides → neither (a saddle).
\(f'(x^*) = 0\) alone never proves an extremum. For \(f(x) = x^3\) at \(x = 0\): \(f'(0) = 0\) and \(f''(0) = 0\), yet the graph keeps rising: a saddle point, neither peak nor valley. Always finish with the SOC (or the first-derivative test).
Convexity and Inflection Points
Read curvature from \(f''\) over a whole interval, not just at a point:
| Condition on an interval | Shape |
|---|---|
| \(f''(x) \geq 0\) | convex (bends up) |
| \(f''(x) \leq 0\) | concave (bends down) |
- Where \(f''\) changes sign, the bend switches: an inflection point.
- An inflection point is not an extremum: \(f'\) need not be zero there.
- Example: \(f(x) = x^3 - 6x^2 + 5\) has \(f''(x) = 6(x - 2)\), concave for \(x < 2\), convex for \(x > 2\), inflection at \(x = 2\).
Local vs Global & Closed Intervals
- A local optimum is best only in its neighbourhood; a global optimum is best on the whole domain.
- Extreme Value Theorem: a continuous function on a closed interval \([a, b]\) always attains a global max and a global min.
- On an unbounded domain a global optimum may fail to exist.
Closed-interval recipe. To optimize \(f\) on \([a, b]\):
- Find the stationary points in the open interval \((a, b)\).
- Evaluate \(f\) at each of them.
- Evaluate \(f\) at both endpoints, \(f(a)\) and \(f(b)\).
- The largest value is the global max; the smallest is the global min.
The global optimum often sits at a boundary. In business, quantities live in \([0, \text{capacity}]\): if the unconstrained optimum lies past the capacity and profit is still rising at the edge, the binding limit itself is the optimum.
Profit: Marginal Revenue = Marginal Cost
Profit is revenue minus cost, \(\;\pi(q) = R(q) - C(q)\). The FOC \(\pi'(q) = 0\) becomes:
\[\pi'(q) = R'(q) - C'(q) = 0 \quad \Longleftrightarrow \quad R'(q) = C'(q)\]
- In words: produce while the next unit earns more than it costs; stop where marginal revenue meets marginal cost.
- Confirm a maximum with the SOC: \(\pi''(q) < 0\) (the profit curve is concave).
- Fixed costs shift profit but vanish under differentiation: they never move the optimal quantity.
Economic Order Quantity (EOQ)
Balance ordering cost against holding cost. With order size \(q\), the annual total is
\[T(q) = \frac{D}{q}\,K + \frac{q}{2}\,h \qquad \Longrightarrow \qquad \boxed{\,q^* = \sqrt{\dfrac{2DK}{h}}\,}\]
| Symbol | Meaning |
|---|---|
| \(D\) | annual demand |
| \(K\) | fixed cost per order |
| \(h\) | holding cost per unit per year |
| \(q\) | order size (the decision) |
- The SOC \(T''(q) = \dfrac{2DK}{q^3} > 0\) confirms \(q^*\) is a minimum.
- At \(q^*\) the ordering cost and holding cost are equal: that balance is the whole point.
- \(q^*\) rises with demand \(D\) and order cost \(K\), and falls as holding cost \(h\) increases. Since \(q^* \propto \sqrt{D}\), quadrupling demand only doubles the order size.
Optimization Checklist
- Did you set \(f'(x) = 0\) and find all stationary points?
- Did you classify each with \(f''\) (or the first-derivative test if \(f'' = 0\))?
- On a closed interval, did you also test both endpoints?
- For a business problem: is the answer inside the feasible range \([0, \text{capacity}]\)?
- Did you state the optimum value, not just the optimal \(x\) or \(q\)?