Words with a dotted underline are glossary terms: click one for its definition. G lists every term in the unit, N shows the notes. Optional extra: interactive figures, linked from the slides they support by a small ▸ chip; skip them freely.
section 1
Splitting
A hard decision, then an easy one.
12 · splitting
The value function of the easy part
$$\begin{aligned} & \min_{y \in Y}\ f^\top y + Q(y) \\ & Q(y) = \min_{x \ge 0} \{\, c^\top x : Wx \ge h - Ty \,\} \end{aligned}$$
$y$: the complicating variables (integer, few). For fixed $y$ the inner problem, the recourse, is an LP, often separable into blocks. Its value is the value function $Q(y)$.
The dual polyhedron $\Pi$ doesn't depend on $y$. So $Q$ is the maximum of finitely many affine functions (one per vertex of $\Pi$): convex and piecewise linear. If $Q(y) = \infty$, an extreme ray of $\Pi$ certifies it.
Dual feasibility: $v_j - \alpha_i \le c_{ij}$, $0 \le v_j \le p$, $\alpha \ge 0$. None of it involves $y$.
12 · splitting · example
A two-scenario example
facility
open cost
capacity
cost per unit served
A
10
5
1
B
6
5
3
One customer. Demand is 4 or 8, each with probability $\tfrac12$ (scenarios 1 and 2). Unmet demand costs 10 per unit.
First decide which facilities to open ($y_A, y_B$); then, once demand is known, serve it as cheaply as the open capacity allows: A first, then B, then pay the penalty.
the value function, for every choice
open
$Q_1$ (demand 4)
$Q_2$ (demand 8)
total
none
40
80
$0 + 60 = 60$
A
4
$5 + 30 = 35$
$10 + 19.5 = 29.5$
B
12
$15 + 30 = 45$
$6 + 28.5 = 34.5$
A, B
4
$5 + 9 = 14$
$16 + 9 = 25$
"Total" is opening cost plus the average recourse. Open both: 25.
With 2 facilities this table is the whole problem. With 30 it has $2^{30}$ rows, and Benders builds only the part it needs.
Solve at $\bar y$ and take an optimal dual $\bar\pi$. For every $y$:
$$\theta \ge \bar\pi^\top (h - Ty)$$
It's tight at $\bar y$, and valid elsewhere by weak duality.
feasibility cut
If the LP at $\bar y$ is infeasible, a ray $\bar r$ of $\Pi$ has $\bar r^\top (h - T\bar y) > 0$. Every feasible $y$ needs
$$\bar r^\top (h - Ty) \le 0$$
In practice: solve the phase-one LP $\min \sum z$, whose duals are bounded ($v \le 1$). A positive value gives the ray.
Lab step 2's tests, on every $y \in \{0, 1\}^F$:
each optimality cut equals $Q_s(\bar y)$ at $\bar y$ and stays below $Q_s(y)$, computed independently by scipy, at every $y$;
each feasibility cut excludes its $\bar y$ and keeps every $y$ at which the scenario is feasible.
complete recourse
A penalty for unmet demand makes every $y$ feasible, so feasibility cuts disappear. That's common modelling practice. Without it, $y = 0$ is the master's first answer and every scenario returns a feasibility cut.
12 · splitting · example
Cuts from duals, with numbers
scenario 2 at "open both"
Demand 8: A serves 5, B serves 3. The duals price what one more unit would change:
$v = 3$: one more unit of demand would come from B, at cost 3.
$\alpha_A = 2$: one more unit of A's capacity would replace a B unit, saving $3 - 1 = 2$.
Check it: at "both" it gives 14 $= Q_2$, tight. At "A only", 14 $\le 35$; at "B only", 24 $\le 45$; at "none", 24 $\le 80$. Valid everywhere, as promised.
scenario 1 at "open none"
Everything unmet: $v = 10$, and each unit of capacity would save $10 - 1 = 9$ at A, $10 - 3 = 7$ at B.
Cut $\theta_1 \ge 40 - 45\,y_A - 35\,y_B$: exact at "none" (40), but weak elsewhere: $-5$ at "A only", $5$ at "B only", $-40$ at "both", where the true values are 4, 12, 4.
without the penalty: a feasibility cut
Drop the unmet-demand option. At "A only", scenario 2 needs 8 but has capacity 5: infeasible. The phase-one LP gives $v = 1$, $\alpha_A = \alpha_B = 1$:
$8 - 5y_A - 5y_B \le 0$: "open at least 8 units of capacity". It removes "A only", "B only" and "none" at once.
12 · splitting
The loop, and why it ends
The master problem over $y$ and $\theta$ with the cuts so far:
$$\min f^\top y + \theta \quad \text{s.t. cuts},\ y \in Y$$
Solve the master: $\text{LB} = $ its value, giving $\bar y$.
Solve the subproblem(s) at $\bar y$: add a feasibility or optimality cut.
If feasible, $\text{UB} = \min(\text{UB},\ f^\top \bar y + Q(\bar y))$.
Stop when $\text{UB} - \text{LB} \le \varepsilon$ (the Benders bounds meet).
It always ends: there are only finitely many dual vertices, hence finitely many cuts (finite convergence).
The master is a MIP that gains cuts every iteration and is re-solved each time. That re-solve is the method's cost centre, and section 4 is mostly about avoiding it.
For an LP, Benders on the primal is Dantzig–Wolfe on its dual. The transpose of the second matrix has the shape of the first: linking variables become linking constraints, and generated rows become generated columns.
where unit 03 sits
DW: reduced costs $c - \pi^\top A_j$ decide which column enters. Benders: dual solutions $\pi$ of the subproblem are the cut coefficients. In both, the same LP duality moves information between master and subproblem, in opposite directions.
section 3
Stochastic programming
Decide now, adapt later, pay the expectation.
12 · stochastic programming
Two-stage recourse and the L-shaped method
$$\min_{y \in Y}\ f^\top y + \sum_{s=1}^S p_s\, Q_s(y)$$
A two-stage stochastic program. The deterministic equivalent (extensive form) copies the second-stage variables once per scenario: size $\times S$, one MIP. Given $y$, the scenarios decouple into $S$ independent LPs.
L-shaped method (Van Slyke & Wets 1969): Benders with one aggregated cut $\theta \ge \sum_s p_s(\cdot)$.
Multi-cut (Birge & Louveaux 1988): one $\theta_s$ and one cut per scenario. It passes more information, so fewer iterations, but the master is bigger.
The scenario LPs are embarrassingly parallel.
where scenarios come from
Sample average approximation (Kleywegt, Shapiro & Homem-de-Mello 2002): draw $S$ samples, solve, and repeat with fresh samples to estimate the optimality gap statistically. More scenarios means a better estimate and a bigger model, which is exactly the regime where decomposition should win.
The expected value of perfect information and the value of the stochastic solution (EVPI, VSS) measure whether solving the stochastic model was worth it. Birge & Louveaux ch. 4.
Tight capacities (1.6× demand): Benders wins from $S = 5$. A handful of cuts settles which facilities must open.
Loose capacities (3×): many near-equivalent facility sets, 20–60 iterations. The extensive form wins until single-cut overtakes it between $S = 50$ and $100$.
Multi-cut always uses fewer iterations, and pays for each with a master that has $S\times$ more cuts.
The crossover depends on iterations × cost per iteration, and the instance family controls the first factor. Report it as a plot per family, never as one number.
section 4
Modern Benders
Stop re-solving the master.
12 · modern
One tree, and better cuts
branch-and-Benders-cut
Solve the master once, with a solver callback. Whenever branch-and-bound finds an integer (or even fractional) $\bar y$, solve the subproblems and add violated cuts as lazy constraints. No re-solving, and one search tree. Fischetti, Ljubić & Sinnl (2017) solved facility location instances with thousands of customers this way.
LP first (McDaniel & Devine 1977)
Generate cuts at the master's LP relaxation first. They're cheap, and valid for integer $y$. Lab then, loose, S = 25: integer iterations 21 → 13, but no wall-time gain in this re-solving implementation.
Pareto-optimal cuts (Magnanti & Wong 1981)
When the subproblem's dual is degenerate, many cuts are tight at $\bar y$. Choose the one highest at a core point $y^0$ in the relative interior of $\mathrm{conv}(Y)$, which needs one extra LP. Lab: on this capacitated family, the same iteration count and slower. Degeneracy is rare here, and the classic gains are on uncapacitated location.
beyond LP subproblems
Combinatorial Benders (Codato & Fischetti 2006): no-good cuts from infeasible subsets. Logic-based Benders (Hooker & Ottosson 2003): any subproblem, CP or SAT included, as long as it returns an inference about the master's choice. It returns in unit 21.
12 · lab
Lab 12 — what you build
about 2½ hours · uv run co lab 12
The recourse LP and its duals.
Optimality and feasibility cuts, checked against every $y$.
The master and the loop: multi-cut and single-cut.
Magnanti–Wong Pareto-optimal cuts.
LP-first warm start.
uv run co test 12
uv run co then 12 # extensive form vs Benders as the scenario count grows, tight and loose
12 · checkpoint
Done when
checkpoint
All five lab steps are ✓.
You can draw Dantzig–Wolfe and Benders on one page as duals of each other (rows vs columns, master vs subproblem) and point at where unit 03 sits in both.
You can explain, from iteration counts and per-iteration cost, why the crossover moves with the instance family.
Reading
Benders, Numerische Mathematik 4 (1962).
Rahmaniani et al., EJOR 259 (2017), survey.
Magnanti & Wong, Operations Research 29 (1981).
Birge & Louveaux, Intro. to Stochastic Programming, ch. 5.
Next
Part III starts: 13 matroids · Benders returns in logic-based form in 21, lazy clause generation.