LPs with more columns than atoms, solved by asking a pricing problem one question at a time. The duals of unit 03 become the objective of a combinatorial subproblem.
needs 08, 16~2 h slides + 3 h lablab: uv run co lab 10
10 · overview
The route
1 · too many columns
Cutting stock two ways; reduced costs; the pricing problem; the loop.
2 · Dantzig–Wolfe
Block-angular structure, convexification, and the bound you get.
3 · bounds and stability
Farley and Lagrangian bounds; dual degeneracy; smoothing.
4 · branch-and-price
Why naive branching breaks pricing; Ryan–Foster; routes and ESPPRC.
getting around
Words with a dotted underline are glossary terms: click one for its definition. G lists every term in the unit, N shows the notes. Optional extra: interactive figures, linked from the slides they support by a small ▸ chip; skip them freely.
section 1
Too many columns
A formulation that is strong because it is enormous.
10 · too many columns
Cutting stock, two ways
Cutting stock: cut rolls of width $W$ into $d_i$ pieces of width $w_i$, using as few rolls as possible.
Kantorovich 1939 · compact
$y_k$: roll $k$ used. $x_{ik}$: copies of item $i$ from roll $k$.
LP bound $= \sum_i w_i d_i / W$: the trivial area bound. Rolls are interchangeable, giving $K!$ symmetric copies of every solution.
Gilmore–Gomory 1961 · extended
One variable per pattern $a \in \mathbb Z_{\ge 0}^m$ with $w^\top a \le W$. Rolls of 10, pieces 3, 4, 6: $(3,0,0)$ is three 3s, $(2,1,0)$ is two 3s and a 4.
Dantzig's rule picks the most negative. With columns defined implicitly as the feasible set of a combinatorial problem, "find the most negative" is an optimisation problem:
$$\min_{a \in \mathcal A}\ c(a) - y^\top a$$
Cutting stock: $c(a) = 1$, so maximise $y^\top a$ over patterns, which is a knapsack with the duals as values: the pricing problem of the restricted master.
the checkpoint, in unit 03's words
The restricted master's optimal duals $y$ are feasible for its dual, over the columns it contains. They are optimal for the full dual iff no column violates $y^\top a \le c(a)$. Pricing is the separation problem for the dual. Its objective coefficients are the duals because the dual constraint for column $a$ has coefficients $a$ and right-hand side $c(a)$.
If pricing returns $\bar c \ge 0$, the master's solution is optimal for the full LP: complementary slackness holds for every column, present or not.
10 · too many columns
The loop
The master grows by one column (or a few) per iteration. The pricing problem never sees the master's size.
The master value never increases: an added column can only help.
Finite: each column enters at most once (with care over degeneracy), and there are finitely many.
Warm starts: add the column to HiGHS's model and re-solve from the previous basis. It's one primal simplex pivot in the common case.
Multiple columns per iteration: return every negative reduced-cost column pricing found (lab step 6 returns up to 10).
Heuristic pricing is allowed until the last iteration. Only an exact "nothing negative" proves optimality.
Lab step 3's test checks the value against the LP over all patterns: 25 instances, exact agreement.
10 · too many columns · example
The loop, on a small instance
Rolls $W = 10$. Pieces of width $3, 4, 6$, demands $5, 3, 2$. Start from one pattern per piece: $(3,0,0)$, $(0,2,0)$, $(0,0,1)$.
The duals price a 3-piece at $\tfrac13$ of a roll, a 4 at $\tfrac12$, a 6 at a full roll. The pattern "a 4 and a 6" fits ($4 + 6 = 10$) and is worth $\tfrac12 + 1 = 1.5$ rolls of value while costing 1 roll: reduced cost $1 - 1.5 = -0.5$. It enters.
the end
At iteration 2 no pattern is worth more than 1, so the LP over all patterns is solved, at value 4. It happens to be integral: one roll of three 3s, two rolls of 4+6, one roll of 3+3+4. Four rolls, and the LP proves no cutting uses fewer.
Convexification: replace each $X_k$ by $\mathrm{conv}(X_k)$, written as a convex combination of its extreme points $\{x_k^p\}$ (Minkowski–Weyl, unit 01):
The Dantzig–Wolfemaster: linking constraints plus one convexity constraint per block, over the $\lambda$ columns. Pricing for block $k$, with linking duals $y$ and convexity dual $\mu_k$:
$$\min_{x \in X_k}\ (c_k - D_k^\top y)^\top x - \mu_k$$
the bound (Geoffrion 1974)
$z_{DW} = \min\{c^\top x : Dx \ge d,\ x_k \in \mathrm{conv}(X_k)\} \ge z_{LP}$, with equality whenever each block's LP relaxation is already integral (the integrality property). This is the Lagrangian dual bound of unit 11.
10 · Dantzig–Wolfe
A hard pricing problem is a feature
why it is fine
The hardness buys the bound. Geoffrion says a pricer with the integrality property gives no improvement.
Pricers are small: one roll, one vehicle, one crew member. Exponential in the size of one block is often cheap.
Many are weakly NP-hard, like knapsack, which is pseudo-polynomial (unit 16). Or they have strong DP structure, like labelling for paths with resources.
Heuristic pricing does most of the work, and exact pricing only certifies the end.
when it is not fine
ESPPRC, pricing for vehicle routing, is strongly NP-hard (Dror 1994). Lab then: roughly ×10–30 pricing time per four more customers, from 0.09 s at $n = 12$ to 27.8 s at $n = 20$.
So production codes relax the pricing problem. They allow non-elementary routes, or ng-routes (Baldacci, Mingozzi & Roberti 2011), which forbid only cycles within small neighbourhoods. That gives a weaker bound with a far cheaper pricer, and cuts recover the strength.
section 3
Bounds and stability
What you know before the end, and why the end takes so long.
10 · bounds and stability
A lower bound at every iteration
Lagrangian bound (Lasdon 1970)
If some optimal solution has $\sum_p x_p \le \kappa$, then
For any $y \ge 0$ with $v = \max_a y^\top a$, $\ y / v$ is dual feasible, so
$$z_{LP} \ge d^\top y \,/\, v$$
At the master duals, that is $z_{RMP} / v$ (lab step 2). On the small instance: 3.44, 3.57, 4.
why: notes (N)
Lab then: early termination when $\lceil\text{bound}\rceil = \lceil z_{RMP}\rceil$, since the integer bound is then known. On the small instance, iteration 1 already has $\lceil 3.57 \rceil = 4$ but $\lceil 4.17 \rceil = 5$, so it can't stop yet.
items
iterations
could stop at
20
56
47
40
139
132
80
203
192
A 5–16% saving here. In branch-and-price, the same bound lets a node be pruned long before its LP converges, and that's where it pays.
The restricted master is highly degenerate: many dual optima. Its duals jump between extreme points of the dual face, far from the full dual optimum. Pricing at a wild dual returns a column that is useless later. The tail of the iteration count is mostly this oscillation.
stabilisation, three ways
Boxstep (Marsten, Hogan & Blankenship 1975): constrain $y$ to a box around a centre.
Penalties (du Merle et al. 1999): penalise leaving the box, with surplus columns in the primal.
Smoothing (Wentges 1997): price at $\alpha \hat y + (1 - \alpha) y$ around a stability centre $\hat y$, and fall back on a mis-price. No change to the master at all.
Lab then, widths 1–20% of W (many items per pattern), iterations:
items
α = 0
0.5
0.8
0.9
25
64
58
58
60
50
121
106
102
108
Fewer iterations (−16% at 50 items), but no gain in seconds, since mis-prices cost a second pricing call. Smoothing pays when pricing is cheap relative to the master, or when oscillation is severe. Neither holds strongly here.
Down branch $x_p \le \lfloor x_p^* \rfloor$, typically $x_p = 0$. The pricing problem doesn't know pattern $p$ is forbidden. Its optimum is that same pattern, so it regenerates it forever. Fixing that needs the $k$-th best pattern, with $k$ growing with depth.
Also unbalanced: $x_p = 0$ removes one of exponentially many columns and barely moves the bound.
Price-and-branch (the lab's restricted IP) sidesteps this with no guarantee. Lab then: exact for cutting stock, and a 10% gap for CVRP at $n = 12$ and $20$.
branch on the original structure
Ryan–Foster 1981 (set partitioning, bin packing): pick items $i, j$ that are fractionally together. One branch says always together (merge them in pricing), the other never together (a conflict constraint in pricing). The pricer changes shape slightly but stays a knapsack-like problem.
VRP: branch on arcs or edges of the original graph (forbid, or force, arc $u \to v$) or on the number of vehicles. Both are easy to impose in labelling.
Vanderbeck (2000, 2011) gives general branching schemes for identical blocks, where aggregation hides which block a column belongs to.
10 · branch-and-price · example
Ryan–Foster, on three items
Bin packing: three items of size 4, bins of size 10. Any two share a bin, all three don't.
the root LP
Patterns $\{1,2\}$, $\{1,3\}$, $\{2,3\}$ at $\tfrac12$ each. Every item is covered exactly once, using 1.5 bins. The optimum is 2.
Items 1 and 2 are "together" in $\tfrac12$ a bin: neither 0 nor 1. Branch on that pair.
branch
pricing sees
allowed patterns
LP
together
1 and 2 merged into one item of size 8
$\{1,2\}$, $\{3\}$
2
apart
a conflict edge between 1 and 2
$\{1,3\}$, $\{2,3\}$, $\{1\}$, $\{2\}$, $\{3\}$
2
Both children have LP 2 = optimum, so the tree ends after one branching. Each child's pricer is still a knapsack, with one merged item or one conflict: something pricing can see.
Branching on "$x_{\{1,2\}} \le 0$" instead would leave pricing free to regenerate $\{1,2\}$.
10 · branch-and-price
Vehicle routing: where column generation stops being a toy
Set partitioning over routes $r$ for the CVRP, with $\alpha_{vr} = 1$ if $r$ visits $v$:
Pricing: an elementary depot-to-depot path within capacity, with arc costs $c_{uv} - \pi_v$. Negative cycles are everywhere, which is why elementarity must be enforced.
Labelling (lab step 6): labels (cost, load, visited set) per vertex, extended forward, with dominance pruning. Bidirectional labelling and completion bounds are the next speed-ups.
n
LP bound
pricing s
restricted IP
8
342.0
0.01
342
12
422.0
0.09
470
16
478.2
2.95
483
20
662.0
27.8
738
the state of the art
Branch-price-and-cut with ng-routes, rounded capacity and limited-memory subset-row cuts, bidirectional labelling, and route enumeration. It solves CVRP instances with several hundred customers to optimality (Pessoa et al., VRPSolver 2020).
10 · lab
Lab 10 — what you build
about 3 hours · uv run co lab 10
The restricted master and its duals.
Pricing with your unit-16 knapsack, and the Farley bound.
The column-generation loop, recording bounds.
Integer solutions: rounding, and the restricted master IP.
Wentges smoothing with the mis-price fallback.
ESPPRC labelling and the CVRP set-partitioning bound.
uv run co test 10
CO_MINE=16 uv run co test 10 # price with your own knapsack
uv run co then 10 # compact model vs CG · Farley stopping · smoothing · CVRP
10 · checkpoint
Done when
checkpoint
All six lab steps are ✓.
You can explain, in unit 03's vocabulary and without hand-waving, why the master's dual values are exactly the objective coefficients of the pricing problem.
You can say why branching on $x_p$ breaks pricing and what Ryan–Foster branching does instead.
Reading
Desrosiers & Lübbecke, "A primer in column generation" (2005).
Lübbecke & Desrosiers, Operations Research 53 (2005).
Gilmore & Gomory, Operations Research 9 (1961).
Costa, Contardo & Desaulniers, Transportation Science 53 (2019).
Next
11 Lagrangian relaxation: the same bound, reached by subgradients instead of an LP · 12 Benders: the row-generation twin, for linking variables.