Part V · where the trunks converge · spine

Combinatorial approximation algorithms

Greedy, matchings, spanning trees and sorting, each with a lower bound on the optimum it can compute, and a proof that fits on one slide.

needs 22, 16, 15, 13~1½ h slides + 2½ h lablab: uv run co lab 23

Unit 22 drew the map of what's possible. This unit fills in the easy half of the "best guarantee" column with combinatorial algorithms: no LPs, no SDPs, only structures from Part III. Units 24 and 25 fill in the rest with relaxations.

The curriculum calls Christofides "the moment trunk B pays off". It needs a minimum spanning tree (unit 13) and a minimum-weight perfect matching (unit 15), composed through an Euler circuit.

How to read this deck

Three small instances carry it: five elements where greedy set cover pays 10 against an optimum of 7, with every price shown; the six points of the double-tree figure, run through both TSP algorithms with their lower bounds; and the two seven-job tight families for list scheduling and LPT. All numbers are from the lab's reference solution and brute force. The price lemma and the scheduling bounds are given as ideas on the slides and proved in the notes.

23 · overview

The route

1 · the shape of a guarantee

A computable lower bound on OPT. Greedy set cover and its prices.

2 · packings

Vertex cover by maximal matching; k-center by farthest-first.

3 · metric TSP

Double tree (2), Christofides (3/2), and the 2021 improvement.

4 · scheduling, and the gap to practice

List scheduling, LPT; tight families; ratios measured against CP-SAT.

getting around

Words with a dotted underline are glossary terms: click one for its definition. G lists every term in the unit, N shows the notes. Optional extra: interactive figures, linked from the slides they support by a small ▸ chip; skip them freely.

section 1

The shape of a guarantee

Compare against something you can compute.

23 · guarantees

Every proof in this unit has one shape

OPT is unknown, and computing it is NP-hard. So find a lower bound $\mathrm{LB}$ with

$$\mathrm{ALG} \le \rho \cdot \mathrm{LB} \quad\text{and}\quad \mathrm{LB} \le \mathrm{OPT}$$

That makes it a $\rho$-approximation. The algorithm constructs its solution and its lower bound together. The lab returns both, so tests check each inequality separately.

the limit of the method

If some instance has $\mathrm{OPT} / \mathrm{LB} = r$, no analysis against that $\mathrm{LB}$ can prove a ratio below $r$. That's the integrality gap of unit 05 again, for bounds that aren't LPs.

problem	lower bound	ρ
set cover	prices ÷ $H_d$: a feasible LP dual (unit 03)	$H_d$
vertex cover	size of any matching	2
k-center	half the spacing of $k + 1$ points	2
metric TSP	minimum spanning tree	2
metric TSP	MST, and ½·tour ≥ matching on odd vertices	3/2
makespan	$\max(\sum p / m,\ \max p)$	$2 - 1/m$
makespan, LPT	… and $2\,p_{m+1}$ after sorting	$4/3 - 1/(3m)$

23 · guarantees

Greedy set cover and its prices

Greedy set cover: buy the set with the least cost per newly covered element. Charge each new element that price.

price lemma

For every set $S$: $\sum_{e \in S} \mathrm{price}_e \le H_{|S|}\, c_S$.

why

While some of $S$ was still uncovered, $S$ itself was on offer at $c_S$ divided by what it still covered, and greedy never pays more than the best offer. The last element of $S$ costs at most $c_S$, the one before at most $c_S/2$, and so on: $H_{|S|}\,c_S$. Add over an optimal cover to get $\mathrm{ALG} \le H_d \cdot \mathrm{OPT}$.

full proof in the notes (N)

Johnson 1974; Lovász 1975; Chvátal 1979. Unit 22: $(1-\varepsilon)\ln n$ is NP-hard, so this is optimal.

tight family

Singletons $\{i\}$ at cost $L/(i+1)$, and the whole universe at $L + 1$ ($L = \mathrm{lcm}(1..n)$). Greedy takes every singleton, at total cost $L \cdot H_n$.

instance	greedy / OPT	bound
tight, $n = 8$	2.715	$H_8 = 2.718$
tight, $n = 16$	3.381	$H_{16} = 3.381$
20 random (120 elements, 80 sets)	mean 1.173, max 1.302	$H_d \approx 3.6$

Lab then, optima by CP-SAT.

Full proof of the price lemma

List $S$'s elements in the order greedy covered them, last first: $e_k, \dots, e_1$. When $e_j$ was covered, at least $j$ elements of $S$ were uncovered, so $S$ offered a ratio of at most $c_S / j$. Greedy paid no more: $\mathrm{price}_{e_j} \le c_S / j$. Summing over $j$ gives $H_k\, c_S$. Applied to each set of an optimal cover, whose sets together contain every element: $\mathrm{ALG} = \sum_e \mathrm{price}_e \le H_d \cdot \mathrm{OPT}$. $\square$

The price lemma is an LP duality argument in disguise. Scaled down by $H_d$, the prices satisfy every dual constraint $\sum_{e \in S} y_e \le c_S$ of the set cover LP. So $\mathrm{ALG}/H_d = \sum_e y_e \le \mathrm{LP} \le \mathrm{OPT}$. This is dual fitting, and the guarantee is really against the LP, which is stronger. Unit 24 turns the idea into the primal–dual method.

The tight family uses weights. For unit costs greedy's worst case is still $\Theta(\log n)$, but the classic instance is more intricate (two disjoint "rows" and exponentially growing "columns") and only reaches about $\tfrac12 \log_2 n$.

23 · guarantees · example

Greedy set cover, on five elements

Elements 0–4. Sets: $S_0 = \{2\}$ cost 5, $\ S_1 = \{0\}$ cost 1, $\ S_2 = \{1,2,3,4\}$ cost 6, $\ S_3 = \{0,1,3,4\}$ cost 4.

greedy, step by step

step	cost per new element	buys	prices charged
1	$5,\ 1,\ \tfrac64,\ \tfrac44$	S₁ (tie, lower index)	element 0: 1
2	$5,\ \cdot,\ \tfrac64,\ \tfrac43$	S₃	1, 3, 4: $\tfrac43$ each
3	$5,\ \cdot,\ 6,\ \cdot$	S₀	element 2: 5

Total $1 + 4 + 5 = $ 10 $=$ the sum of the prices.

the optimum

S₁ $+$ S₂: cost 7. Greedy's step 2 looked cheapest ($\tfrac43$ against $\tfrac32$) but left element 2 to be bought alone at 5.

the price lemma, checked

On S₂, four elements, cost 6: its prices are $\tfrac43 + 5 + \tfrac43 + \tfrac43 = 9 \le H_4 \cdot 6 = 12.5$.

On S₁: $1 \le H_1 \cdot 1 = 1$.

Adding over the optimal cover: $10 \le 12.5 + 1$, so greedy is within $H_4 \approx 2.08$ of optimal. Actual ratio $10/7 = 1.43$.

explore interactively

Step 1 in detail. Nothing is covered yet, so each set's ratio is cost over size: S₀ 5, S₁ 1, S₂ $6/4 = 1.5$, S₃ $4/4 = 1$. $S_1$ and $S_3$ tie at 1; the reference breaks ties by lowest index. Step 2. Element 0 is covered: $S_2$ still covers 4 new ($1.5$), $S_3$ covers 3 new ($\tfrac43$). Step 3. Only element 2 remains: S₀ at 5 beats S₂ at 6.

Watching the lemma's order. Greedy covered S₂'s element 2 last, when only 1 of $S_2$'s elements was uncovered: price $5 \le 6/1$. Its elements 1, 3, 4 were covered when all 4 were uncovered: $\tfrac43 \le 6/4$. The proof's bounds $6/1, 6/2, 6/3, 6/4$ are generous here.

From the reference greedy_set_cover (prices as Fractions) and unit 00's brute_force over the 16 subsets, which finds 7 at S₁ and S₂. The instance came from a search for small cases where greedy loses by at least 40%.

section 2

Packings

Objects no single choice can serve twice.

23 · packings

Vertex cover by matching; k-center by farthest-first

vertex cover

Take a maximal matching $M$ (scan edges, keep those with both ends free). Cover: all $2|M|$ endpoints (matching-based vertex cover).

Feasible: an uncovered edge could have joined $M$.
$\mathrm{OPT} \ge |M|$: matching edges share no vertex, so each needs its own.

Tight on $K_{n,n}$: $2n$ against $n$. UGC-hard to beat (unit 22).

Measured, G(n, p) with n = 50 to 200: mean 1.53–1.59, worst 1.74.

k-center (Gonzalez 1985)

$k$-center by farthest-first traversal: start anywhere; repeatedly add the point farthest from the chosen centres. Radius $r$ = the farthest distance left.

The $k$ centres and the final farthest point are $k + 1$ points pairwise $\ge r$ apart: a packing witness.
Two share an optimal centre, so $r \le 2\,\mathrm{OPT}$ by the triangle inequality.

NP-hard to beat $2 - \varepsilon$ (Hsu & Nemhauser 1979).

Measured, 80 points: mean 1.64 (k = 3), 1.42 (k = 6), 1.28 (k = 10); worst 1.91.

section 3

Metric TSP

Trees, matchings, Euler, and a shortcut.

23 · metric TSP

The double tree: 2

The double-tree algorithm for metric TSP.

Solid: the MST. Dashed: the one shortcut, replacing 5 → 4 → 2 → 1 → 0 → 3.

$\mathrm{MST} \le \mathrm{OPT}$ (minimum spanning tree): delete one edge of the optimal tour and a spanning path remains.
Walk around the tree, using every edge twice: length $2\,\mathrm{MST}$.
Skip vertices already visited (shortcutting). Each skip replaces a path by one edge, no longer by the triangle inequality.

So $\mathrm{tour} \le 2\,\mathrm{MST} \le 2\,\mathrm{OPT}$. The walk's shortcut is exactly a preorder of the tree.

Measured, 30 instances with $n \le 60$: mean 1.30–1.37, worst 1.49.

without the triangle inequality

Step 3 fails, and unit 22's Hamiltonian-cycle gadget shows no ratio at all is possible.

23 · metric TSP

Christofides: 3/2

Christofides' algorithm, five steps, each a unit of the course.

MST $T$ (unit 13). $w(T) \le \mathrm{OPT}$.
Odd vertices $O$ of $T$: an even number (handshake lemma).
Min-weight perfect matching $M$ on $O$ (unit 15's blossom, weighted). $w(M) \le \mathrm{OPT}/2$.
Euler circuit of $T + M$: every degree is now even.
Shortcut: triangle inequality.

why the matching costs at most half the tour

Shortcut the optimal tour to visit only $O$: length $\le \mathrm{OPT}$. Its edges alternate between two perfect matchings on $O$. The cheaper is $\le \mathrm{OPT}/2$, and $M$ is cheaper still.

n	double tree	Christofides	MST/OPT	M/OPT
20	1.323 (max 1.486)	1.112 (1.194)	0.819	0.337
40	1.301 (1.446)	1.101 (1.130)	0.830	0.336
60	1.370 (1.443)	1.136 (1.191)	0.867	0.332

Lab then: 10 instances per size, mean (max), optima by CP-SAT's circuit constraint (under 2 s on average).

45 years at 3/2

Christofides (1976) and Serdyukov (1978). Karlin, Klein & Oveis Gharan (2021): $3/2 - 10^{-36}$, by sampling a random spanning tree from the max-entropy distribution fitted to the Held–Karp LP.

explore interactively

The checkpoint asks you to point at the line of code for each step of this proof. In the reference solution christofides is three lines, and each of the five steps above is one call. The proof's structure is the code's structure.

Christofides' 3/2 is tight: there are instances (a "zig-zag" strip of unit triangles, where the MST is a path whose two ends are far apart) where the matching edge is nearly OPT/2 and no shortcut helps. The Held–Karp LP's integrality gap for metric TSP is between 4/3 and 3/2, and closing that interval is open. A follow-up by the same authors (2022) showed the gap is strictly below 3/2 as well.

Serdyukov's paper was in Russian and went unnoticed in the West for decades. Some authors now call it the Christofides–Serdyukov algorithm.

23 · metric TSP · example

Both TSP algorithms, on six points

The six points of the double-tree figure, distances rounded to integers (one unit = 10 px).

	0	1	2	3	4	5
0	·	8	16	10	17	23
1		·	9	12	13	17
2			·	14	9	9
3				·	10	18
4					·	8

the lower bounds

MST: $01, 03, 12, 24, 45$ = $8 + 10 + 9 + 9 + 8 =$ 44 $\le$ OPT.

Odd vertices of the MST: 3 and 5 (leaves). Their only perfect matching: edge $35$, weight 18 $\le$ OPT/2.

Optimal tour (brute force): $0\,1\,2\,5\,4\,3$, length 54. Check: $44 \le 54$ and $18 \le 27$.

algorithm	tour	length	promised at most
double tree	0 1 2 4 5 3	62	$2 \cdot 44 = 88$
Christofides	0 3 5 4 2 1	62	$44 + 18 = 62$
optimum	0 1 2 5 4 3	54

Both tours are the same cycle traversed from different ends: ratio $62 / 54 = 1.15$. Christofides lands exactly on its own bound $w(T) + w(M) = 62$, because shortcutting its Euler circuit $0\,3\,5\,4\,2\,1\,0$ saved nothing.

The optimum avoids the long edge 35 (18) by visiting 5 from 2 and 4, and 3 from 4 and 0.

explore interactively

Double tree. The reference DFS from 0 visits children in increasing order: 0, 1, 2, 4, 5, back up, then 3. Tour edges $01 (8) + 12 (9) + 24 (9) + 45 (8) + 53 (18) + 30 (10) = 62$. The walk around the tree has length 88; shortcutting $5 \to 4 \to 2 \to 1 \to 0 \to 3$ into $5 \to 3$ saved 26.

Christofides. The MST's degrees are 2, 2, 2, 1, 2, 1, so only 3 and 5 are odd, and the matching is forced. $T + M$ is a single cycle through all six vertices, so the Euler circuit visits each vertex once and the shortcut changes nothing: the tour is that cycle, read as 0 3 5 4 2 1. With more odd vertices the matching has choices and shortcuts start to help.

All from the reference prim, double_tree, odd_vertices, christofides and a brute force over the 120 tours starting at 0.

section 4

Scheduling

And the distance between guarantee and practice.

23 · scheduling

List scheduling and LPT (Graham 1966, 1969)

$n$ jobs, $m$ identical machines, minimise the makespan (identical machines). Lower bounds: $\mathrm{OPT} \ge \sum_j p_j / m$ and $\mathrm{OPT} \ge \max_j p_j$.

list scheduling: each job to the least-loaded machine

Let job $j$ finish last. Its machine was least loaded, so it held $\le (\sum p - p_j)/m$:

$$\begin{aligned} C &\le \tfrac{1}{m}\textstyle\sum p + (1 - \tfrac1m)\,p_j \\ &\le (2 - \tfrac1m)\,\mathrm{OPT}\end{aligned}$$

LPT: sort longest first

If the last job $j$ sets the makespan and $j > m$, then $p_j \le p_{m+1}$ and two of the first $m+1$ jobs share a machine in OPT: $p_j \le \mathrm{OPT}/2$, giving $3/2$. A finer counting argument gives $4/3 - 1/(3m)$.

$n / m$	list sched.	LPT
12 / 3	1.135 (1.282)	1.025 (1.070)
30 / 5	1.107 (1.233)	1.012 (1.030)
60 / 10	1.131 (1.178)	1.012 (1.027)

20 instances each, times uniform on 1..100, mean (max).

tight family	m = 3	m = 10
$m(m-1)$ ones, then $m$: list sched.	5/3	19/10
pairs $2m{-}1 \dots m{+}1$, three $m$: LPT	11/9	39/30

Better: a PTAS (Hochbaum & Shmoys 1987) and, for fixed $m$, an FPTAS (Sahni 1976). Strongly NP-hard when $m$ is part of the input, so no FPTAS then.

23 · scheduling · example

The tight families, on seven jobs

six jobs of 1, then one of 3; three machines

List scheduling spreads the ones evenly: loads $2, 2, 2$. The 3 then lands on a machine with 2: makespan 5.

Optimum: the 3 alone, three ones on each other machine: 3. Ratio $5/3 = 2 - \tfrac13$, exactly the bound.

LPT places the 3 first and then the ones: 3, optimal.

jobs 5, 5, 4, 4, 3, 3, 3; three machines

job	goes to	loads after
5, 5, 4	one each	5, 5, 4
4	the 4	5, 5, 8
3	a 5	8, 5, 8
3	the other 5	8, 8, 8
3	any	11, 8, 8

Optimum: $\{5,4\}, \{5,4\}, \{3,3,3\}$, makespan 9. LPT's ratio $11/9 = \tfrac43 - \tfrac19$, exactly its bound for $m = 3$.

23 · beyond

Local search as an approximation algorithm: k-median

$k$-median: open $k$ facilities, minimising the sum of distances from clients to their nearest open facility (k-center minimises the maximum).

Arya et al. (2004)

Local search that swaps up to $p$ facilities at a time, stopping at a local optimum, is a $(3 + 2/p)$-approximation: local search with a guarantee.

The proof pairs each optimal facility with a swap and adds up the local optimality inequalities of all the swaps: the same "compare against OPT piece by piece" as the price lemma.

k-median	ratio
single swaps	5
$p$ swaps	$3 + 2/p$
LP-based (Byrka et al. 2015, and later)	below 2.7
NP-hard (Jain, Mahdian & Saberi 2002)	$1 + 2/e \approx 1.736$

To stop in polynomial time, accept a swap only if it improves by a factor $1 - \varepsilon/\mathrm{poly}(n)$. That costs a $(1 + \varepsilon)$ factor in the ratio.

Unit 26 uses local search for speed; this is local search with a proof.

23 · lab

Lab 23 — what you build

about 2½ hours · uv run co lab 23

Greedy set cover with prices, $H_k$, and the tight family.
Packings: maximal-matching vertex cover; farthest-first k-center with its witness.
Prim and the double tree.
Christofides: odd vertices, Hierholzer, and the composition.
List scheduling and LPT, with both tight families.

uv run co test 23
uv run co then 23     # ratios against CP-SAT optima · tight families · the lower bounds themselves

23 · checkpoint

Done when

checkpoint

All five lab steps are ✓.
Your Christofides never exceeds 1.5× the optimum on instances you can solve exactly, and you can point at the line of code for each step of the proof.
For each algorithm you can name its lower bound, its tight family, and how far random instances sit from both.

Reading

Williamson & Shmoys (2011), ch. 1–2.
Vazirani, Approximation Algorithms (2001), ch. 2–3.
Arya et al., SIAM J. Comput. 33 (2004).
Karlin, Klein & Oveis Gharan, STOC 2021.

24 LP rounding and primal–dual: weighted vertex cover, randomized set cover, facility location · 26 metaheuristics.

Combinatorial approximation algorithms

The route

The shape of a guarantee

Every proof in this unit has one shape

Greedy set cover and its prices

Greedy set cover, on five elements

Packings

Vertex cover by matching; k-center by farthest-first

Metric TSP

The double tree: 2

Christofides: 3/2

Both TSP algorithms, on six points

Scheduling

List scheduling and LPT (Graham 1966, 1969)

The tight families, on seven jobs

Local search as an approximation algorithm: k-median

Lab 23 — what you build

Done when

Reading

Next