Part V · where the trunks converge · spine

LP rounding and the primal–dual method

One lower bound for everything: the LP relaxation. Round its solution, or grow its dual and buy what the dual pays for. Either way, its integrality gap is the limit.

needs 23, 03~1½ h slides + 2½–3 h lablab: uv run co lab 24

Part II used LP relaxations to prove optimality inside branch and bound. Here the same relaxation is used without branching: solve it once (or not even that), and turn it into an integral solution whose cost you can bound relative to the LP.

Duality (unit 03) is the load-bearing idea. Rounding compares against the primal LP value, and primal–dual compares against a dual feasible solution it builds itself. Weak duality makes both lower bounds on OPT.

How to read this deck

Every method is run once on something small enough to check by hand: a weighted 5-cycle for threshold rounding and primal–dual vertex cover, three sets on three elements for randomized rounding, four sets on a 4-cycle for primal–dual set cover, and two facilities on a line for Jain–Vazirani. All numbers come from the lab's reference solution. The longer analyses (the factor 3 of Jain–Vazirani, the Markov step) are in the notes under Full proof.

24 · overview

The route

1 · rounding

Vertex cover at ½; half-integrality; randomized rounding for set cover and its union bound.

2 · primal–dual

Raise duals, buy tight sets: set cover; Jain–Vazirani facility location; Steiner forest.

3 · integrality gaps

K_n and GF(2)^k: instances that bound every LP-based proof.

4 · measured

LP, optimum, algorithm on the same instances, and where greedy wins.

getting around

Words with a dotted underline are glossary terms: click one for its definition. G lists every term in the unit, N shows the notes. Optional extra: interactive figures, linked from the slides they support by a small ▸ chip; skip them freely.

section 1

Rounding

Solve the relaxation, then make it integral.

24 · rounding

Vertex cover at ½

$$\begin{aligned}\min\ & \textstyle\sum_v w_v x_v \\ \text{s.t. } & x_u + x_v \ge 1 \quad \forall uv \in E \\ & 0 \le x \le 1\end{aligned}$$

why it works (Hochbaum 1982)

Threshold rounding: take $v$ iff $x_v \ge \tfrac12$. Feasible: $x_u + x_v \ge 1$ forces one of them to $\ge \tfrac12$. Cost: every chosen $v$ has $w_v \le 2 w_v x_v$, so the cost is $\le 2\,\mathrm{LP} \le 2\,\mathrm{OPT}$.

half-integrality (Nemhauser & Trotter 1975)

Every vertex of this polytope has all $x_v \in \{0, \tfrac12, 1\}$.

C₅: LP 2.5, OPT 3, rounding takes all 5.

Lab then, 24 weighted random graphs (weights 1–20):

LP / OPT	rounding / OPT	primal–dual / OPT	half-integral
0.89	1.74–1.75	1.38–1.40	24 / 24

"Genuinely surprising", as the curriculum says, because the matching argument of unit 23 fails for weights: a heavy vertex and a light neighbour make "take both endpoints" arbitrarily bad. The LP handles weights for free.

Half-integrality explains the measurement. A basic solution has many exact halves, and threshold rounding takes every one of them. Nemhauser and Trotter's stronger statement is persistence: some optimal integral cover contains every vertex at 1 and avoids every vertex at 0. So the LP solves part of the instance exactly, which makes it a standard kernelisation in parameterised algorithms. Only the ½ part is hard.

Threshold rounding for set cover at $1/f$ (with $f$ the largest element frequency) gives Hochbaum's $f$-approximation, the same guarantee as primal–dual in section 2.

24 · rounding · example

Rounding a weighted 5-cycle

Vertex weights inside the circles. The LP puts ½ on every vertex.

method	takes	weight	/ OPT
LP relaxation	all at ½	7	0.875
optimum (brute force)	$v_0, v_1, v_3$	8	1
threshold at ½	all five	14	1.75
primal–dual (section 2)	$v_0, v_1, v_2, v_3$	10	1.25

reading it

Every $x_v = \tfrac12$ meets the threshold, so rounding pays the full weight 14, exactly $2 \times \mathrm{LP}$: the guarantee is met with equality. The integrality gap of this instance is $8 / 7$, so no rounding of this LP can prove better than 1.14 here.

explore interactively

Why all halves is optimal for the LP. Each edge needs $x_u + x_v \ge 1$. Summing over the 5 edges counts every vertex twice, so $\sum_v x_v \ge 2.5$. All halves costs half the total weight, $14/2 = 7$, and no cheaper fractional cover exists for these weights. The reference vertex_cover_lp with weights $(3,2,2,3,4)$ returns value 7 at all halves.

The optimum. A cover of an odd cycle needs 3 vertices. The cheapest triple that touches all five edges is v₀, v₁, v₃: $3 + 2 + 3 = 8$ (it covers $v_0v_1, v_1v_2, v_2v_3, v_3v_4, v_4v_0$).

Primal–dual treats vertices as sets and edges as elements. It raises the edge duals in order and buys v₀, v₁, v₂, v₃ for 10, with dual sum 6 ($y = (2, 0, 2, 1, 1)$ on edges $v_0v_1, \dots, v_4v_0$), so its own certificate says $10 \le 2 \cdot 6$.

24 · rounding

Randomized rounding for set cover

Randomized rounding: repeat $t$ times, picking each set $j$ independently with probability $x_j$. Take the union.

why it works

Miss. The LP makes each element's sets add up to at least 1, so one round misses it with probability

$$\textstyle\prod_{j \ni e}(1 - x_j) \le e^{-\sum_{j \ni e} x_j} \le e^{-1}$$

and $t$ rounds with probability $\le e^{-t}$. The union bound over $n$ elements: $\Pr[\text{not a cover}] \le n e^{-t}$.

Cost. $\mathbb{E}[\text{cost}] \le t \cdot \mathrm{LP}$, by linearity of expectation. Take $t \approx \ln n$.

combining both: notes (N)

Lab then: failure rate of the whole cover, 4 000 draws over 10 instances (150 elements):

t	measured	bound $150e^{-t}$
1	0.896	≥ 1
2	0.728	≥ 1
4	0.123	≥ 1
7	0.0033	0.137

The union bound is loose by 40× at $t = 7$. Tests check the per-element probabilities exactly, against simulation.

Raghavan & Thompson (1987) introduced randomized rounding for routing, where Chernoff bounds are needed because loads add up. For set cover the union bound alone gives $O(\log n)$.

Full proof: cost and coverage together

With $t = \ln n + 2$ the failure probability is at most $e^{-2} < 0.14$. By Markov's inequality, the cost exceeds $4t \cdot \mathrm{LP}$ with probability at most $\tfrac14$. So with probability at least $1 - \tfrac14 - e^{-2} > 0.6$ the result is a cover costing at most $4t \cdot \mathrm{LP} = O(\log n)\,\mathrm{OPT}$. Repeat until that happens; the expected number of repetitions is below 2.

Why 1 − x ≤ e^−x is the whole trick: it converts the product of miss probabilities, which depends on how x is spread across sets, into an exponential of the LP constraint's left-hand side, which is at least 1 however it's spread.

Derandomisation: the method of conditional expectations with a pessimistic estimator (the union bound's sum plus a cost term) turns this into a deterministic algorithm. It's also one way to derive greedy set cover, which is part of why greedy's H_n and rounding's ln n match.

The curriculum mentions the Chernoff bound for set cover. Strictly, set cover needs only the union bound. Chernoff enters when you round under packing constraints (edge capacities in routing, multicommodity flow), where the danger is too much load rather than too little coverage.

24 · rounding · example

Randomized rounding, on three elements

Elements $\{0, 1, 2\}$, unit-cost sets $\{0,1\}$, $\{1,2\}$, $\{0,2\}$. Any two sets cover; one doesn't. OPT $= 2$.

The LP puts $x = \tfrac12$ on each set: every element is in two sets, $\tfrac12 + \tfrac12 = 1$. LP $= 1.5$.

element 0, one round

It is missed only if neither $\{0,1\}$ nor $\{0,2\}$ is picked: $(1 - \tfrac12)(1 - \tfrac12) = \tfrac14$. The general bound says $\le e^{-1} = 0.37$.

$t$	miss, one element	union bound $3e^{-t}$	expected cost
1	0.25	1.10	1.50
2	0.0625	0.41	2.25
3	0.0156	0.15	2.63
5	0.00098	0.020	2.91

Misses fall like $4^{-t}$, faster than the bound's $e^{-t}$. The cost rises towards 3, all sets, as $t$ grows: more rounds buy coverage with money.

section 2

Primal–dual

Raise duals, buy primals, pay for what you bought.

24 · primal–dual

The schema, on set cover

$$\begin{aligned}\textbf{primal}\quad \min\ & \textstyle\sum_j c_j x_j \ \ \text{s.t.}\ \textstyle\sum_{j \ni e} x_j \ge 1 \\ \textbf{dual}\quad \max\ & \textstyle\sum_e y_e \ \ \text{s.t.}\ \textstyle\sum_{e \in S_j} y_e \le c_j\end{aligned}$$

Start with $y = 0$, nothing bought.
For each uncovered $e$: raise $y_e$ until some set containing $e$ is tight.
Buy every tight set containing $e$.

Primal–dual set cover, Bar-Yehuda & Even (1981).

why the factor is f, the largest frequency

Each bought set is paid for by its elements' $y$, and an element sits in at most $f$ sets, so it pays at most $f$ times:

$$\begin{aligned}\text{cost} &= \textstyle\sum_{j \text{ bought}} c_j = \sum_{j \text{ bought}} \sum_{e \in S_j} y_e \\ &\le f \textstyle\sum_e y_e \le f \cdot \mathrm{LP} \le f \cdot \mathrm{OPT}\end{aligned}$$

complementary slackness, relaxed

Primal CS exact: $x_j > 0 \Rightarrow$ set $j$ tight. Dual CS relaxed: $y_e > 0 \Rightarrow$ $e$ covered at most $f$ times, not exactly once. The approximation ratio is exactly the relaxation factor.

Never solves an LP. Vertex cover is $f = 2$: a 2-approximation in linear time.

"Pay for what you bought": the dual variables are money, each element's y_e is what it contributes, and a set is bought only once its elements have paid its full cost. Charging each bought set's cost back to the y of its elements counts each y_e at most f times.

That's the general recipe. Write the LP and its dual. Grow the dual continuously, keeping it feasible. When a dual constraint goes tight, set the matching primal variable. Stop when the primal is feasible. Then bound how many times each dual variable is charged. The factor comes from that count, and designing the growth to keep it small is the art. Facility location and Steiner forest are the famous cases. This is the primal–dual method.

On the GF(2)^k family every element has frequency 2^k−1 and primal–dual pays exactly that (lab then: 64 for k = 7, where OPT is 7).

24 · primal–dual · example

Primal–dual set cover, traced

Four elements on a cycle. Sets $S_0 = \{0,1\}$ cost 3, $S_1 = \{1,2\}$ cost 2, $S_2 = \{2,3\}$ cost 3, $S_3 = \{3,0\}$ cost 2. Each element is in 2 sets: $f = 2$.

element	sets, still unpaid	raise $y$ by	tight, buy
0	S₀: 3, S₃: 2	2	S₃ (covers 3, 0)
1	S₀: 1, S₁: 2	1	S₀ (covers 0, 1)
2	S₁: 1, S₂: 3	1	S₁ (covers 1, 2)
3	already covered

the bill

Bought S₃, S₀, S₁: cost $2 + 3 + 2 = 7$.

Dual $y = (2, 1, 1, 0)$, sum 4, feasible: each set's elements pay at most its cost. So $\mathrm{OPT} \ge 4$.

Guarantee $7 \le f \cdot 4 = 8$ ✓. Optimum: S₁ $+$ S₃ covers everything for 4.

S₀ was bought and turned out redundant: S₃ and S₁ already cover 0 and 1. Primal–dual has no clean-up step here; Steiner forest's reverse delete is exactly such a step.

explore interactively

24 · primal–dual

Facility location: Jain & Vazirani (2001)

$$\begin{aligned}\max\ & \textstyle\sum_j \alpha_j \\ \text{s.t. } & \textstyle\sum_j \beta_{ij} \le f_i \\ & \alpha_j - \beta_{ij} \le d_{ij},\ \beta \ge 0\end{aligned}$$

The dual of the facility location LP. Phase 1. All $\alpha_j$ grow at rate 1. Once $\alpha_j \ge d_{ij}$, client $j$ pays $\beta_{ij} = \alpha_j - d_{ij}$ toward $i$. When $i$ is fully paid, it opens temporarily. A client stops growing (freeze) once it's tight with an open facility.

Phase 2. Temporarily open facilities conflict if a client pays both. Keep a maximal non-conflicting set, in opening order. Clients go to the nearest kept facility.

why 3 (metric), in words

Each client's $\alpha_j$ is a lower bound on what it must cost. If the facility it froze with was kept, it is within $\alpha_j$. If not, a kept facility is reachable by a three-hop detour, each hop at most $\alpha_j$ long: at most $3\alpha_j$.

full proof in the notes (N)

Jain–Vazirani, lab then:

F × C	LP integral	JV / OPT	Σα / OPT
10 × 40	10/10	1.031 (1.119)	0.990
20 × 80	10/10	1.057 (1.140)	0.995
30 × 120	10/10	1.054 (1.103)	0.992

Full proof of the factor 3

Let $i$ be the facility client $j$ froze with, so $d_{ij} \le \alpha_j$ and $t_i \le \alpha_j$. If $i$ was kept, $j$ is within $\alpha_j$. If not, $i$ conflicts with a kept $i'$ through a client $h$, and $i'$ opened no later. Then $d_{hi}, d_{hi'} \le \alpha_h \le t_{i'} \le t_i \le \alpha_j$, and by the triangle inequality $d_{i'j} \le d_{ij} + d_{hi} + d_{hi'} \le 3\alpha_j$.

The step "$\alpha_h \le t_{i'}$" needs care: client h strictly pays $i'$, so it reached $i'$ before freezing, and it froze no later than $i'$ opened. The step "$t_i \le \alpha_j$" holds because i is the facility j froze with, so j froze no earlier than i opened. The lab's tests check the invariant behind both: each client froze at $\min_i \max(t_i, d_{ij})$.

The analysis actually proves the stronger Lagrangian-multiplier-preserving (LMP) bound $\text{connection} + 3 \cdot \text{facility} \le 3\sum\alpha$: facilities are paid for exactly by the part of α spent on them. That's what lets JV give a 6-approximation for k-median by Lagrangian relaxation (unit 11) over the facility cost.

The best known ratio is 1.488 (Li 2013), and 1.463 is hard (Guha & Khuller 1999). On random metric instances the LP was integral 30 times out of 30: the facility location LP is strong, and its bad cases have to be constructed.

24 · primal–dual · example

Jain–Vazirani on a line

Facilities A and B (squares, opening cost 4 each), clients at positions 1, 2, 9. Distance is the gap on the line.

time	event
1	c₀ reaches A, c₂ reaches B: both start paying
2	c₁ reaches A; A now collects $(t-1) + (t-2)$
3.5	A has collected $2.5 + 1.5 = 4$: opens; c₀, c₁ freeze at $\alpha = 3.5$
5	B has collected $5 - 1 = 4$ from c₂: opens; $c_2$ freezes at $\alpha = 5$

the result

No client pays both, so both are kept. Cost: opening $4 + 4$, serving $1 + 2 + 1$: 12. The dual $\sum\alpha = 3.5 + 3.5 + 5 = 12$ is a lower bound, so this is optimal, and the certificate proves it.

explore interactively

The ascent. Every $\alpha$ grows with time $t$ until it freezes. A client pays a facility $\max(0, \alpha_j - d_{ij})$. A's receipts after time 2 are $(t - 1) + (t - 2) = 2t - 3$, which reaches 4 at $t = 3.5$. At that moment c₀ and c₁ are within reach of the open A, so they freeze. c₂ is 9 away from A, so it keeps growing and pays only B: $t - 1 = 4$ at $t = 5$.

Pruning. c₀ and c₁ pay only A ($3.5 < 9, 8$), c₂ pays only B ($5 < 9$). No conflicts, both kept. Brute force over the three open sets gives $\{A\}$: $4 + 1 + 2 + 9 = 16$; $\{B\}$: $4 + 9 + 8 + 1 = 22$; both: 12. On bigger instances the pruning step does real work, and the factor 3 is paid only there.

The instance is FacilityLocation((4, 4), ((1, 2, 9), (9, 8, 1))); the reference jv_dual_ascent returns $\alpha = (\tfrac72, \tfrac72, 5)$ and opening times $\tfrac72$, 5, and jain_vazirani keeps both.

24 · beyond

Steiner forest, and iterative rounding

Steiner forest

Pairs $(s_i, t_i)$ in a weighted graph: the cheapest edge set connecting every pair. The LP has one covering constraint per cut separating some pair, exponentially many.

Primal–dual (Agrawal, Klein & Ravi 1995; Goemans & Williamson 1995): grow duals on all active components at once, buy tight edges, then reverse delete unnecessary ones. Ratio 2, by showing active components touch at most twice as many bought edges as there are components, on average.

iterative rounding (Jain 2001)

Steiner network: each pair needs $r_{ij}$ edge-disjoint paths. Theorem: every extreme point of the LP has some edge with $x_e \ge \tfrac12$. Algorithm: solve, fix that edge to 1, update the residual requirements, re-solve. Ratio 2.

The proof is a counting argument on tight constraints forming a laminar family, the same structure as the matroid intersection polytope (unit 13). It set off a line of results on degree-bounded spanning trees (Singh & Lau 2007: optimal cost, degrees exceeded by at most 1).

section 3

Integrality gaps

The limit of every LP-based proof.

24 · gaps

Instances that realise the gap

integrality gap

$\sup_I \mathrm{OPT}(I) / \mathrm{LP}(I)$ for minimisation. An algorithm proved via $\mathrm{ALG} \le \rho\,\mathrm{LP}$ has $\rho \ge$ the gap, since it can't beat OPT on the worst instance.

Gap instances. Vertex cover, $K_n$. All ½ gives $n/2$. Two uncovered vertices leave their edge uncovered, so $\mathrm{OPT} = n - 1$. Gap $2 - 2/n$.

Set cover, GF(2)^k. The smallest case, $k = 2$: three sets on three elements, each set missing one element; LP 1.5, OPT 2: the three-element example of section 1. Elements and sets are the nonzero vectors; set $v$ holds $\{u : u \cdot v = 1\}$. Every set has $2^{k-1}$ elements, so $x = 2^{1-k}$ gives LP $< 2$. Sets $v_1, \dots, v_r$ miss every nonzero $u$ orthogonal to all of them, so $\mathrm{OPT} = k$.

k	sets	LP	OPT	OPT/LP	greedy	primal–dual
3	7	1.750	3	1.71	3	4
4	15	1.875	4	2.13	4	8
5	31	1.938	5	2.58	5	16
6	63	1.969	6	3.05	6	32
7	127	1.984	7	3.53	7	64

a limit on proofs, not on algorithms

Greedy is optimal on every member. The gap says no analysis against this LP can certify better than $\approx \tfrac12\log_2 n$ here, not that finding a good cover is hard.

24 · measured

LP bound, optimum, algorithm: what the measurements say

Set cover, 10 instances, 150 elements and 100 sets, optimum by CP-SAT:

algorithm	mean / OPT	worst	guarantee
LP bound	0.937	1.000	—
randomized, $t = 7$, best of 20	1.728	2.140	$O(\log n)$
primal–dual	1.964	2.556	$f = 21$
greedy (unit 23)	1.148	1.326	$H_d \approx 5$

Greedy beats both LP-based algorithms on random set cover, while guaranteeing about the same.
Primal–dual beats rounding on weighted vertex cover (1.39 against 1.75), with the same guarantee of 2. Rounding takes every half.
The LP is tight on random facility location: integral 30 times of 30.
The gap families are built by hand. Nothing random came close to them.

An approximation ratio is a certificate, and practice asks a different question. Unit 26 answers it with heuristics, and unit 28 with a method for comparing them honestly.

24 · lab

Lab 24 — what you build

about 2½–3 hours · uv run co lab 24

Vertex cover LP and threshold rounding.
Randomized rounding for set cover, with exact miss probabilities and the union bound.
Primal–dual set cover.
Jain–Vazirani: event-driven dual ascent in exact arithmetic, pruning, assignment.
Integrality gaps: $K_n$ and GF(2)^k.

uv run co test 24
uv run co then 24     # LP · optimum · algorithm on 72 instances · failure rates · out/then.png

24 · checkpoint

Done when

checkpoint

All five lab steps are ✓.
You can exhibit an instance realising the integrality gap of a given LP, and explain why it proves that no rounding of that LP can beat that ratio.
You can say why greedy solves that same instance optimally without contradicting anything.

Reading

Williamson & Shmoys (2011), ch. 1, 4, 7.
Vazirani (2001), ch. 14–15, 24.
Jain & Vazirani, JACM 48 (2001).
Jain, Combinatorica 21 (2001).

25 semidefinite relaxations: Goemans–Williamson Max-Cut · 26 metaheuristics, benchmarked honestly.

LP rounding and the primal–dual method

The route

Rounding

Vertex cover at ½

Rounding a weighted 5-cycle

Randomized rounding for set cover

Randomized rounding, on three elements

Primal–dual

The schema, on set cover

Primal–dual set cover, traced

Facility location: Jain & Vazirani (2001)

Jain–Vazirani on a line

Steiner forest, and iterative rounding

Integrality gaps

Instances that realise the gap

LP bound, optimum, algorithm: what the measurements say

Lab 24 — what you build

Done when

Reading

Next