Part V · where the trunks converge · spine

Hardness, and the shape of what is possible

Reductions as programs, the numbers that make a problem hard, and the hierarchy of how close to optimal a polynomial algorithm can promise to get.

needs 11 (and 00, 16, 20)~1½ h slides + 2 h lablab: uv run co lab 22

Parts I–IV built exact methods, and the best of them (CP-SAT, a MIP solver with cuts) prove optimality on remarkably large instances. They still take exponential time in the worst case, and Part V takes that seriously. If you must answer in polynomial time, what can you promise?

This unit is the map, and units 23–25 are the territory. The theory here says, for each problem, how good a guarantee is possible. It tells you when a better algorithm is worth looking for and when the search is over unless P = NP.

How to read this deck

Four small instances carry the unit: a two-clause 3-CNF pushed through the vertex-cover gadget and back, a four-item knapsack run through the FPTAS, and two partition instances, six numbers and three, that show bin packing's 3/2 gap. Every number came from the lab's reference solution or brute force. The hardness theorems are stated with a one-line reason; the proofs that are short enough are in the notes.

22 · overview

The route

1 · reductions

Karp's chain, the vertex-cover gadget, and what a reduction does and doesn't preserve.

2 · numbers

Pseudo-polynomial time, weak vs strong NP-hardness, and the knapsack FPTAS.

3 · the hierarchy

FPTAS ⊂ PTAS ⊂ APX ⊂ log-APX ⊂ poly-APX; the gap technique; PCP; Unique Games.

4 · the nine problems

Best ratio, best hardness, and the open gap, for each of unit 00's problems.

getting around

Words with a dotted underline are glossary terms: click one for its definition. G lists every term in the unit, N shows the notes. Optional extra: interactive figures, linked from the slides they support by a small ▸ chip; skip them freely.

section 1

Reductions

A program between problems, with bugs.

22 · reductions

Karp's chain

polynomial (Karp) reduction

Written $A \le_p B$.

A polynomial-time map $f$ from instances of $A$ to instances of $B$ with $x \in A \iff f(x) \in B$. For optimisation, add a polynomial map $g$ taking solutions of $f(x)$ back to solutions of $x$.

If $B$ is in P, so is $A$. Read the other way: if $A$ is NP-hard, so is $B$. The direction matters. Every NP-hardness proof runs from a known hard problem to the new one.

Cook (1971) and Levin (1973): SAT is NP-complete. Karp (1972): 21 more, by reductions.

Shaded: the links the lab mechanises (with CNF → 3-CNF). Every leaf is one of unit 00's nine problems.

22 · reductions

3-SAT → vertex cover, and back

The reduction: one edge per variable, one triangle per clause, each corner joined to its literal.

Filled: the cover of size $k = n + 2m = 7$ built from $x_1 = x_2 = 1$, $x_3 = 0$. The next slide checks it.

Every cover takes ≥ 1 vertex per variable edge and ≥ 2 per triangle, so ≥ $k$.
Satisfiable ⇒ cover of size $k$: the true literals, plus each triangle minus one corner whose literal is true. That corner's link is covered from above.
Cover of size $k$ ⇒ satisfiable: exactly one literal per variable (read it as true) and two corners per triangle. The third corner's link must be covered by its literal, which is therefore true.

n	m	SAT?	vertices	k	min cover
40	170	yes	590	380	380
40	170	no	590	380	381
60	360	no	1 200	780	782

Lab then: both sides solved by CP-SAT; 6 of 6 agree.

explore interactively

The reduction needs exactly three literals per clause, which is why lab step 2 (clause splitting) exists. With a clause of width 2 the gadget is an edge, not a triangle, and $k$ changes per clause. That's doable, but it's the kind of special case where a reduction grows its first bug.

The backward map in the lab takes any cover of size ≤ $k$, not only minimum ones. The lower bound argument shows it's then exactly $k$ with the structure above. Tests hand it a minimum cover from HiGHS, which has no reason to look like the one you would build.

The instance grows from $n$ variables and $m$ clauses to $2n + 3m$ vertices and $n + 6m$ edges: linear. Size matters when you transfer running-time lower bounds (the exponential time hypothesis) rather than just NP-hardness.

22 · reductions · example

The gadget, on the example

Formula $(x_1 \lor \lnot x_2 \lor x_3) \land (\lnot x_1 \lor x_2 \lor \lnot x_3)$: $n = 3$, $m = 2$. The graph has $2n + 3m = 12$ vertices and $n + 6m = 15$ edges, and $k = n + 2m = 7$.

forward: assignment to cover

piece	rule	in the cover
variable edges	the true literal	$x_1$, $x_2$, $\lnot x_3$
clause 1	skip one corner with a true literal ($x_1$)	corners $\lnot x_2$, $x_3$
clause 2	skip one corner with a true literal ($x_2$)	corners $\lnot x_1$, $\lnot x_3$

$3 + 2 + 2 = 7$ vertices. Every one of the 15 edges has an end in the cover; the two skipped corners are covered through $x_1$ and $x_2$ above them.

backward: a cover the solver picked

Brute force over all $2^{12} = 4096$ vertex sets finds 234 covers, the smallest of size 7. The first one found takes $\lnot x_1$, $\lnot x_2$, $\lnot x_3$ at the top.

Read it as $x_1 = x_2 = x_3 = \text{false}$. Clause 1 is satisfied by $\lnot x_2$, clause 2 by $\lnot x_1$. A different satisfying assignment from the one we started with, and just as good.

The solution map never needed the cover to be the one we built: any cover of size $k$ has one vertex per variable edge and two per triangle, and that is enough.

explore interactively

22 · reductions

What a reduction preserves, and what it doesn't

three strengths

Decision: yes-instances to yes-instances. Enough for NP-hardness.
Optimum: $\mathrm{OPT}(f(x))$ determined by $\mathrm{OPT}(x)$, with solutions mapped back. Vertex cover → set cover is this: the same 0/1 vector, the same cost.
Gap: near-optimal solutions of $f(x)$ map back to near-optimal solutions of $x$ (L-reductions, gap-preserving reductions). Needed to transfer hardness of approximation.

measured: the gadget is decision-only

An assignment leaving $u$ clauses unsatisfied gives a cover of $k + u$. So $\text{min cover} - k \le \text{fewest unsatisfied}$. Equality fails: covering both $x_v$ and $\lnot x_v$ costs 1 and can rescue several clause gadgets.

300 small random 3-CNFs, MaxSAT by RC2 against the minimum cover by CP-SAT: the excess was strictly smaller on 19.

So a 1.01-approximate cover tells you nothing useful about a near-satisfying assignment. Approximation-hardness results need reductions built for it.

section 2

Numbers

Where knapsack's hardness lives.

22 · numbers

Weak and strong NP-hardness

Unit 16's knapsack DP runs in $O(nC)$. Polynomial in the value of $C$, exponential in its length $\log C$: pseudo-polynomial.

definitions

Weakly NP-hard: NP-hard, but solvable in time polynomial in $n$ and the largest number.
Strongly NP-hard: still NP-hard when every number is at most a polynomial in $n$ (written in unary).

weak	strong
knapsack, subset sum, partition	bin packing, 3-partition
makespan on 2 machines	makespan on $m$ machines, $m$ in the input
	TSP, set cover, vertex cover, colouring (no numbers at all)

theorem (Garey & Johnson 1978)

If a problem is strongly NP-hard, has integer objective values, and $\mathrm{OPT} \le p(n, \text{max number})$ for a polynomial $p$, then it has no FPTAS unless P = NP.

why

With small numbers the optimum is a small integer, so an error below 1 is no error. Pick $\varepsilon$ that small and the FPTAS becomes an exact polynomial algorithm for a strongly hard problem.

full proof in the notes (N)

An FPTAS is a pseudo-polynomial exact algorithm in disguise, and vice versa (next slide). Weak hardness is the precondition.

Full proof

Run the FPTAS with $\varepsilon = 1/(p + 1)$. Its error is below $\varepsilon\,\mathrm{OPT} < 1$, and values are integers, so it's exact, in time $\mathrm{poly}(n, 1/\varepsilon) = \mathrm{poly}(n, p)$. On the unary-sized instances where the problem is still NP-hard, $p$ is polynomial in $n$, so that's a polynomial exact algorithm, which is impossible unless P = NP. $\square$

Bin packing is the instructive case. It contains partition, which is only weakly hard, so why is bin packing strongly hard? Because 3-partition (split $3m$ numbers into $m$ triples of equal sum) is strongly NP-hard, and it's bin packing with $m$ bins where every bin must be exactly full. The number of bins in the input is what makes the difference: partition is the special case with two bins.

Problems with no numbers (vertex cover, colouring) are trivially strongly NP-hard, since their only numbers are bounded by $n$.

22 · numbers

The knapsack FPTAS (Ibarra & Kim 1975)

An FPTAS: ratio $1 - \varepsilon$ in time polynomial in $n$ and $1/\varepsilon$. This one is the knapsack FPTAS.

Drop items that don't fit alone. Let $v_{\max}$ be the largest remaining value.
$K = \varepsilon\, v_{\max} / n$; round $v_i' = \lfloor v_i / K \rfloor$.
Exact DP by value on $v'$: $W[p]$ = least weight reaching value exactly $p$. At most $n \cdot n/\varepsilon$ values, so $O(n^3/\varepsilon)$ time.

why it loses at most a fraction ε $$\begin{aligned} \textstyle\sum_{S} v_i &\ge K \textstyle\sum_{S} v_i' \ge K \textstyle\sum_{O} v_i' \\ &\ge \textstyle\sum_{O} v_i - nK = \mathrm{OPT} - \varepsilon\, v_{\max} \\ &\ge (1 - \varepsilon)\,\mathrm{OPT}\end{aligned}$$

The last step needs $\mathrm{OPT} \ge v_{\max}$: step 1's filter. Both it and the $/n$ have a lab test that breaks without them.

Lab then: 100 items, values to $10^6$, weights to $10^4$.

ε	allowed loss	actual loss	s
0.5	50%	0.000%	0.11
0.2	20%	0.000%	0.23
0.1	10%	0.000%	0.51
0.05	5%	0.000%	0.99
0.02	2%	0.000%	2.43

Exact weight DP (unit 16): 0.05 s. CP-SAT: 0.23 s. The guarantee is loose by orders of magnitude, and at these sizes the exact method is faster. An FPTAS earns its place when values and weights are huge.

22 · numbers · example

The FPTAS, on four items

Capacity 24, $\varepsilon = 0.5$.

item	value	weight	$v/K$	rounded $v'$
0	16	8	4.57	4
1	28	12	8	8
2	16	7	4.57	4
3	25	7	7.14	7

$v_{\max} = 28$, so $K = 0.5 \cdot 28 / 4 = 3.5$. Values are divided by $K$ and rounded down.

what rounding changes

Exact optimum: items 0, 2, 3, weight 22, value 57.

In rounded values that set scores $4 + 4 + 7 = 15$, but so do items 1, 3 ($8 + 7 = 15$, weight 19). Rounding erased the difference, and the DP returns items 1, 3: true value 53.

Loss $4/57 = 7\%$, well inside the promised 50%. The proof's bound says at least $57 - nK = 57 - 14 = 43$.

Rounding by $K$ loses less than $K$ per item, at most $nK = \varepsilon\,v_{\max}$ in all. That is the whole proof on the previous slide.

explore interactively

section 3

The hierarchy

How close can a polynomial algorithm promise to get?

22 · hierarchy

FPTAS ⊂ PTAS ⊂ APX ⊂ log-APX ⊂ poly-APX

class	guarantee	example
FPTAS	$1 \pm \varepsilon$, time poly$(n, 1/\varepsilon)$	knapsack
PTAS	$1 \pm \varepsilon$, time poly$(n)$ for each fixed $\varepsilon$, e.g. $n^{1/\varepsilon}$	Euclidean TSP (Arora 1998); makespan on $m$ identical machines (Hochbaum & Shmoys 1987)
APX	some constant ratio	vertex cover (2), metric TSP (3/2), Max-Cut (0.878), Max-3SAT (7/8)
log-APX	$O(\log n)$	set cover ($\ln n$)
poly-APX	$n^{c}$	colouring, clique
none	no ratio at all	general TSP; 0/1 IP feasibility

Every inclusion is strict unless P = NP.

the gap technique

If it's NP-hard to tell $\mathrm{OPT} \le a$ from $\mathrm{OPT} \ge b$, no algorithm beats ratio $b/a$ (unless P = NP).

Bin packing, 3/2. Partition asks whether 2 bins suffice. Otherwise 3 are needed. A ratio below 3/2 would decide partition (next slide).
General TSP, nothing. Hamiltonian cycle: edges weigh 1, non-edges $\rho n + 1$. A tour costs $n$ if a cycle exists, and $> \rho n$ otherwise.
Colouring, 4/3. Deciding 3-colourability separates 3 from ≥ 4.

Bin packing's 3/2 is an absolute ratio: it comes from instances with OPT = 2. It says nothing about large instances, and indeed bin packing has an asymptotic PTAS (Fernandez de la Vega & Lueker 1981) and an algorithm using OPT + O(log OPT) bins (Hoberg & Rothvoss 2017). Whether OPT + 1 is achievable in polynomial time is open. "Bin packing is hard to approximate" is true only in the narrow sense.

Metric TSP is where the triangle inequality changes everything: the reduction from Hamiltonian cycle needs non-edges much longer than twice an edge, which the metric forbids. Unit 23 shows the triangle inequality is exactly what the double-tree and Christofides proofs use.

Our TSP.random instances are Euclidean distances rounded to integers. Rounding can break the triangle inequality by one unit, which matters only for proofs, not for measurements.

22 · hierarchy · example

The 3/2 gap, on two instances

a yes-instance: 2 bins

Sizes $3, 1, 1, 2, 2, 1$, total 10, bins of capacity 5.

$\{3, 2\}$ and $\{1, 1, 2, 1\}$: both exactly 5. OPT = 2.

a no-instance: 3 bins

Sizes $4, 4, 2$, total 10, capacity 5.

No subset sums to 5 (the subset sums are 0, 2, 4, 6, 8, 10). Two bins would need exactly 5 each, so OPT = 3: $\{4\}$, $\{4\}$, $\{2\}$.

why no ratio below 3/2

Suppose an algorithm always uses fewer than $\tfrac32\,\mathrm{OPT}$ bins.

On a yes-instance it uses fewer than $\tfrac32 \cdot 2 = 3$ bins: exactly 2. On a no-instance it can't use fewer than 3.

So "did it use 2 bins?" answers partition, which is NP-hard. Unless P = NP, no such algorithm runs in polynomial time.

Both instances have total size exactly twice the capacity. That's the only kind of instance the argument needs, and it's why the 3/2 says nothing about instances needing 100 bins.

22 · hierarchy

The PCP theorem: where approximation hardness comes from

PCP theorem (Arora–Safra; Arora–Lund–Motwani–Sudan–Szegedy 1998)

$\mathrm{NP} = \mathrm{PCP}(O(\log n), O(1))$: every NP statement has a proof that a verifier can check by reading a constant number of bits chosen with $O(\log n)$ random bits, always accepting a correct proof and rejecting a false claim with probability ≥ ½.

equivalently: a gap

There is an $\varepsilon > 0$ such that telling a satisfiable 3-CNF from one where every assignment violates ≥ $\varepsilon m$ clauses is NP-hard.

Dinur (2007) gave a combinatorial proof by gap amplification.

Before PCP, the gap technique needed a lucky NP-hard question with a built-in gap (2 vs 3 bins). Afterwards, every NP problem has one, via Max-3SAT, and gap-preserving reductions carry it onward.

Håstad (2001): optimal inapproximability

A verifier reading 3 bits and checking their parity. Consequence: for every $\varepsilon > 0$, Max-3SAT is NP-hard to approximate within $7/8 + \varepsilon$.

A uniformly random assignment satisfies $7/8$ of the clauses in expectation: a clause of three distinct literals fails only when all three come out false, probability $\tfrac12 \cdot \tfrac12 \cdot \tfrac12 = \tfrac18$. The dumbest algorithm is optimal.

22 · hierarchy

Three thresholds, and the Unique Games Conjecture

set cover

Greedy: $H_n$ $\le \ln n + 1$ (Johnson 1974; Lovász 1975; Chvátal 1979).

NP-hard within $(1 - \varepsilon)\ln n$ (Dinur & Steurer 2014; Feige 1998 under a weaker assumption).

Closed: greedy is optimal up to lower-order terms.

vertex cover

Maximal matching or LP rounding: 2. Nothing known beats $2 - o(1)$.

NP-hard: 1.3606 (Dinur & Safra 2005), then $\sqrt 2 - \varepsilon$ (Khot, Minzer & Safra 2018).

UGC-hard: $2 - \varepsilon$ (Khot & Regev 2008).

Max-Cut

Goemans–Williamson SDP: $\alpha_{GW} \approx 0.87856$ (unit 25).

NP-hard above $16/17 \approx 0.941$ (Håstad 2001; Trevisan et al. 2000).

UGC-hard above $\alpha_{GW}$ (Khot, Kindler, Mossel & O'Donnell 2007).

Unique Games Conjecture (Khot 2002)

For every $\varepsilon > 0$ there is a $k$ such that, for constraint problems on a graph whose edge constraints are permutations of $k$ labels, it's NP-hard to tell instances where $1 - \varepsilon$ of the constraints can be satisfied from those where at most $\varepsilon$ can. Open. If true, the simple algorithms for vertex cover and Max-Cut, and SDP rounding for every CSP (Raghavendra 2008), are optimal.

section 4

The nine problems

Unit 00's list, with Part V's answers.

22 · checkpoint

Best ratio, best hardness, the gap

problem (unit 00)	best polynomial guarantee	hardness (unless P = NP)	gap
assignment	exact (Hungarian, unit 15)	in P	none
knapsack	FPTAS (Ibarra & Kim 1975)	weakly NP-hard: no exact polynomial algorithm	none
set cover	$\ln n + 1$ greedy (unit 23)	$(1 - \varepsilon)\ln n$	closed
vertex cover	2 (units 23, 24)	$\sqrt 2 - \varepsilon$; $2 - \varepsilon$ under UGC	$\sqrt 2$ vs 2
Max-Cut	0.878 (GW, unit 25)	16/17; $\alpha_{GW}$ under UGC	0.878 vs 0.941
TSP, metric	$3/2 - 10^{-36}$ (Karlin, Klein & Oveis Gharan 2021)	123/122 (Karpinski, Lampis & Schmied 2015)	wide open
TSP, Euclidean (our generator)	PTAS (Arora 1998; Mitchell 1999)	strongly NP-hard, so no FPTAS	none
bin packing	OPT + O(log OPT) (Hoberg & Rothvoss 2017); FFD $\frac{11}{9}$OPT + $\frac69$	absolute 3/2	OPT + 1?
graph colouring	$O\big(n (\log\log n)^2 / \log^3 n\big)$ (Halldórsson 1993)	$n^{1-\varepsilon}$ (Zuckerman 2007)	lower-order
job shop, makespan	polylogarithmic in $m$ (Shmoys, Stein & Wein 1994, and refinements)	5/4 (Williamson et al. 1997)	wide open

The table has ten rows for nine problems because TSP splits: our generator makes (rounded) Euclidean instances, which have a PTAS, while the general metric case is the famous open problem. Karlin, Klein and Oveis Gharan's improvement on Christofides is tiny in value and enormous in significance: it broke a 45-year barrier.

For job shop the known super-constant hardness results need assumptions stronger than P ≠ NP and apply to variants (Mastrolilli & Svensson 2011). Treat the row as "somewhere between 5/4 and polylog, and nobody knows where".

Colouring's "lower-order" gap is between $n^{1-\varepsilon}$ for every $\varepsilon$ and a ratio of $n$ divided by polylog factors: essentially nothing is achievable, and that's known.

FFD's bound is Dósa's tight version (2007). The asymptotic results are why bin packing is easy in practice (unit 27): the additive loss is at most a few bins.

22 · lab

Lab 22 — what you build

about 2 hours · uv run co lab 22

3-SAT → vertex cover: the gadget and both solution maps.
Clause splitting: any CNF to exactly-3-CNF, and assignment extension.
Vertex cover → set cover, and the composed chain.
Knapsack: exact DP by value, then the FPTAS.

uv run co test 22
uv run co then 22     # both sides by CP-SAT · optima agree · FPTAS loss vs guarantee

22 · checkpoint

Done when

checkpoint

All four lab steps are ✓.
For each of unit 00's nine problems you can state the best known approximation ratio and hardness bound, and say where the gap is still open.
You can say why knapsack has an FPTAS and bin packing can't (unless P = NP), and why bin packing still packs well in practice.

Reading

Garey & Johnson, Computers and Intractability (1979), ch. 3–4, 6.
Williamson & Shmoys, The Design of Approximation Algorithms (2011), ch. 3, 16.
Arora & Barak, Computational Complexity (2009), ch. 11, 22.
Khot, "On the Unique Games Conjecture", CCC 2010.

23 combinatorial approximation: greedy, Christofides, LPT · 24 LP rounding and primal–dual.

Hardness, and the shape of what is possible

The route

Reductions

Karp's chain

3-SAT → vertex cover, and back

The gadget, on the example

What a reduction preserves, and what it doesn't

Numbers

Weak and strong NP-hardness

The knapsack FPTAS (Ibarra & Kim 1975)

The FPTAS, on four items

The hierarchy

FPTAS ⊂ PTAS ⊂ APX ⊂ log-APX ⊂ poly-APX

The 3/2 gap, on two instances

The PCP theorem: where approximation hardness comes from

Three thresholds, and the Unique Games Conjecture

The nine problems

Best ratio, best hardness, the gap

Lab 22 — what you build

Done when

Reading

Next