Part III · trunk B · combinatorial algorithms · spine
Shortest paths in a DAG of states; knapsack and what "pseudo-polynomial" really means; Held–Karp; trees, treewidth and Courcelle; the vocabulary of parameterised complexity.
needs 15~2 h slides + 3 h lablab: uv run co lab 16
16 · overview
DP as shortest path in a DAG; knapsack; pseudo-polynomial vs strongly NP-hard.
Bitmask DP; Held–Karp's $O(n^2 2^n)$ and where it stands 60 years later.
Trees, tree decompositions, elimination orderings, DP on bags, Courcelle's theorem.
FPT, XP, kernels: the honest way to say "hard but not hopeless".
Words with a dotted underline are glossary terms: click one for its definition. G lists every term in the unit, N shows the notes. Optional extra: interactive figures, linked from the slides they support by a small ▸ chip; skip them freely.
section 1
Merge partial solutions that have the same future.
16 · states
Time = (number of states) × (transitions per state). Correctness = optimal substructure: an optimal path's prefixes are optimal paths to their states.
| problem | state | states |
|---|---|---|
| knapsack | items so far, capacity used | $nC$ |
| Held–Karp | cities visited, current city | $n 2^n$ |
| tree VC | vertex, in or out | $2n$ |
| decomposition VC | bag, subset of it | $\le n\,2^{w+1}$ |
| TSP, naively | the whole path | $n!$ |
The last row is why the others matter. Going from paths to (set, endpoint) merges all orderings of the same set.
16 · states
$n$ items, capacity $C$: $O(nC)$ time. Knapsack is NP-hard. So have we shown P = NP?
$C$ is written in $\log_2 C$ bits. $O(nC) = O(n \cdot 2^{\log C})$ is exponential in the input length. It is polynomial only in the numeric value: a pseudo-polynomial algorithm.
Lab then, 50 items, weights scaled by $10^k$:
| capacity | your DP s | HiGHS s |
|---|---|---|
| 1 032 | 0.000 | 0.027 |
| 103 947 | 0.013 | 0.020 |
| 10 404 313 | 2.44 | 0.019 |
Knapsack is weakly NP-hard: hard only when the numbers are huge. Bin packing and TSP are strongly NP-hard: still hard with numbers bounded by a polynomial in $n$. So they have no pseudo-polynomial algorithm unless P = NP.
16 · states · example
Capacity $C = 6$. Items (weight, value): $1{:}\ (2, 3)$, $\ 2{:}\ (3, 4)$, $\ 3{:}\ (4, 5)$, $\ 4{:}\ (5, 9)$. Row $i$ = best value using items $1..i$; column = capacity.
| items | 0 | 1 | 2 | 3 | 4 | 5 | 6 |
|---|---|---|---|---|---|---|---|
| none | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| 1 | 0 | 0 | 3 | 3 | 3 | 3 | 3 |
| 1–2 | 0 | 0 | 3 | 4 | 4 | 7 | 7 |
| 1–3 | 0 | 0 | 3 | 4 | 5 | 7 | 8 |
| 1–4 | 0 | 0 | 3 | 4 | 5 | 9 | 9 |
Answer: 9, by item 4 alone. $5 \times 7 = 35$ cells, each one max of two numbers.
Row 1–2, capacity 5: skip item 2 → 3; take it → row above at $5 - 3 = 2$, plus 4 → $3 + 4 = 7$. Keep 7.
Row 1–3, capacity 6: skip → 7; take item 3 → row above at $6 - 4 = 2$, plus 5 → 8. Keep 8.
Row 1–4, capacity 6: skip → 8; take item 4 → row above at $6 - 5 = 1$, plus 9 → 9. Keep 9.
The state "capacity used" forgets which items filled it. That forgetting is the merge that makes this $O(nC)$ instead of $2^n$.
section 2
From $n!$ to $n^2 2^n$.
16 · subsets
The Held–Karp DP: $S \subseteq \{1, \dots, n-1\}$, $k \in S$. Tour length $= \min_k D(\text{all}, k) + d_{k0}$.
No $O((2 - \varepsilon)^n)$ algorithm is known for general weighted TSP. For undirected Hamiltonicity, Björklund (2014) broke $2^n$ with algebraic methods.
Lab then, Euclidean, seconds:
| n | Held–Karp | B&B (07) MTZ | lazy DFJ (09) | SCIP MTZ |
|---|---|---|---|---|
| 8 | 0.001 | 0.49 | 0.023 | 0.29 |
| 12 | 0.003 | 5.6 | 0.006 | 0.043 |
| 14 | 0.008 | 6.1 | 0.014 | 1.12 |
| 16 | 0.049 | — | 0.013 | 0.68 |
| 18 | 0.144 | — | 0.031 | 5.15 |
| 20 | 0.787 | — | 0.029 | 0.28 |
"—": unit 07's pure-Python B&B hit 20 000 nodes on the weak MTZ formulation.
16 · subsets · example
Distances (symmetric): $d_{01} = 2,\ d_{02} = 9,\ d_{03} = 10,\ d_{12} = 6,\ d_{13} = 4,\ d_{23} = 3$. Start and end at city 0.
| $S$ | end $k$ | $D(S, k)$ | came from |
|---|---|---|---|
| $\{1\}, \{2\}, \{3\}$ | 1, 2, 3 | 2, 9, 10 | city 0 |
| $\{1,2\}$ | 1 / 2 | 15 / 8 | 2 / 1 |
| $\{1,3\}$ | 1 / 3 | 14 / 6 | 3 / 1 |
| $\{2,3\}$ | 2 / 3 | 13 / 12 | 3 / 2 |
| $\{1,2,3\}$ | 1 / 2 / 3 | 16 / 9 / 11 | 3 / 3 / 2 |
$D(\{1,2,3\}, 2)$: come from 1 via $D(\{1,3\}, 1) + d_{12} = 14 + 6 = 20$, or from 3 via $D(\{1,3\}, 3) + d_{32} = 6 + 3 = 9$. Keep 9.
Close the tour: $16 + d_{10} = 18$, $\ 9 + d_{20} = 18$, $\ 11 + d_{30} = 21$. Optimum 18.
Read it back: end at 1, came from 3, which came from 2, which came from 0: tour $0\!-\!2\!-\!3\!-\!1\!-\!0$, $9 + 3 + 4 + 2 = 18$.
12 table entries instead of $3! = 6$ orderings here; at $n = 20$ it is $19 \cdot 2^{19} \approx 10^7$ entries instead of $19! \approx 10^{17}$.
section 3
How far from a tree is this graph?
16 · treewidth
The tree DP for vertex cover, rooted anywhere:
$$\begin{aligned} \mathrm{in}(v) &= w_v + \textstyle\sum_{c} \min(\mathrm{in}(c), \mathrm{out}(c)) \\ \mathrm{out}(v) &= \textstyle\sum_{c} \mathrm{in}(c) \end{aligned}$$$O(n)$. The state is (vertex, in or out) because a subtree talks to the rest of the tree only through its root.
Lab step 3 runs it on a 5 000-vertex path, so it must not recurse. Python's recursion limit is 1 000.
A tree DP works because removing a vertex splits the tree into independent parts. If a graph splits into independent parts by removing small sets of vertices, recursively, the same DP works with the small set as the state.
"Small" is the treewidth, and the state space is $2^{\text{set size}}$ for vertex cover, or $q^{\text{set size}}$ for $q$-colouring.
The same argument makes independent set, dominating set, colouring and Hamiltonicity linear-time on trees, and FPT in treewidth.
16 · treewidth · example
Vertex weights $w$. Filled: the optimal cover $\{1, 2\}$, weight 6.
| vertex | in | out | why |
|---|---|---|---|
| leaves 3, 4, 5, 6 | 1, 1, 2, 2 | 0 | in = own weight |
| 1 (children 3, 4) | 2 | 2 | in: $2 + 0 + 0$; out: $1 + 1$ |
| 2 (children 5, 6) | 4 | 4 | in: $4 + 0 + 0$; out: $2 + 2$ |
| root 0 | 9 | 6 | in: $3 + 2 + 4$; out: $2 + 4$ |
Root: out (6 < 9). So both children must be in: 1 and 2. Below them the parents are in, so each leaf takes its better value, out (0). Cover $\{1, 2\}$, weight $2 + 4 = 6$.
16 · treewidth
Every edge sits in a bag, and the bags holding each vertex are contiguous (5 is in the top three, 2 in the bottom three).
A tree decomposition: a tree $T$ with a bag $B_t \subseteq V$ per node, such that
Width $= \max |B_t| - 1$. Treewidth $\mathrm{tw}(G)$ = the minimum width over all decompositions.
Trees: 1. Cycles: 2. Series–parallel graphs: $\le 2$. The $k \times k$ grid: $k$. $K_n$: $n - 1$. The lab's $k$-trees: exactly $k$.
Property 2 is what makes a bag a separator: the parts of the graph below and above it share only the bag's vertices.
16 · treewidth
$\mathrm{tw}(G) = \min$ over elimination orderings of the largest neighbourhood at elimination time. Every decomposition comes from some ordering.
Computing treewidth is NP-hard (Arnborg, Corneil & Proskurowski 1987). For fixed $k$, width $\le k$ is decidable in linear time (Bodlaender 1996), with constants too large to run.
Min-degree: eliminate the vertex of smallest current degree. Min-fill: the one adding fewest fill edges. Min-fill is exact on chordal graphs (a simplicial vertex adds no fill); both are exact on $k$-trees; both are often close elsewhere.
A star eliminated centre-first turns the leaves into a clique, giving width $n - 1$. Leaves first gives width 1. Lab step 4 tests both.
networkx ships both heuristics (treewidth_min_degree, treewidth_min_fill_in). Exact solvers from the PACE 2016–17 challenges handle graphs of hundreds of vertices.
16 · treewidth · example
Vertices 0–4, edges $01, 02, 12, 13, 23, 34$: triangles $\{0,1,2\}$ and $\{1,2,3\}$ sharing edge 12, and a pendant edge 34.
| eliminate | neighbours then | bag | fill added |
|---|---|---|---|
| 4 (degree 1) | 3 | $\{3,4\}$ | none |
| 0 (degree 2) | 1, 2 | $\{0,1,2\}$ | none (12 exists) |
| 1 | 2, 3 | $\{1,2,3\}$ | none (23 exists) |
| 2, then 3 | 3; none | $\{2,3\}$, $\{3\}$ | none |
Largest bag 3: width 2. No fill at all, because the graph is chordal.
Eliminating 1 first: its neighbours 0, 2, 3 must become a clique. Edges 02 and 23 exist, but 03 is added. Bag $\{0,1,2,3\}$: width 3.
Same graph, same algorithm, one bad choice at the start: the DP over this decomposition would use $2^4$ states per bag instead of $2^3$.
Hanging bags together: each bag attaches to the bag of its earliest-eliminated remaining neighbour. From min-degree: $\{3,4\} - \{3\}$, $\{0,1,2\} - \{1,2,3\} - \{2,3\} - \{3\}$.
16 · treewidth
The DP over a decomposition. Root it. For each bag $t$ and each $S \subseteq B_t$ covering the edges inside $B_t$:
$$\begin{aligned} T_t(S) &= w(S) + \textstyle\sum_{c\ \text{child}} \beta_c(S \cap B_c) \\ \beta_c(P) &= \min_{S' \subseteq B_c,\ S' \cap B_t = P} T_c(S') - w(P) \end{aligned}$$Group each child's table by its restriction to the shared vertices once, then each parent entry costs $O(1)$ per child. Total $\approx \sum_t 2^{|B_t|}$, times small factors.
Every restriction has some valid child entry: put all non-shared vertices in the cover.
Lab then, partial $k$-trees on 150 vertices, calibrated once at $k = 4$:
| width | $\sum 2^{|B_t|}$ | DP s | predicted | HiGHS s |
|---|---|---|---|---|
| 5 | 5 406 | 0.009 | 0.011 | 0.022 |
| 7 | 19 166 | 0.024 | 0.040 | 0.042 |
| 9 | 61 934 | 0.070 | 0.128 | 0.054 |
| 11 | 224 990 | 0.250 | 0.465 | 0.073 |
The prediction is within 2× at every width. HiGHS doesn't care about the width on these instances.
16 · treewidth
Every graph property expressible in monadic second-order logic (MSO₂: quantify over vertices, edges, and sets of them) can be decided in time $f(k, |\varphi|) \cdot n$ on graphs of treewidth $\le k$.
Arnborg, Lagergren & Seese (1991) extended it to optimisation: find a minimum or maximum set satisfying an MSO formula.
Vertex cover in MSO:
$$\exists S\ \forall u\,\forall v\ \bigl(\mathrm{adj}(u,v) \rightarrow u \in S \lor v \in S\bigr)$$3-colourability, Hamiltonicity, connectivity and dominating set are all expressible too, so all are linear-time for bounded treewidth.
$f$ is a tower of exponentials whose height grows with the formula's quantifier alternation. Courcelle tells you a DP exists. You still write the $2^w$ one by hand.
section 4
Hard, but not hopeless.
16 · parameters
A polynomial-time reduction to an equivalent instance of size $g(k)$. A decidable problem is FPT iff it has a kernel.
why: notes (N)
Buss's kernel for vertex cover $\le k$: a vertex of degree $> k$ must be in the cover, so take it and set $k \leftarrow k - 1$. Once none remain, more than $k^2$ edges means "no". Result: $O(k^2)$ edges. The LP (unit 24's half-integrality) gives a $2k$-vertex kernel.
Kernels are exactly what presolve does in a MIP solver (unit 27), with a guarantee.
16 · parameters · example
Is there a vertex cover of size $\le k = 3$? Graph on 7 vertices: a star $01, 02, 03, 04$ and a path $15, 56$.
| rule | what happens | $k$ |
|---|---|---|
| degree $> k$? | vertex 0 has degree 4 > 3: take it, delete its edges | 2 |
| degree $> k$? | left: $15, 56$; highest degree 2, not > 2 | 2 |
| edges $> k^2$? | 2 edges $\le 4$: keep going | 2 |
| isolated vertices | drop 2, 3, 4 | 2 |
Kernel: the path $1\!-\!5\!-\!6$, asking for a cover of size $\le 2$.
If 0 were left out, all 4 of its neighbours would have to be in, and $4 > 3 = k$. So any cover of size $\le 3$ contains 0.
$\{5\}$ covers the path. Put back the forced vertex: cover $\{0, 5\}$, size 2 $\le 3$. Yes.
The kernel's size depends only on $k$ (at most $k^2$ edges), however many leaves the star had.
16 · lab
uv run co test 16
uv run co then 16 # TSP race vs units 07, 09 and SCIP · knapsack scaling · runtime vs width16 · checkpoint
Kleinberg & Tardos, ch. 6 and §10.4.
Cygan et al., Parameterized Algorithms, ch. 2 (kernels), 7 (treewidth).
Held & Karp, J. SIAM 10 (1962).
Bodlaender, "A tourist guide through treewidth" (1993).
10 column generation, with your knapsack as the pricer · 11 Lagrangian relaxation · 22 the knapsack FPTAS.