Part III · trunk B · combinatorial algorithms · spine

Matroids and the exact reach of greedy

Greedy is optimal for every choice of weights exactly when the feasible sets form a matroid, which answers "when does this heuristic work?" completely.

needs 00~2 h slides + 2½ h lablab: uv run co lab 13

Trunk B is independent of Parts I–II: you can start it the day after unit 00. It is about the problems that are genuinely easy, and the structural reasons why. Unit 06 (total unimodularity) is the polyhedral explanation of the same islands, and you'll see the matroid polytope make that link here.

This unit opens the trunk with the cleanest theorem in the subject: a characterization of when the simplest possible algorithm is exactly right.

How to read this deck

One five-vertex graph shows greedy (Kruskal) and the exchange axiom; a three-edge path shows matroid intersection's augmenting path; four sets show greedy coverage falling short. All were checked with the lab's reference oracles. The two directions of Rado–Edmonds are given as ideas, with the arguments in the notes.

13 · overview

The route

1 · greedy and matroids

Independence systems, the matroid axioms, the Rado–Edmonds theorem, and Kruskal as a special case.

2 · two matroids

Matroid intersection: bipartite matching and more, by augmenting paths. Three matroids: NP-hard.

3 · beyond matroids

Submodularity, and greedy's 1 − 1/e guarantee when it is no longer exact.

getting around

Words with a dotted underline are glossary terms: click one for its definition. G lists every term in the unit, N shows the notes. Optional extra: interactive figures, linked from the slides they support by a small ▸ chip; skip them freely.

section 1

Greedy and matroids

An axiom that makes the obvious algorithm correct.

13 · greedy

Independence systems, and one algorithm for all of them

independence system

A ground set $E$ and a family $\mathcal I$ of "independent" subsets, containing $\emptyset$ and closed under taking subsets.

greedy (max weight)

Sort $E$ by decreasing weight. Take each element with positive weight if the chosen set stays independent.

system	greedy
forests of a graph	optimal (Kruskal)
at most $k$ elements	optimal
linearly independent vectors	optimal
matchings of a graph	can fail
knapsack-feasible sets	can fail

The lab's greedy is one function, and the independence oracle decides which row you are in.

Matchings failing: path with edge weights 2, 3, 2. Greedy takes the 3 and blocks both ends: 3. Taking both ends gives 4.

explore interactively

13 · greedy

The matroid axioms

matroid

$\emptyset \in \mathcal I$
$B \in \mathcal I,\ A \subseteq B \Rightarrow A \in \mathcal I$ (hereditary)
$A, B \in \mathcal I,\ |A| < |B| \Rightarrow \exists\, b \in B \setminus A$ with $A + b \in \mathcal I$ (exchange)

Consequence of exchange: all maximal independent sets (bases) of any subset have the same size, its rank $r(S)$.

matroid	independent sets
graphic	forests of a graph
uniform $U_{k,n}$	sets of size $\le k$
partition	at most $c_b$ elements from each block $b$
linear	linearly independent sets of vectors
transversal	sets matchable into a bipartite graph's other side

Linear algebra is where the axioms come from: Whitney (1935) abstracted exactly the exchange property of bases of a vector space.

13 · greedy · example

Greedy and exchange, on one graph

Edge weights on the edges. Thick: the maximum-weight forest greedy finds, weight 18.

greedy on the graphic matroid (Kruskal)

edge	weight	still a forest?
34	6	yes, take
23	5	yes, take
01	4	yes, take
12	3	yes, take: now spanning
13, 24, 02	2, 2, 1	no: each closes a cycle

the exchange axiom, checked

$A = \{01, 12\}$ (2 edges), $B = \{02, 13, 34\}$ (3 edges). Try each edge of $B$ in $A$: $02$ closes the triangle 0-1-2 ✗; $13$ ✓; $34$ ✓. At least one works, as the axiom promises.

explore interactively

13 · greedy

Rado–Edmonds: greedy works exactly on matroids

theorem (Rado 1957; Edmonds 1971)

For an independence system $(E, \mathcal I)$: greedy finds a maximum-weight independent set for every weight function if and only if $(E, \mathcal I)$ is a matroid.

matroid ⇒ greedy optimal, the idea

If some better set had a heavier $i$-th element than greedy's, exchange would hand greedy an element it could have taken earlier. Greedy never skips anything it could use.

full proof in the notes (N)

not a matroid ⇒ weights that fool greedy

Find $A, B$ where exchange fails, give $A$ slightly more weight than $B$. For matchings on the path $a\!-\!b\!-\!c\!-\!d$: $A = \{bc\}$, $B = \{ab, cd\}$, neither edge of $B$ fits next to $bc$. Weights $2, 3, 2$ are exactly such weights: greedy 3, optimum 4.

Lab step 3 finds the $(A, B)$, an exchange witness. Step 2's greedy on the graphic oracle is Kruskal, matched against networkx on 25 graphs.

Full proof

Matroid ⇒ greedy optimal. Let greedy pick $g_1, g_2, \dots$ in order, and let $O$ be optimal, sorted by weight. If $w(o_i) > w(g_i)$ at the first index where they differ, exchange between $\{g_1..g_{i-1}\}$ and $\{o_1..o_i\}$ gives an element of weight $\ge w(o_i) > w(g_i)$ that greedy could have taken before $g_i$: a contradiction.

Not a matroid ⇒ some weights fool greedy. Take $A, B$ violating exchange with $|A| < |B|$. Give $A$ weight $1 + \varepsilon$, $B \setminus A$ weight 1, everything else 0. Greedy takes all of $A$ first, then gets stuck at $|A|(1+\varepsilon)$, while $B$ is worth at least $|B|$.

The second direction is constructive and practical: it tells you how to build the failing instance for any proposed greedy algorithm whose feasible sets aren't a matroid. That is the curriculum's checkpoint.

For the weights: with $|A| < |B|$ and $\varepsilon$ small enough that $|A|(1+\varepsilon) < |B|$, greedy's total is below $|B|$. After taking $A$, no element of $B \setminus A$ can be added (by the choice of $A$, $B$), and the rest have weight 0.

Beyond cardinality: the minimum-weight base problem (spanning trees) is the same theorem with weights negated and "stop at non-positive" removed.

13 · greedy

The matroid polytope

Edmonds 1970

The convex hull of incidence vectors of independent sets, the matroid polytope, is

$$\begin{aligned} P(M) = \{x \ge 0 :\ & \textstyle\sum_{e \in S} x_e \le r(S) \\ & \text{for all } S \subseteq E\}\end{aligned}$$

Greedy's output is an optimal vertex, and a dual solution built from the same order proves it by LP duality (unit 03): the greedy dual certificate.

Why unit 06 and unit 09 meet here:

The system is TDI: integrality comes from the dual, not from total unimodularity.
There are exponentially many constraints, but separation over $P(M)$ is submodular function minimization (min over $S$ of $r(S) - x(S)$), which is polynomial.
So by GLS, optimizing over two matroids' polytopes together is polynomial too: that is intersection, next.

section 2

Two matroids

Greedy stops working, and augmenting paths start.

13 · intersection

Matroid intersection

Schematic. Arcs out of I-vertices are M1-exchanges; arcs into them are M2-exchanges. A shortest source→sink path is flipped, and |I| grows by one.

Matroid intersection: the largest set independent in both $M_1$ and $M_2$.

Edmonds' min–max theorem $$\max |I| = \min_{A \subseteq E}\; r_1(A) + r_2(E \setminus A)$$

Bipartite matching: two partition matroids, "one edge per left vertex" and "one edge per right vertex". König's theorem is this min–max.
Rainbow spanning forests: graphic ∩ "one edge per colour".
Arborescences in digraphs, and more.

Algorithm (lab step 4): the exchange graph, then a shortest augmenting path, then flip. The next slide does one. Polynomial in the number of oracle calls.

The exchange graph arcs: $x \to y$ ($x \in I$, $y \notin I$) when $I - x + y$ is independent in $M_1$; $y \to x$ when $I - x + y$ is independent in $M_2$. Sources are $y$ with $I + y \in \mathcal I_1$, sinks are $y$ with $I + y \in \mathcal I_2$. A source→sink path alternates out, in, …, out, and flipping it adds one element. Shortest paths are required. The correctness proof shows the flipped set stays independent in each matroid using a "unique perfect matching" lemma, and that lemma only applies to shortest paths. Hint rung 3 says why.

When no path exists, the set $A$ of vertices reachable from the sources gives the min–max certificate: $|I| = r_1(A) + r_2(E \setminus A)$. It is the same shape as max-flow min-cut (unit 14): the unreachable set is the cut.

The figure is a schematic of the structure, not a lab run; the next slide is a real one.

13 · intersection · example

One augmenting path, on three edges

Bipartite graph: left $\ell_0, \ell_1$, right $r_0, r_1$, edges $e_0 = \ell_0 r_0$, $e_1 = \ell_1 r_0$, $e_2 = \ell_1 r_1$ (a path). $M_1$: one edge per left vertex. $M_2$: one per right vertex. A common independent set is a matching.

Start from $I = \{e_1\}$. It is maximal: $e_0$ clashes at $r_0$, $e_2$ at $\ell_1$. Greedy-style "add what fits" is stuck at size 1.

building the exchange graph

question	answer
sources: $I + y \in M_1$?	$e_0$ ✓ ($\ell_0$ free); $e_2$ ✗
sinks: $I + y \in M_2$?	$e_2$ ✓ ($r_1$ free); $e_0$ ✗
$e_1 \to y$ if $I - e_1 + y \in M_1$	$e_1 \to e_0$, $\ e_1 \to e_2$
$y \to e_1$ if $I - e_1 + y \in M_2$	$e_0 \to e_1$, $\ e_2 \to e_1$

the path, and the flip

Source $e_0 \to e_1 \to e_2$ sink. Flip membership along it: $e_0$ in, $e_1$ out, $e_2$ in.

$I = \{e_0, e_2\}$: one edge at each left vertex, one at each right vertex. Size 2, and no larger matching exists on 2 + 2 vertices.

This is the augmenting path of bipartite matching (unit 15), found by asking two independence oracles instead of looking at the graph.

explore interactively

13 · intersection

Two is polynomial, three is NP-hard

three matroids

A directed Hamiltonian path is a set of $n - 1$ arcs that is independent in three matroids at once:

graphic (no cycle, ignoring directions);
partition: at most one arc into each vertex;
partition: at most one arc out of each vertex.

So a size-$(n-1)$ common independent set exists iff a Hamiltonian path does. That is NP-complete: three-matroid intersection is NP-hard.

This is where the islands end. The sequence runs:

one matroid: greedy;
two: augmenting paths, the polytope still integral;
three: NP-hard, and the TSP of unit 09 lives here;
matroid parity (matching's generalization): polynomial for linear matroids (Lovász), intractable in the oracle model.

Unit 22 turns reductions like this one into code.

section 3

Beyond matroids: submodularity

When greedy is no longer exact, but still provably close.

13 · submodular

Diminishing returns, and 1 − 1/e

submodular

$f : 2^E \to \mathbb R$ with $\;f(A + e) - f(A) \ge f(B + e) - f(B)\;$ for $A \subseteq B$, $e \notin B$: the marginal gain shrinks.

Adding $e$ helps a small set at least as much as a big one. Matroid rank functions and coverage functions are submodular.

Nemhauser, Wolsey & Fisher (1978)

For monotone submodular $f$ and a cardinality limit $k$, greedy (add the element of largest marginal gain) achieves $f(G) \ge (1 - \tfrac1e)\,\mathrm{OPT}$.

maximum coverage, k = 2

Sets $\{1,2,3,4\}$, $\{1,2,5\}$, $\{3,4,6\}$, $\{7\}$.

Greedy: takes $\{1,2,3,4\}$ (4 new), then any other set adds only 1: covers 5.

Optimum: $\{1,2,5\} + \{3,4,6\}$ covers 6. Ratio $\tfrac56 = 0.83$, above the guarantee for $k = 2$, $1 - (\tfrac12)^2 = 0.75$.

why greedy is never worse than 1 − 1/e: notes (N)

and it's tight

For max coverage, no polynomial algorithm beats $1 - 1/e$ unless P = NP (Feige 1998).

Proof idea

Each step, the $k$ optimal elements together would add at least $\mathrm{OPT} - f(G_i)$, so the best single one adds at least $\frac1k(\mathrm{OPT} - f(G_i))$. The gap shrinks by a factor $(1 - \frac1k)$ per step, and $(1 - \frac1k)^k \le \frac1e$.

The example. The biggest set looks best first, but it overlaps both halves of the optimal pair. The reference greedy_coverage picks indices 0 then 1 (a tie with 2 and 3, broken by lowest index), covering 5; brute force over the six pairs gives 6.

Tightness of the analysis: there are coverage instances on which greedy achieves exactly $(1 - (1-1/k)^k)\,\mathrm{OPT}$, which tends to $1 - 1/e$. They are built with nested sets of carefully decreasing sizes. Feige's hardness result says no cleverer polynomial algorithm does better in the worst case, so greedy is the best possible algorithm for max coverage in that sense.

Submodular maximization is everywhere in practice: sensor placement, influence maximization in networks, document summarization, active learning. In all of them greedy, or a lazy-evaluation version of it, is the default method because of this theorem.

13 · submodular

The guarantee against the typical case

Lab then: 40 random sets over 120 elements, exact optimum by HiGHS MIP.

$k$	mean ratio	worst ratio	guarantee
3	0.987	0.950	0.632
6	0.992	0.944	0.632
10	0.980	0.926	0.632

read it as

A worst-case guarantee is a floor, not a forecast. Random instances rarely come near the tight examples, which have to be constructed.

Unit 23 makes the same point for set cover's $H_n$ bound, and unit 28 turns "what did you actually measure?" into method.

13 · lab

Lab 13 — what you build

about 2½ hours · uv run co lab 13 · oracles, not data structures

Four matroid oracles: graphic, uniform, partition, linear.
One greedy, which is Kruskal on the graphic oracle.
An axiom checker that returns the exchange counterexample.
Matroid intersection by shortest augmenting paths, which gives bipartite matching.
Greedy max coverage, checked against its 1 − 1/e bound.

uv run co test 13
uv run co then 13     # greedy vs Kruskal · intersection vs Hopcroft–Karp · coverage ratios vs MIP optimum

13 · checkpoint

Done when

checkpoint

All five lab steps are ✓.
Shown a greedy algorithm someone proposes, you can either name the matroid underneath it or construct the instance where it fails.
You can explain why three-matroid intersection is NP-hard while two is polynomial.

Reading

Korte & Vygen, ch. 13.
Schrijver, Combinatorial Optimization, vol. B, ch. 39–41 (consulted).
Krause & Golovin, "Submodular function maximization" (2014), §1–3.

14 network flows · then 15 matching and 16 DP. 23 uses this unit's MST inside Christofides.

Matroids and the exact reach of greedy

The route

Greedy and matroids

Independence systems, and one algorithm for all of them

The matroid axioms

Greedy and exchange, on one graph

Rado–Edmonds: greedy works exactly on matroids

The matroid polytope

Two matroids

Matroid intersection

One augmenting path, on three edges

Two is polynomial, three is NP-hard

Beyond matroids: submodularity

Diminishing returns, and 1 − 1/e

The guarantee against the typical case

Lab 13 — what you build

Done when

Reading

Next