Part V · where the trunks converge · branch

Semidefinite relaxations

Give each vertex a vector instead of a number, constrain only inner products, and round with a random hyperplane: Goemans–Williamson's 0.878, the Lovász theta function, and the hierarchies above them.

needs 24, 04~1½ h slides + 2¼ h lablab: uv run co lab 25

A branch unit: nothing later depends on it, but it finishes the story of relaxations. Unit 22 said Max-Cut's 0.878 is optimal under the Unique Games Conjecture. This unit builds the algorithm and measures it.

The curriculum lists "needs 21" for this unit, which looks like a slip for 04: the SDP is solved by an interior-point method, and unit 04 is where you met them. Nothing from lazy clause generation is used.

How to read this deck

The 5-cycle $C_5$ carries the unit: its SDP vectors are drawn and cut by one hyperplane, and its theta number is squeezed between two integers you can check by hand. A 2×2 matrix shows what PSD means before any of that. Every value was computed with the lab's reference solution (CLARABEL through cvxpy). The analysis of the constant 0.878 is on the slide as an idea, with the calculation in the notes.

25 · overview

The route

1 · the PSD cone

SDPs, vector programs, and what they can say that an LP can't.

2 · Goemans–Williamson

The relaxation, hyperplane rounding, α ≈ 0.878, and why it's tight.

3 · theta

Lovász's θ between the independence number and the clique cover number.

4 · hierarchies and cost

Lasserre / sum of squares; why SDPs stay a theoretical tool.

getting around

Words with a dotted underline are glossary terms: click one for its definition. G lists every term in the unit, N shows the notes. Optional extra: interactive figures, linked from the slides they support by a small ▸ chip; skip them freely.

section 1

The PSD cone

Linear programming over matrices.

25 · PSD

Semidefinite programs are vector programs

SDP $$\max\ \langle C, X \rangle \ \ \text{s.t.}\ \langle A_k, X \rangle = b_k,\ \ X \succeq 0$$

$X \succeq 0$, positive semidefinite: symmetric with all eigenvalues $\ge 0$. The PSD cone is convex, but not polyhedral. $\langle C, X\rangle$ is $\sum_{ij} C_{ij}X_{ij}$.

Cholesky, read as modelling

$X \succeq 0 \iff X = V V^\top$ for some $V$. So $X_{ij} = v_i \cdot v_j$ ($X$ is their Gram matrix): an SDP is a vector program, an optimisation over vectors $v_1, \dots, v_n$ with linear constraints on their inner products.

With $v_i$ restricted to $\{\pm 1\} \subset \mathbb{R}^1$ you get the integer problem. Allowing $\mathbb{R}^n$ is the relaxation.

Solvable to precision ε in polynomial time: interior-point methods with the barrier $-\log\det X$ (unit 04). The lab uses CLARABEL.
Duality as in unit 03, except strong duality needs a Slater point, and optimal values need not be attained.
Low rank exists (Barvinok 1995; Pataki 1998): some optimal $X$ has rank $r$ with $r(r+1)/2 \le$ the number of constraints.
Not an LP in disguise: for Max-Cut, no LP of size $n^{o(\log n / \log\log n)}$ beats ratio ½ (Chan, Lee, Raghavendra & Steurer 2013), while this SDP gets 0.878.

Lab step 1: solve, then factor $X$ back into unit vectors (eigendecomposition, clip tiny negative eigenvalues, renormalise).

25 · PSD · example

PSD, on 2×2 matrices

one that is PSD $$X = \begin{pmatrix} 1 & -\tfrac12 \\ -\tfrac12 & 1 \end{pmatrix}$$

Eigenvalues $\tfrac12$ and $\tfrac32$, both $\ge 0$. Factor it: $v_1 = (1, 0)$, $v_2 = (-\tfrac12, \tfrac{\sqrt3}2)$. Check $v_1 \cdot v_1 = 1$, $v_2 \cdot v_2 = \tfrac14 + \tfrac34 = 1$, $v_1 \cdot v_2 = -\tfrac12$ ✓.

Two unit vectors at 120°: the matrix is that picture.

one that is not $$Y = \begin{pmatrix} 1 & 2 \\ 2 & 1 \end{pmatrix}$$

Eigenvalues $3$ and $-1$. No vectors exist: two unit vectors have inner product between $-1$ and $1$, never 2.

For 2×2 with unit diagonal, PSD means exactly $|X_{12}| \le 1$: a cosine. In general PSD is "some unit vectors realise all these cosines at once", which is what makes it a constraint across all pairs.

explore interactively

The eigenvalues of $\begin{pmatrix}1 & c\\ c & 1\end{pmatrix}$ are $1 + c$ and $1 - c$, both nonnegative exactly when $|c| \le 1$. For $c = -\tfrac12$: $\tfrac12, \tfrac32$; for $c = 2$: $3, -1$.

The reference vectors_from_gram on X returns $(-0.5, -0.866)$ and $(-0.5, 0.866)$: a rotated copy of the vectors on the slide, still at 120°. Factorizations are unique only up to rotation, which is exactly why the random hyperplane has to be uniform.

Why the 3×3 case already goes beyond cosines: three unit vectors can't be pairwise at 180°. The matrix with 1 on the diagonal and $-1$ everywhere else has eigenvalue $-1$, so PSD forbids it, while each 2×2 submatrix is fine. This is the triangle inequality a cut must satisfy, and an LP over pairs cannot see it.

section 2

Goemans–Williamson

Vectors, a random hyperplane, and 0.878.

25 · GW

The relaxation and the rounding (1995)

$$\begin{aligned}\max\ & \textstyle\sum_{uv \in E} w_{uv} \frac{1 - v_u \cdot v_v}{2} \\ \text{s.t. } & \|v_u\| = 1\end{aligned}$$

A cut is $v_u \in \{\pm 1\}$: an edge contributes $w$ iff its ends differ. So SDP ≥ OPT.

The Max-Cut SDP. Round: draw $r$ with independent $N(0,1)$ entries (a uniform direction), and put $u$ on side $[v_u \cdot r \ge 0]$.

why 0.878, in words

A random line separates two vectors with probability equal to their angle over 180° (the separation probability). The SDP credits that edge a bit more, $(1 - \cos\theta)/2$. Edge by edge, rounding keeps at least $\alpha_{GW}$ $= 0.87856$ of the credit, and the credit totals at least OPT.

the calculation: notes (N)

Worst per-edge angle: 133.6°, ratio 0.87856. At 90° and 180° rounding recovers everything.

Full proof of the constant

$u, v$ are separated with probability $\theta_{uv}/\pi$, and the SDP counts $(1 - \cos\theta_{uv})/2$ for that edge. By linearity of expectation, $\mathbb{E}[\text{cut}] = \sum_{uv} w_{uv}\,\theta_{uv}/\pi \ge \min_\theta \frac{\theta/\pi}{(1-\cos\theta)/2} \cdot \sum_{uv} w_{uv} \frac{1 - \cos\theta_{uv}}{2} = \alpha_{GW} \cdot \mathrm{SDP} \ge \alpha_{GW} \cdot \mathrm{OPT}$, using nonnegative weights. The minimum is at $\theta \approx 133.6°$.

Why the projection argument works. The component of r in the plane spanned by v_u and v_v is a 2D Gaussian, so its direction is uniform on the circle. Only that component decides the two signs. The hyperplane's trace in the plane is the line perpendicular to it, and it separates the two vectors with probability θ/π.

This whole procedure is the Goemans–Williamson algorithm. The same SDP-and-hyperplane recipe gives 0.878 for Max-2SAT. Lewin, Livnat and Zwick (2002) reached 0.940 for Max-2SAT with a more careful rotation of the vectors before rounding. Raghavendra (2008) showed that, under UGC, a canonical SDP followed by a suitable rounding is optimal for every constraint satisfaction problem.

Weights must be nonnegative. With negative weights the per-edge bound fails for edges whose SDP angle is small, and the guarantee has to be restated.

25 · GW · example

Goemans–Williamson on the 5-cycle

The SDP optimum for $C_5$: $v_k$ at $k \times 144°$, so neighbours $v_k, v_{k+1}$ are 144° apart. Dashed: one hyperplane. Coloured: the side $\{v_0, v_2, v_3\}$.

quantity	value
SDP: $5 \times (1 - \cos 144°)/2$	4.5225
E[GW]: $5 \times 144/180$	4.0000
OPT (an odd cycle loses one edge)	4

one hyperplane, by hand

Sides: $\{v_0, v_2, v_3\}$ against $\{v_1, v_4\}$. Edges $01, 12, 34, 40$ cross; edge $23$ doesn't. 4 edges cut: an optimal cut.

Every hyperplane in general position cuts exactly 4 here, so rounding always gets OPT. The whole loss, $4.52 \to 4$, is between the SDP and the integers.

explore interactively

Why the vectors spread like that. The SDP wants neighbours as far apart as possible, but five vectors can't be pairwise opposite around an odd cycle. Walking round the cycle in steps of 144° closes up after five steps ($5 \times 144° = 720°$), so every edge gets the same large angle. The reference maxcut_sdp returns $X_{k,k+1} = -0.809 = \cos 144°$ and vectors_from_gram recovers a planar configuration; expected_cut gives 4.0000.

The hyperplane. Its normal points at 10°. $v_0$ (0°), $v_3$ (72°) and $v_2$ (288°) have positive inner product with it; $v_1$ (144°) and $v_4$ (216°) negative. Normals at 50° and 100° also give cuts of 4, with different sides.

Per edge the ratio is $(144/180) / ((1 - \cos 144°)/2) = 0.8 / 0.9045 = 0.8845$, the C₅ row of the table two slides on.

25 · GW

Measured: the bound ladder and the rounded cuts

Lab then: ratios to the optimum proved by CP-SAT, mean of 5 graphs, 200 hyperplanes each.

graph	edge LP	tri LP	SDP	E[GW]	worst	best
G(12, ½)	1.259	1.000	1.013	0.962	0.752	1.000
G(20, ½)	1.392	1.000	1.025	0.959	0.785	1.000
G(30, 0.3)	1.357	1.000	1.039	0.956	0.862	1.000
G(40, 0.2)	1.318	—	1.038	0.956	0.832	0.998

"worst" is the single worst hyperplane among 1 000 per row. "best" is the mean over graphs of the best of 200.

Expected GW: 0.956–0.962 of OPT, far above 0.878.
Single hyperplanes vary: one cut only 0.752 of OPT. The theorem is about the expectation.
The edge LP is useless: $x = \tfrac12$ makes every $z_e = 1$.
The triangle LP beat the SDP on these small graphs (exact to three decimals). But all $z = \tfrac23$ is feasible for it, so on large dense random graphs it's stuck at ⅔ of the total weight, while the optimum tends to ½. It also needs $4\binom{n}{3}$ constraints.

25 · GW

Why 0.878 is nonetheless tight

The three senses of tightness:

1 · the analysis is tight per edge

Rounding recovers $\frac{\theta/\pi}{(1-\cos\theta)/2}$ of each edge's SDP value. On C₅ the SDP vectors sit 144° apart: SDP 4.5225, E[GW] 4.0000, ratio 0.8845. Graphs whose vectors sit at 133.6° attain α (Karloff 1999).

2 · the relaxation has that gap

Feige & Schechtman (2002): graphs on points of a high-dimensional sphere, joined when their angle is about 133.6°, with OPT / SDP → α. No rounding of this SDP can beat α.

3 · no algorithm can, under UGC

Khot, Kindler, Mossel & O'Donnell (2007), with Majority is Stablest (Mossel, O'Donnell & Oleszkiewicz 2010): beating $\alpha_{GW} + \varepsilon$ is Unique-Games-hard. Without UGC, the NP-hardness threshold is 16/17 ≈ 0.941 (unit 22).

angle	rounding / SDP per edge
90°	1.0000
133.56° (worst)	0.8786
144° (C₅)	0.8845
180°	1.0000

The empirical 0.96 says random graphs rarely put their vectors at the bad angle. The theorem says someone can.

section 3

Theta

A number squeezed between two NP-hard ones.

25 · theta

The Lovász theta function (1979)

$$\begin{aligned}\vartheta(G) = \max\ & \textstyle\sum_{ij} X_{ij} \\ \text{s.t. } & \mathrm{tr}\,X = 1,\ X \succeq 0 \\ & X_{uv} = 0 \quad (uv \in E)\end{aligned}$$

sandwich $$\alpha(G) \le \vartheta(G) \le \chi(\bar G)$$

Independence number ≤ theta ≤ clique cover number, the colouring number of the complement. Both outer numbers are NP-hard; θ is computable to any precision.

Perfect graphs have $\alpha = \chi(\bar G)$, so θ computes both, and colourings too, in polynomial time (Grötschel, Lovász & Schrijver 1981). No combinatorial polynomial algorithm for colouring perfect graphs is known.

graph	α	θ	χ(Ḡ)
C₅	2	2.2361	3
C₇	3	3.3177	4
Petersen	4	4.0000	5
5-cube	16	16.0000	16
G(14, ½) #0	6	6.0000	6
G(14, ½) #2	5	5.1088	6

Why Lovász invented it: the Shannon capacity of C₅, the number of symbols per use a noisy 5-letter channel can send with zero error, is $\sqrt 5 = \vartheta(C_5)$.

The lower bound is a one-liner: for an independent set S, the matrix $\mathbf{1}_S \mathbf{1}_S^\top / |S|$ has trace 1, is zero on edges (no edge inside S), is PSD (rank one), and has entry sum |S|. The upper bound goes through the dual SDP, or through orthonormal representations. Lovász's original definition was a minimum over assignments of unit vectors to vertices, orthogonal on non-adjacent pairs.

The Shannon capacity problem: in k uses of the channel, how many length-k words can be sent so that no two can be confused? For C₅, two uses give 5 words (not 4), so the capacity is at least √5. Lovász proved θ is an upper bound that multiplies over graph products, and so settled a question open since Shannon's 1956 paper. The capacity of C₇ is still unknown.

The 5-cube is bipartite, hence perfect: α = θ = χ(Ḡ) = 16.

25 · theta · example

Theta of the 5-cycle, squeezed

independence number: 2

$\{0, 2\}$ has no edge. Three vertices of a 5-cycle always include two neighbours. So $\alpha(C_5) = 2$.

It feeds theta a feasible matrix: $X = \tfrac12 \mathbf 1_S \mathbf 1_S^\top$ with $S = \{0,2\}$ has trace 1, is 0 on every edge, is PSD, and its entries sum to 2.

clique cover number: 3

The cliques of C₅ are its edges and vertices. Cover the 5 vertices with $\{0,1\}, \{2,3\}, \{4\}$: 3 cliques. Two can't do it, since each covers at most 2 vertices. So $\chi(\bar C_5) = 3$.

The sandwich for $C_5$:

$$2 \;\le\; \vartheta(C_5) = \sqrt5 = 2.2361 \;\le\; 3$$

θ lands strictly between, so $C_5$ is not perfect: the two NP-hard numbers differ, and the SDP says exactly how far apart the truth could be.

Compare the 5-cycle's Max-Cut on the previous slides: the same odd cycle, a different SDP, and again a gap caused by oddness.

section 4

Hierarchies and cost

Ever-tighter, ever-bigger.

25 · hierarchies

Lasserre, sum of squares, and lower bounds against them

the idea (Lasserre 2001; Parrilo 2000)

The Lasserre hierarchy: GW gave each vertex a vector. Level $r$ gives a variable to every group of up to $2r$ vertices, standing for "how often are these all 1 together" under a pseudo-distribution, and asks the table of them, the moment matrix, to be PSD.

precise definition: notes (N)

Level 1 on Max-Cut is GW's SDP. Adding triangle inequalities is part of level 2.
Level $n$ is exact: the moment matrix of a true distribution over cuts.
Size $n^{O(r)}$, so solving level $r$ takes time $n^{O(r)}$.

The LP analogue is Sherali–Adams (1990): same lifting, with linear constraints only.

lower bounds

Grigoriev (2001): random 3-XOR needs level Ω(n) to refute. So does a simple knapsack instance, $\sum x_i = n/2$ for odd $n$.
Schoenebeck (2008): random 3-SAT likewise. Tulsiani (2009) turned such gaps into hardness-of-approximation-style lower bounds for problems like vertex cover.

The hierarchies are a model of "all reasonable convex relaxations". A lower bound against level Ω(n) rules out a whole family of algorithms, without any complexity assumption.

Precise definition

One variable $y_S$ for every set $S$ of at most $2r$ variables, standing for $\mathbb{E}[\prod_{i \in S} x_i]$ under a pseudo-distribution. Require the moment matrix $(y_{S \cup T})_{|S|,|T| \le r}$ to be PSD, plus localising constraints for each problem constraint.

Why "sum of squares": the dual of the level-r moment relaxation certifies $f \ge c$ by writing $f - c$ as a sum of squares of degree-r polynomials (a sum-of-squares certificate), modulo the constraints. A refutation of a formula is an SOS certificate that the number of satisfied clauses is below m. Grigoriev's result says no such certificate of low degree exists for 3-XOR, even though Gaussian elimination refutes it in polynomial time. So SOS is strong but not all-powerful.

This is also the modern lens on UGC. If UGC is true, constant-level SOS can't refute unique games in the hard regime. The best known algorithms for unique games (Arora, Barak & Steurer 2010, subexponential time) are SOS-based.

25 · cost

Convex, polynomial, and expensive

Lab then, Max-Cut on G(n, 0.3), seconds:

n	SDP (CLARABEL)	triangle LP (HiGHS)	CP-SAT proof
15	0.02	0.03	0.01
25	0.07	0.27	0.14
35	0.26	1.92	1.38
60	3.14	—	timeout (60 s)
100	18.8	—	—
150	150.4	—	—

The SDP has $n^2$ variables. Interior-point iterations factor dense matrices, and the measured time grows like $n^{3.5}$ to $n^5$ here.
An exact proof of optimality ran out of time at n = 60. The SDP bound is still available at 150, if slowly.
In practice: first-order and low-rank methods (Burer & Monteiro 2003) reach much larger n with modest accuracy, and SDP bounds inside branch and bound (BiqMac, Rendl, Rinaldi & Wiegele 2010) solve dense Max-Cut instances that LP-based solvers can't.

So "a theoretical tool" means: the ideas (vectors, rounding, hierarchies) are central, while off-the-shelf interior-point SDP solvers rarely are.

25 · lab

Lab 25 — what you build

about 2¼ hours · uv run co lab 25

The Max-Cut SDP in cvxpy, and vectors from its Gram matrix.
Hyperplane rounding, its exact expectation, and $\alpha_{GW}$.
LP bounds: the edge LP and the triangle LP.
Lovász theta, and the sandwich.
Local search, and GW end to end.

uv run co test 25
uv run co then 25     # bound ladder vs exact Max-Cut · tight angles · theta · seconds as n grows

25 · checkpoint

Done when

checkpoint

All five lab steps are ✓.
You can show empirically that your GW cuts average well above 0.878 of optimum, and explain why the guarantee is nonetheless tight, in all three senses.
You can say what θ computes exactly on perfect graphs, and why that's remarkable.

Reading

Goemans & Williamson, JACM 42 (1995).
Williamson & Shmoys (2011), ch. 6.
Lovász, IEEE Trans. Inf. Theory 25 (1979).
Laurent, "A comparison of the Sherali–Adams, Lovász–Schrijver and Lasserre relaxations", Math. OR 28 (2003).

26 metaheuristics, benchmarked honestly: from guarantees to measurements.

Semidefinite relaxations

The route

The PSD cone

Semidefinite programs are vector programs

PSD, on 2×2 matrices

Goemans–Williamson

The relaxation and the rounding (1995)

Goemans–Williamson on the 5-cycle

Measured: the bound ladder and the rounded cuts

Why 0.878 is nonetheless tight

Theta

The Lovász theta function (1979)

Theta of the 5-cycle, squeezed

Hierarchies and cost

Lasserre, sum of squares, and lower bounds against them

Convex, polynomial, and expensive

Lab 25 — what you build

Done when

Reading

Next