Part II · trunk A · integer programming · spine

Formulations, relaxations, and the integrality gap

The same integer solutions, described by different polyhedra, and why the choice decides whether a solver finishes.

needs 03~2 h slides + 2 h lablab: uv run co lab 05

Integer programming is where hardness enters the course. Everything in Part II is a strategy for exploiting the LP relaxation from Part I: tighten it (this unit and 08), branch on it (07), decompose it (10, 12), or dualize it (11).

This unit writes no algorithm. It is about modelling, and it makes the point that surprises most people coming from continuous optimization: two models with exactly the same feasible integer points can differ by orders of magnitude in solve time, and the difference can be predicted from their LP relaxations.

Convention: MILPs are minimization problems (as in colib.mip), so LP bounds are lower bounds.

How to read this deck

A facility-location instance small enough to solve by hand (2 facilities, 3 customers) runs through the unit: its integer optimum, the LP bound of each formulation, and the valid inequality that separates them. The large tables are from the lab's then; the small example is there to make them readable.

05 · overview

The route

1 · relaxations

A formulation is a polyhedron whose integer points are the solutions. Its LP bound, its gap, and the ideal formulation.

2 · strength

Big-M and why M matters. Aggregated against disaggregated. Extended formulations, and their proven limits.

3 · what solvers do

Presolve and cuts repair weak models, which hides the problem until they can't.

getting around

Words with a dotted underline are glossary terms: click one for its definition. G lists every term in the unit, N shows the notes. Optional extra: interactive figures, linked from the slides they support by a small ▸ chip; skip them freely.

Lab: three facility-location models with identical integer optima. With presolve and cuts off, SCIP needs 5 644 nodes on one and 1 node on another.

section 1

Formulations and relaxations

One set of integer points, many polyhedra.

05 · relaxations

Integer programs and their LP relaxation

definitions

A MILP: $\min\{c^\top x : Ax \le b,\ x_j \in \mathbb Z \text{ for } j \in I\}$. If every variable is integer it is an IP.

Its LP relaxation drops integrality. Its optimum $z_{LP}$ is a lower bound: $z_{LP} \le z_{IP}$.

integrality gap

Of an instance: $z_{IP} - z_{LP}$, or the ratio $z_{IP}/z_{LP}$.
Of a formulation for a problem class: the worst ratio over all instances (unit 24).

Why the bound matters for solving:

Pruning (unit 07): a branch whose LP bound is at least the best known solution can be discarded.
Proof: optimality is proved when the global bound meets the incumbent.
A weak bound means both happen late, after many more nodes.

A MILP solver's log is largely the story of this bound rising.

Next: all of this on an instance small enough to solve by hand.

05 · relaxations · example

The running example: two facilities, three customers

UFL, uncapacitated facility location: open some facilities ($y_i \in \{0,1\}$, cost 6 each) and serve every customer from an open one ($x_{ij}$, the share of customer $j$ served by $i$).

serving cost	customer 0	customer 1	customer 2
facility 0	1	1	5
facility 1	5	5	1

Facility 0 is near customers 0 and 1; facility 1 is near customer 2.

the integer optimum, by brute force

open	opening	serving	total
$\{0\}$	6	$1 + 1 + 5$	13
$\{1\}$	6	$5 + 5 + 1$	17
$\{0, 1\}$	12	$1 + 1 + 1$	15

$z_{IP} = 13$: open facility 0 alone. Opening the second facility saves 4 in serving but costs 6.

Two slides on, two LP relaxations of this same instance give 9 and 13. The integer points are identical; only the polyhedron differs.

05 · relaxations

Formulations are polyhedra, and some are better

Three polyhedra containing exactly the same nine integer points (black). The hull conv(S) is the ideal formulation.

definitions

A formulation of $S \subseteq \mathbb Z^n$ is a polyhedron $P$ with $P \cap \mathbb Z^n = S$.

$P_2$ is at least as strong as $P_1$ if $P_2 \subseteq P_1$. Its LP bound is then at least as good for every objective.

the ideal formulation

For rational data, $\mathrm{conv}(S)$ is a polyhedron (Meyer 1974). Over it, the LP optimum is an integer point, so the MILP is an LP.

The catch: its description is usually exponentially large, and finding it is as hard as the problem.

"At least as strong for every objective" is the partial order. Two formulations can also be incomparable: neither contains the other, and each gives a better bound for some objectives. Adding the constraints of both gives a formulation stronger than either, which is exactly what cutting planes do (unit 08).

Meyer's theorem is the reason $\mathrm{conv}(S)$ has an H-description. For bounded $S$ it follows from Minkowski–Weyl (unit 01), since $S$ is finite. For unbounded $S$ it needs rational data: with irrational data the convex hull of the integer points in a polyhedron need not be polyhedral.

Units 06 and 15 are about the happy cases where the natural formulation already is conv(S): totally unimodular matrices and bipartite matching. Then no branching is needed at all.

section 2

Strength

What makes one formulation tighter than another.

05 · strength

Big-M, and why the M matters

LP-feasible $(x, y)$ for $0 \le x \le M y$, $x \le u$, $0 \le y \le 1$. Integer-feasible: $(0,0)$ and the thick segment at $y = 1$. Shaded: what $M = 2u$ adds to the tight triangle.

To switch a continuous quantity off with a binary, a big-M constraint: $\;x \le M\,y,\; y \in \{0,1\}$.

Any $M$ at least the true upper bound $u$ of $x$ is valid.
The relaxation lets $y = x/M$. With $M \gg u$, the LP opens a facility "a little": $y$ tiny, cost tiny, bound weak.
The tightest valid M is the best upper bound on $x$ you can prove. Finding it is modelling work, and presolve can sometimes do it for you.

on the running example

A facility serves at most 3 customers, so $\sum_j x_{ij} \le 3\,y_i$ uses the tightest valid $M = 3$. The LP bound is 9.
Write $M = 30$ instead, equally valid: the LP bound drops to 3.6, with $y = (\tfrac1{15}, \tfrac1{30})$.

two diseases at once

A big M weakens the bound and mixes magnitudes in one constraint, which is unit 02's tolerance problem.

explore interactively

The example box. Every customer still goes to its nearest facility ($x_{00} = x_{01} = x_{12} = 1$, serving cost 3), and the LP pays only for the opening it is forced to: $y_0 \ge 2/M$ and $y_1 \ge 1/M$. With $M = 3$ that costs $6 \cdot \tfrac23 + 6 \cdot \tfrac13 = 6$, total 9. With $M = 30$ it costs $6 \cdot \tfrac{2}{30} + 6 \cdot \tfrac{1}{30} = 0.6$, total 3.6. Both values are from HiGHS on the reference ufl_aggregated.

In the lab, $\sum_j x_{ij} \le M y_i$ with $x_{ij} \le 1$ has smallest valid $M = C$ (every customer could be served by one facility). $M = 10C$ is what a careless modeller writes. The LP bounds were 1542 and 1416 against an optimum of 2629 on the 20×60 instance: gaps of 41% and 46%.

The numerical side: with $M = 10^6$ and $y$ at tolerance $10^{-9}$ from 0, the constraint allows $x$ up to $10^{-3}$ while the solver reports $y = 0$. Solvers report "integer" solutions that aren't really feasible in this way, and it is one of the most common modelling bugs in practice.

Alternatives to big-M: indicator constraints ($y = 1 \Rightarrow a^\top x \le b$), which solvers handle by branching or by internally choosing M from bound propagation; and disjunctive formulations (Balas 1979), which give the convex hull of a union of polyhedra at the price of copying variables. Vielma's 2015 survey in the reading list covers both.

05 · strength

Aggregated against disaggregated

Aggregated, $F$ constraints: $\;\sum_j x_{ij} \le C\, y_i$

Disaggregated, $F \cdot C$ constraints: $\;x_{ij} \le y_i$

claim

Disaggregated is strictly stronger: it implies aggregated (sum its constraints over $j$), and not the other way round.

a point only the weak model allows

Facility $i$ serves customer 1 alone: $x_{i1} = 1$, all other $x_{ij} = 0$. Aggregated allows $y_i = 1/C$. Disaggregated forces $y_i \ge x_{i1} = 1$ and pays the full opening cost.

Lab then, 20 facilities and 60 customers:

formulation	LP gap	nodes*
aggregated, M = 10C	46.1%	5 644
aggregated, M = C	41.3%	4 293
disaggregated	0.0%	1

* SCIP with presolve and cuts off. The disaggregated LP is 60× bigger, and it still wins by three orders of magnitude.

05 · strength · example

The two LPs, on the running example

formulation	LP optimum $y$	$z_{LP}$	gap to 13
aggregated, $M = 30$	$(\tfrac1{15}, \tfrac1{30})$	3.6	72%
aggregated, $M = 3$	$(\tfrac23, \tfrac13)$	9	31%
disaggregated	$(1, 0)$	13	0%

In the aggregated LP, facility 1 serves customer 2 completely ($x_{12} = 1$) while being open only a third. That is legal there: the constraint only asks $x_{10} + x_{11} + x_{12} \le 3y_1$, i.e. $1 \le 3 \cdot \tfrac13$.

a valid inequality that cuts it off

$x_{12} \le y_1$: "you can't serve customer 2 from facility 1 more than it is open". Every integer solution satisfies it (if $y_1 = 0$ then $x_{12} = 0$).

The aggregated LP point has $x_{12} = 1 > \tfrac13 = y_1$: violated. Adding all six such constraints is the disaggregated formulation, and its LP optimum is already the integer optimum.

Same three integer solutions, same 13, and bounds of 3.6, 9 and 13. A branch-and-bound on the first must branch; on the last it stops at the root.

explore interactively

05 · strength

When a constraint is implied, and still helps

Capacitated facility location: $\;\sum_j d_j x_{ij} \le u_i\, y_i$.

For integer $y$ this already implies $x_{ij} \le y_i$: if $y_i = 0$, facility $i$ serves nothing. So the linking constraints remove no integer solution.

For fractional $y$ they do. $y_i = 0.3$ lets the capacity constraint serve 30% of the capacity, but $x_{ij} \le 0.3$ also stops any single customer being served more than 30% by it.

The cover constraint $\sum_i u_i y_i \ge \sum_j d_j$ is valid for the same reason, and it forbids opening "too little capacity in total".

Lab then, 20×60, capacity ratio 1.5:

CFL	LP gap	nodes*
capacity constraints only	5.5%	102
+ linking + cover	1.2%	20

valid inequality

An inequality satisfied by every integer solution. Adding one never changes the MILP, and may tighten its relaxation. That sentence is all of unit 08.

05 · strength

Extended formulations: add variables, lose facets

An extended formulation: a polytope with exponentially many facets can be the projection of a polytope in more variables with few facets.

Spanning trees: exponentially many subtour constraints in edge variables, but $O(n^3)$ constraints with flow variables (Martin 1991).
Permutahedron: $2^n - 2$ facets, but $O(n \log n)$ variables and constraints through a sorting network (Goemans).
Disjunctions: $\mathrm{conv}(P_1 \cup P_2)$ by copying variables (Balas).

Unit 01 said projection can create facets. This is the useful direction of that fact.

and the limits

Fiorini, Massar, Pokutta, Tiwary & de Wolf (2012): the TSP polytope, the cut polytope and the stable-set polytope have no extended formulation of polynomial size, of any kind.

Rothvoss (2014): neither does the perfect matching polytope, even though matching is solvable in polynomial time.

So "polynomial algorithm" does not imply "polynomial LP". Matching needs Edmonds' algorithm, or separation (unit 09).

05 · strength

Symmetry is a formulation defect

Symmetry. Bin packing with $K$ identical bins: $x_{ik} = 1$ if item $i$ goes in bin $k$.

Every packing appears $K!$ times, once per relabelling of the bins.
The LP bound is the same in every symmetric branch, so branch-and-bound proves the same thing $K!$ times.
With 20 bins, $20! \approx 2.4 \times 10^{18}$ copies.

Cures: symmetry-breaking constraints ($y_1 \ge y_2 \ge \dots$, or item $i$ only in bins $\le i$); orbital fixing in the solver; or a formulation without labels at all (column generation, unit 10).

3 items, 2 bins, every copy

packing	bin of each item
all together	`000` · `111`
$\{0,1\}\ \{2\}$	`001` · `110`
$\{0,2\}\ \{1\}$	`010` · `101`
$\{0\}\ \{1,2\}$	`011` · `100`

$2^3 = 8$ assignments, 4 packings, each twice. "Item 0 goes in bin 0" keeps the left column only.

this is unit 00 again

Set partitions versus maps from items to bins. The restricted growth string is exactly "item $i$ may only open bin $\le$ one more than the largest used so far".

section 3

What solvers do with your model

They repair a lot, which is both good news and a trap.

05 · solvers

Presolve and cuts repair weak models

The same 20×60 models, default settings:

formulation	nodes, bare SCIP	HiGHS	SCIP
UFL aggregated, M = 10C	5 644	7	1
UFL disaggregated	1	1	1
CFL capacity constraints only	102	11	27
CFL + linking + cover	20	1	4

Default solvers derive much of the missing strength themselves:

bound tightening in presolve shrinks M to what is provable (on the running example it can take $M = 30$ back to 3, since each $x_{ij} \le 1$);
implied bound cuts recover $x_{ij} \le y_i$ from bounds and linking constraints;
knapsack and flow cover cuts recover the capacity structure.

the trap

A weak model looks fine at this size, then falls apart at 10× the size, or once a side constraint defeats the presolve reduction that was quietly saving it.

05 · solvers

A caught bug, and why brute force stays in the course

what happened while building this lab

On a 4-facility, 7-customer instance of the aggregated model, HiGHS 1.14.0 reported 881 as optimal.

The brute-force oracle said 831. So did SCIP, and so did HiGHS with presolve switched off. Across 60 such instances, 1.14.0 was wrong on this one, and 1.10–1.13.1 were right on all of them.

What the course does about it:

the project pins HiGHS 1.13.1;
unit 05's tests require HiGHS and SCIP to agree before trusting an optimum;
every lab keeps a small-instance brute-force check.

"Optimal" in a solver log is a claim backed by floating-point arithmetic and millions of lines of presolve code. On small instances you can check it, so check it.

05 · lab

Lab 05 — what you build

about 2 hours · uv run co lab 05 · models as data

Aggregated big-M UFL, for any M.
Disaggregated UFL, and a check that its bound is stronger.
Brute force over open sets: the oracle.
Capacitated FL, weak and strong, with the same integer optimum.
Gap closed, for the report.

uv run co test 05
uv run co then 05     # LP gaps · bare SCIP node counts · default HiGHS and SCIP

05 · checkpoint

Done when

checkpoint

All five lab steps are ✓.
Shown two formulations of one problem, you can predict which solves faster from their relaxation bounds alone, and be right more often than not.
You can explain, with the one-variable picture, why a larger M gives a weaker relaxation.

Reading

Wolsey, Integer Programming (2nd ed.), ch. 1 and §2.1–2.2.
Conforti, Cornuéjols & Zambelli, §2.1–2.4 and §4.1–4.3.
Vielma, SIAM Review 57 (2015), §1–3 (optional).

07 branch and bound, on these models · 06 total unimodularity: when the relaxation is already ideal (branch).