Part V · where the trunks converge · spine
Local search engineered properly, the handful of metaheuristics worth knowing, and what a comparison between randomised algorithms has to hold fixed before it means anything.
needs 24, 19~1½ h slides + 3 h lablab: uv run co lab 26
26 · overview
2-opt, Or-opt, neighbour lists, don't-look bits; Lin–Kernighan and LKH.
Annealing, tabu, GRASP, ILS, ALNS, and a frank account of genetic algorithms.
Equal budgets, tuning sets, seeds, time-to-target plots, and a test allowed to say no.
Your heuristics against each other and against OR-Tools, TSP and CVRP.
Words with a dotted underline are glossary terms: click one for its definition. G lists every term in the unit, N shows the notes. Optional extra: interactive figures, linked from the slides they support by a small ▸ chip; skip them freely.
section 1
The same neighbourhood, a hundred ways to search it.
26 · local search
2-opt (Croes 1958): remove edges $ab$ and $cd$, add $ac$ and $bd$, reversing the stretch between.
An improving move has $d_{ac} + d_{bd} < d_{ab} + d_{cd}$, so $d_{ac} < d_{ab}$ or $d_{bd} < d_{cd}$. Either way, one endpoint sees the move while scanning its sorted neighbour list, and can stop at the first neighbour farther than its tour edge.
Don't-look bits (Bentley 1992): a city whose surroundings haven't changed since it last failed isn't examined again. Keep a queue, and re-queue the four endpoints of every move.
Lab then, n = 1000, from the nearest neighbour tour (278 829):
| variant | length | s |
|---|---|---|
| sweep, full lists | 244 044 | 0.02 |
| sweep, k = 10 | 246 410 | 0.02 |
| don't-look bits, k = 10 | 245 287 | 0.01 |
| … + Or-opt | 239 655 | 3.2 |
| don't-look bits, random start | 257 969 | 0.13 |
The stopping rule does most of the work: even full lists are cheap once scanning stops early. Building the lists (0.4 s) cost more than searching with them. A good start matters: from a random tour 2-opt ends 5% worse.
26 · local search · example
Cities $A$–$F$ at $(0,0), (3,0), (6,0), (6,4), (3,4), (0,4)$. Dashed: the tour $A\,B\,D\,C\,E\,F$, length 24, with a crossing. Thick: the tour after one 2-opt move, length 20.
Remove $BD$ and $CE$ (diagonals, 5 each), add $BC$ and $DE$ (3 each). Reverse the stretch between: $A\,B\,\underline{D\,C}\,E\,F \to A\,B\,C\,D\,E\,F$.
$\Delta = (3 + 3) - (5 + 5) = -4$: 24 becomes 20. From $B$, neighbour $C$ (3) is closer than tour neighbour $D$ (5), so $B$'s list finds the move before the scan stops.
Tour $A\,C\,B\,D\,E\,F$, also 24. Take segment $[B]$ out: saves $CB + BD - CD = 3 + 5 - 4 = 4$. Put it between $A$ and $C$: costs $AB + BC - AC = 3 + 3 - 6 = 0$.
$\Delta = 0 - 4 = -4$: again the rectangle, 20. No single 2-opt reversal of the right stretch is needed; one city just moves.
26 · local search
For symmetric TSP, Concorde proves optimality on instances of thousands of cities, and LKH finds tours within a small fraction of a percent on much larger ones. A new TSP heuristic that doesn't compare against LKH, under the same time budget, hasn't been compared.
LKH isn't installed in this course environment. then section 2 has a hook for the elkai wrapper, untested here because it isn't installed.
section 2
Ways to leave a local optimum.
26 · metaheuristics
Each metaheuristic below is local search plus a rule for leaving a local optimum.
| method | escapes local optima by | the parameter that matters |
|---|---|---|
| Simulated annealing (Kirkpatrick, Gelatt & Vecchi 1983; Černý 1985) | accepting a worsening $\Delta$ with probability $e^{-\Delta/T}$, $T$ falling | the cooling schedule, in units of typical $\Delta$ |
| Tabu search (Glover 1986) | taking the best non-tabu move even if worsening; recent moves are forbidden | tenure; aspiration criteria |
| GRASP (Feo & Resende 1995) | restarting from randomised greedy constructions, each followed by local search | the greediness α of the restricted candidate list |
| Iterated local search (Martin, Otto & Felten 1991) | kicking the local optimum (a double bridge for TSP) and descending again | kick strength; acceptance |
| (A)LNS (Shaw 1998; Ropke & Pisinger 2006) | destroying part of the solution and repairing it; operators chosen by adaptive weights | destroy size; operator portfolio |
With $T_k \ge c / \log k$ and $c$ large enough, annealing converges to a global optimum in probability (Hajek 1988). That schedule is far too slow to use. Practical schedules carry no guarantee.
A 4-opt move $A\,B\,C\,D \to A\,C\,B\,D$ that sequential 2-opt and 3-opt can't undo in one step. That's why it's a good kick: the descent doesn't just fall back.
26 · metaheuristics · example
Back on the six cities, from the rectangle (20), annealing proposes a 2-opt move that recreates the crossing: $\Delta = +4$. Metropolis acceptance takes it with probability $e^{-\Delta/T}$.
| $T$ | $e^{-4/T}$ | with $u = 0.5$ |
|---|---|---|
| 10 | 0.670 | accepted |
| 2 | 0.135 | rejected |
| 0.5 | 0.0003 | rejected |
Hot, the search wanders almost freely; cold, it is plain descent. The cooling schedule decides how long it spends in between.
$T = 2$ rejects most moves costing 4 but accepts most costing 0.5 ($e^{-0.25} = 0.78$). Scale every distance by 1000 and the same $T$ rejects everything. So $T$ is tuned relative to a typical $\Delta$, such as the mean edge length.
With $T_k \ge c/\log k$, annealing reaches a global optimum in probability (Hajek 1988). A logarithmic schedule is hopelessly slow, so practical geometric schedules carry no such guarantee.
26 · metaheuristics
Hybrid genetic search for vehicle routing (Vidal et al. 2012; HGS-CVRP, Vidal 2022) is among the best CVRP heuristics. It works because every child is improved by a strong local search, crossover (OX on giant tours) preserves route structure, and the population is managed for diversity as well as quality.
Read the exception carefully: the local search does most of the optimisation. The population chooses where to start it.
section 3
What has to be held fixed.
26 · honest
| common flaw | honest practice |
|---|---|
| unequal time or evaluation budgets | an equal budget: the same wall-clock budget on the same machine, or the same number of evaluations |
| parameters tuned on the test instances | tune on a separate training set (lab: one instance, four temperatures) |
| best of N runs reported | report the distribution over seeds: median, quartiles, TTT plot |
| a weak or badly implemented baseline | compare against the state of the art (LKH, HGS, OR-Tools), engineered equally |
| a few hand-picked instances | standard benchmark sets, or a generator with stated parameters |
| "better on average" from 5 seeds | a test, and the willingness to report no difference |
Fix a target value. Run each algorithm from many seeds and record the time to reach it (or a failure at the cap). Plot the empirical CDF, with failures kept in the denominator.
Rank-based, so no normality assumption, and tolerant of capped runs (count failures as the cap). It tests whether one algorithm's times tend to be smaller. Not rejecting is not "equal", only "these data can't tell".
26 · honest · example
Times to reach the target: 0.4, 0.9, failed, 1.2, 0.6 s. Sort the successes and give the $i$-th a height of $(i + 0.5)/5$:
| time | 0.4 | 0.6 | 0.9 | 1.2 |
|---|---|---|---|---|
| height | 0.1 | 0.3 | 0.5 | 0.7 |
Read the height as a share of all runs: at 0.9 s it steps from 2 of 5 to 3 of 5, and the plotted 0.5 is the middle of that step. The curve stops at 0.7, not 1: the failure stays in the denominator.
Separated: times $1, 2, 3, 4, 5$ against $6, 7, 8, 9, 10$. Every pair favours the first, $U = 0$, $p = 0.012$: the first is faster.
Interleaved: $1, 3, 5, 7, 9$ against $2, 4, 6, 8, 10$. $U = 10$ of a possible 25, $p = 0.68$: no verdict, even though the first sample's median (5) is lower than the second's (6).
That second case is the ILS $k = 10$ versus $k = 6$ result on the next slide, in miniature.
26 · measured
n = 150, target 0.5% above the best of three 5 s ILS runs, 4 s cap:
| heuristic | reached | median | quartiles |
|---|---|---|---|
| ILS, k = 10 | 28/30 | 0.86 s | 0.49–1.17 |
| ILS, k = 6 | 25/30 | 0.66 s | 0.40–1.10 |
| multi-start | 14/30 | 1.36 s | 0.58–1.77 |
| annealing | 1/30 | — | — |
| pair | p | verdict |
|---|---|---|
| ILS vs multi-start | < 0.0001 | ILS |
| multi-start vs annealing | 0.0002 | multi-start |
| ILS k = 10 vs k = 6 | 0.76 | no winner declared |
A schematic sketch of out/ttt.png, not to scale; the plot file has the data. The two ILS curves cross.
26 · measured
TSP, n = 200, 10 instances, 3 s each: % above the best found tour by any method.
| method | mean | worst | best on |
|---|---|---|---|
| ILS (double bridge) | 0.06% | 0.34% | 8/10 |
| multi-start 2-opt + Or-opt | 0.48% | 1.51% | 2/10 |
| annealing + polish (tuned) | 1.78% | 3.06% | 0/10 |
| OR-Tools routing, GLS | 2.65% | 5.64% | 0/10 |
CVRP, n = 60, 5 instances, 10 s budget:
| seed | greedy | your ALNS | ALNS s | OR-Tools GLS |
|---|---|---|---|---|
| 0 | 2 039 | 1 059 | 6.9 | 1 067 |
| 1 | 2 368 | 1 160 | 8.6 | 1 170 |
| 2 | 2 262 | 1 237 | 9.2 | 1 261 |
| 3 | 2 326 | 1 182 | 7.6 | 1 230 |
| 4 | 2 546 | 1 266 | 9.2 | 1 304 |
Before believing "ALNS beats OR-Tools": 5 instances, no lower bound, one machine, and OR-Tools pays for a Python callback on every arc. The claim that survives: at this size and budget, a simple ALNS is competitive.
26 · lab
uv run co test 26
uv run co then 26 # speed-ups · equal budgets vs OR-Tools · TTT over 30 seeds · CVRP (≈ 12 min)26 · checkpoint
Johnson & McGeoch, in Local Search in Combinatorial Optimization (1997).
Helsgaun, EJOR 126 (2000).
Ropke & Pisinger, Transportation Science 40 (2006).
Sörensen, "Metaheuristics — the metaphor exposed", ITOR 22 (2015).
Part VI: 27 modelling for a real solver · 28 experimental method.