Part IV · trunk C · constraint programming and SAT · addendum

Backjumping, and why search is complete

When a dead end has nothing to do with the last choice, go back to the choice that caused it. Why a search that skips subtrees this way still misses no solution, and why it always stops.

needs 19~1½ h slidesno lab

Unit 19's slide "What to remember across restarts" defined a nogood and said that the strong, reason-based form is clause learning, which unit 20 builds. Unit 20 then does first-UIP learning and non-chronological backjumping on clauses, and states that a formula is unsatisfiable iff the empty clause can be derived. Between the two, three things go unsaid. What does backjumping look like in a plain CSP search, with no clauses in sight? What is a conflict set? And why is a search that skips whole subtrees still complete, and why does it stop, even when unit 19's first-fail or dom/wdeg picks the variables in a different order every time?

This addendum answers those three questions, and section 4 spends the most time on the last one: what a variable-ordering heuristic may do without breaking either proof, a bug that tests run in the input order cannot expose, and restarts. It is theory only: no lab, and no measurements beyond a few small instances and a correctness check of the reference implementation. It is the CSP version of what unit 20's engine does with clauses, and it is where the words "sound", "complete" and "terminates" get an argument instead of an assertion.

How to read this deck

One four-variable instance runs through the whole deck: hover it wherever it is named. Every number in the deck comes from the unit's reference implementation, slides/viz/traces/u19b.py: CBJ with explicit frames, conflict sets and the termination measure, which it asserts drops on every step; the same loop in chronological mode; the five variable orders of section 4; the pitfall's variants; and the instances. From the course root, uv run python -m slides.viz.traces.u19b prints, labelled, every number the deck quotes. The slides give each argument's idea, and the notes give the full proofs.

19b · overview

The route

1 · conflict sets and backjumping

Why chronological search thrashes; frames, closed values, reasons; Prosser's jump rule, step by step.

2 · completeness

Reasons are nogoods; one invariant; why an empty conflict set means "no solution"; learned nogoods.

3 · termination

A number that drops on every step, even when a jump sends the digits below it up.

4 · choosing the variable at each level

The heuristic's contract, and why both proofs survive it; orders against CBJ, measured; a jump-target bug that tests in the input order cannot show; restarts; where the argument stops; optimization.

getting around

Words with a dotted underline are glossary terms: click one for its definition. Solid-underlined names are pieces of the running example, or of the other small instances: hover to see them. G lists every term in the unit, N shows the notes. Optional extra: interactive figures, linked from the slides they support by a small ▸ chip; skip them freely.

The checkpoint asks for these explanations: why CBJ needs 17 steps on the example where chronological search needs 33; why an empty conflict set at an exhausted frame proves unsatisfiability while an empty set at a new frame proves nothing; how the measure drops on a jump even though the digits below the jump go up; what a variable-ordering heuristic must do, and why neither proof looks at its choice; why a jump target taken by index is wrong under any order but the input order, and what test would catch it; and why Luby restarts still end.

Section 4 is the longest because it answers the question a solver writer actually meets: unit 19's heuristics choose a different variable on every branch, so is the search still complete, and does it still stop? Yes, under one condition on the heuristic, and one condition on the engine that is easy to get wrong.

There is no lab. In a plain backtracking search with no propagation, adding a conflict set to each frame and the jump rule to its backtrack step is a short exercise, and the running example is then a ready-made test: 17 steps and 6 rejections with the static order. The engine of units 17 and 19 propagates, and there it is not short: after $a = 1$, propagation already empties $c$ or $d$, and the values it removes need reasons of their own (unit 21's explanations; see the fine print in section 2).

section 1

Conflict sets and backjumping

Go back to the choice that caused the failure.

19b · backjumping

Thrashing, on four variables

The running example: $a, b, c, d \in \{1, 2\}$, constraints $a \ne c$, $a \ne d$, $c \ne d$. $b$ is in no constraint. Unsatisfiable: $a, c, d$ form a triangle, which needs three colours. Static order $a, b, c, d$.

Chronological backtracking undoes the most recent choice first. When the $c, d$ part fails under $a = 1, b = 1$, it tries $b = 2$ and fails in exactly the same way: $b$ played no part. That is thrashing, the same failure rediscovered under an irrelevant choice.

Conflict-directed backjumping (CBJ) sees that the failure below $b$ only blames $a$, and jumps straight back to $a$.

Chronological search under $a = 1$. Red: rejected, with the constraint that rejected it. No rejection involves $b$, so trying $b = 2$ (shaded) replays the identical failure. CBJ jumps from the exhausted $c$ frame straight back to $a$ (step 8) and never enters it. Under $a = 2$ the same happens again, mirrored.

explore interactively

Counting the waste. The tree has four $b$ subtrees and each ends in three rejections: one of $c$'s values and both of $d$'s. So chronological search makes 12 rejections; CBJ enters only the two $b = 1$ subtrees and makes 6. Steps count every move of the search, not only rejections: descending into a new frame, rejecting a value, going back up, and the final stop. Chronological search needs 33 of them, CBJ 17. Both full runs: chronological, CBJ.

Why it is called thrashing. The failure of $c$ and $d$ under $a = 1$ is a fact about $a$, $c$ and $d$ alone. Chronological search does not know that, so it rediscovers the fact once for each combination of values of the variables between $a$ and $c$. With one irrelevant variable that doubles the work; with $k$ irrelevant two-valued variables between $a$ and $c$ it multiplies it by $2^k$. For $k = 0, 1, 2, 3, 4$ chronological search makes 6, 12, 24, 48, 96 rejections (15, 33, 69, 141, 285 steps), while CBJ makes 6 every time (15, 17, 19, 21, 23 steps): slides/viz/traces/u19b.py, whose uv run python -m slides.viz.traces.u19b prints both full runs of this slide and these counts. Propagation (unit 17) is one cure, stopping the failure before it is reached. This deck is about the other cure, going back to the right place after the failure.

Why $b$ is in the example at all: it stands in for everything in a real model that is assigned between the variables of a conflict, which in practice is most variables.

19b · backjumping

Frames, closed values, reasons, conflict sets

the search stack

A frame is one level of the stack: the variable chosen for that depth, its candidate values in order with a cursor at the current one, and a conflict set.

A value is closed at a frame when search is finished with it while the assignments above stay put: a constraint rejected it, a learned nogood matched it, or everything below it failed and a backjump landed here.

Each closed value has a reason: the earlier variables that caused it to be closed.

The conflict set of a frame starts empty when the frame is opened, and is the union of the reasons of its closed values.

closed because	reason
rejected by a constraint	the constraint's other variables (assigned, earlier)
a deeper frame was exhausted and jumped here	that frame's conflict set, minus this frame's variable
a stored nogood matched	the nogood's other variables

after step 6 of the CBJ run

Stack $a = 1$, $b = 1$, $c = 2$, and a frame for $d$ with both values closed: $d = 1$ by $a \ne d$, reason $\{a\}$; $d = 2$ by $c \ne d$, reason $\{c\}$. Conflict set of $d$: $\{a, c\}$.

Frame $c$: $c = 1$ closed by $a \ne c$, reason $\{a\}$; cursor on $c = 2$. Frame $b$: nothing closed, conflict set empty. $b$ appears in no reason.

Why frames, and not just an assignment. Chronological backtracking can be written as a recursive function, and the stack is then hidden in the call stack. Backjumping has to leave several levels at once and carry information up with it, so it helps to make the stack explicit. Each frame is a small record: which variable, which values in which order, how far along the list we are, and a set of variables. Everything in this deck is a statement about that stack.

"Closed" is deliberately broader than "rejected". A value can be closed without ever being tested against a constraint: when a deeper frame runs out of values and jumps here, the current value is closed because everything below it failed. All three ways of closing a value come with a reason, and the rest of the deck only ever uses the reason.

Reasons point upwards. In each of the three rows of the table the reason names only variables of frames above the one being closed. For a rejection this is because only assigned variables are checked; for a jump, because the conflict set of a deeper frame names variables above it and the target's own variable is removed; for a nogood, because it can only match if its other variables are already assigned. This is the fact the completeness proof rests on.

Some texts store one reason per closed value; the reference implementation (slides/viz/traces/u19b.py), and Prosser's CBJ, keep only their union, the conflict set. The union is all the jump rule needs.

19b · backjumping

The backjump rule, and the example step by step

the backjump rule (Prosser 1993)

When every value of a frame is closed (the frame is exhausted), let $E$ be its conflict set.

If $E$ is empty, stop: unsatisfiable.
Otherwise jump to the deepest frame whose variable is in $E$, discard the frames below it, merge $E$ minus that variable into its conflict set, and close its current value.

Chronological backtracking is the same loop with the jump always going to the frame just above, and stopping only when the top frame is exhausted. It differs from step 8: "$c$ exhausted, back to $b$", and then it repeats the $c, d$ failure under $b = 2$, for $a = 1$ and again for $a = 2$.

step	what happens	reason, or $E$
1–2	descend $a = 1$, $b = 1$
3	reject $c = 1$ by $a \ne c$	$\{a\}$
4	descend $c = 2$
5	reject $d = 1$ by $a \ne d$	$\{a\}$
6	reject $d = 2$ by $c \ne d$	$\{c\}$
7	$d$ exhausted: jump to $c$	$E = \{a, c\}$
8	$c$ exhausted: jump to $a$, skipping $b$	$E = \{a\}$
9–13	$a = 2$, $b = 1$, $c = 1$; $d = 1$, $d = 2$ rejected	$\{c\}$, $\{a\}$
14	$d$ exhausted: jump to $c$	$E = \{a, c\}$
15	reject $c = 2$ by $a \ne c$	$\{a\}$
16	$c$ exhausted: jump to $a$	$E = \{a\}$
17	$a$ exhausted: unsatisfiable	$E = \emptyset$

All 17 steps: hover the CBJ run; compare the chronological run.

explore interactively

Following the conflict sets. At step 7 the frame of $d$ has $E = \{a, c\}$. The deepest frame whose variable is in $E$ is $c$'s (depth 2), so the jump goes there, which happens to be the frame just above. It merges $E \setminus \{c\} = \{a\}$ into $c$'s conflict set, which was already $\{a\}$ from step 3, and closes $c = 2$. Now both values of $c$ are closed, and at step 8 $E = \{a\}$. The deepest frame with a variable in $\{a\}$ is $a$'s, at depth 0: the jump discards the frames of $b$ and $c$, merges $\{a\} \setminus \{a\} = \emptyset$ into $a$'s conflict set, and closes $a = 1$. That is the step where CBJ and chronological search part ways.

Under $a = 2$ the order of events inside the $c, d$ subtree changes ($c = 1$ now passes and $c = 2$ is rejected), but the conflict sets come out the same, and step 16 jumps from $c$ to $a$ again. At step 17 both values of $a$ are closed, each with an empty reason, so $a$'s conflict set is empty and the search stops.

History. Gaschnig's backjumping (1979) jumped back only from a frame whose values were all rejected by constraints directly. Dechter's graph-based backjumping (1990) used the constraint graph instead of the actual conflicts. Prosser's CBJ (1993) is the version here: it keeps a conflict set per frame, and carries it up through a jump, so it can jump again from an internal frame (step 8 is such a second jump).

The reference implementation, solve in slides/viz/traces/u19b.py, checks a new value against every constraint whose other variable is already assigned, and takes the first one violated, in the model's order, as the reason. With several violated constraints, the choice changes the reason, and possibly the jump, but not correctness.

section 2

Completeness

Skipping a subtree is safe when the reason says it must fail.

19b · completeness

Reasons are nogoods, and one invariant

what a reason says

A reason $R$ for closing value $v$ of frame variable $x$, under the current assignment $A$: no solution agrees with $A$ on $R$ and has $x = v$.

Equivalently, $\{y = A(y) : y \in R\} \cup \{x = v\}$ is a nogood in unit 19's sense. That is what makes it safe to store.

the invariant

Every closed value of every frame has a valid reason that names only variables of frames above it.

the combining lemma

If every value of $x$ is closed, with reasons $R_v$, then no solution agrees with $A$ on $E = \bigcup_v R_v$.

Why: a solution gives $x$ some value $v$ of its domain, and agrees with $A$ on $R_v \subseteq E$: exactly what the reason for $v$ rules out.

So $E$ minus the jump target's variable is a valid reason for closing the target's value. Discarding the frames below the target breaks nothing: reasons mention only variables above their own frame, and those keep their values.

full proof in the notes (N)

step 7

$d = 1$ has reason $\{a\}$, $d = 2$ has $\{c\}$, so no solution has $a = 1$ and $c = 2$. Closing $c = 2$ with reason $\{a\}$ says exactly that.

Reading a reason. "Reason $\{a\}$ for closing $d = 1$ with $a = 1$" means: there is no solution with $a = 1$ and $d = 1$. It says nothing about $b$ or $c$, which is the whole point: the claim holds whatever values they take, so it survives when they change.

Full proof

Write $A$ for the current assignment: each frame's variable maps to the value under its cursor. The claim is that the invariant holds after every step: for each frame $f$ with variable $x_f$, its conflict set $\mathrm{CS}(f)$ contains only variables of frames above $f$, and for every closed value $v$ of $f$, no solution agrees with $A$ on $\mathrm{CS}(f)$ and has $x_f = v$. (This is the per-value statement with $R_v$ replaced by the union: agreeing on the larger set $\mathrm{CS}(f) \supseteq R_v$ is a stronger hypothesis, so the claim is weaker and follows from the per-value one.) By cases on the step.

Start. One frame, nothing closed, empty conflict set: nothing to prove.

Reject. Value $v$ of the deepest frame $f$ violates a constraint whose other variable $y$ is assigned above $f$. Any solution with $y = A(y)$ and $x_f = v$ would violate the same constraint, so $\{y\}$ is a valid reason; it is added to $\mathrm{CS}(f)$ and is above $f$. The values of $f$ closed earlier keep valid reasons, since $\mathrm{CS}(f)$ only grew and agreeing on a larger set is a stronger hypothesis. The frames above $f$ are untouched, and their reasons mention only variables above them, whose values did not change.

Descend. A new frame is opened with nothing closed and an empty conflict set. $A$ gains one variable, below every existing frame, so no existing reason is affected.

Jump. Frame $f$ has every value closed, and $E = \mathrm{CS}(f) \ne \emptyset$. Combining lemma: suppose a solution $s$ agrees with $A$ on $E$. It gives $x_f$ some value $v$ from $f$'s candidate list (see the fine print on the next slide), $v$ is closed, and $s$ agrees with $A$ on $E = \mathrm{CS}(f)$ with $x_f = v$, which the invariant for $f$ rules out. So no solution agrees with $A$ on $E$. Let $t$ be the deepest frame whose variable is in $E$. Every variable of $E$ is above $f$, and $t$ is the deepest of them, so $E \setminus \{x_t\}$ consists of variables above $t$. "No solution agrees with $A$ on $E$" is the same statement as "no solution agrees with $A$ on $E \setminus \{x_t\}$ and has $x_t = A(x_t)$", which is a valid reason for closing $t$'s current value. Merging it into $\mathrm{CS}(t)$ keeps the conflict set above $t$, and the reasons of $t$'s older closed values stay valid because agreeing on a larger set is a stronger hypothesis. The frames below $t$ are discarded, along with their invariants. The frames at and above $t$ keep their values except $t$ itself, whose cursor moves; but $x_t$ appears in no reason of $t$ or of any frame above $t$, since those reasons name only variables above their own frame.

Stop, unsatisfiable. Frame $f$ has every value closed and $E = \emptyset$. The combining lemma with $E = \emptyset$ says no solution agrees with $A$ on the empty set, which every assignment does: there is no solution. So the answer is correct.

Stop, satisfiable. Every value on the stack passed every constraint whose other variable was assigned above it. Each constraint is checked when the later of its two variables is assigned, so the full assignment satisfies them all.

Both answers are therefore correct whenever the search stops. That it always stops is section 3. Together: the search is complete, it finds a solution if and only if there is one.

19b · completeness

Why an empty conflict set means "no solution"

the stopping rule, justified

Take $E = \emptyset$ in the combining lemma: no solution agrees with $A$ on nothing, that is, there is no solution at all.

two different empty sets

A conflict set is also empty when its frame is opened, and then it claims nothing. The conclusion needs both: every value closed, and $E$ empty.

At the top frame $E$ is necessarily empty, since there are no variables above it. It can also become empty deeper, and search stops there.

unit 20's empty clause

CDCL's learned clause is the same kind of object: a nogood written as a clause. Both go back past decisions it does not mention: CBJ to the deepest variable of $E$, closing its value; CDCL further up, to the second-highest level, asserting the flipped UIP. The empty clause is $E = \emptyset$.

in the example

Step 17: $a$'s two values were closed at steps 8 and 16, each with reason $\{a\} \setminus \{a\} = \emptyset$. Every value closed, $E = \emptyset$: unsatisfiable.

Fine print. "Every value" means every value of the domain. A value propagation removed under the current assignment is closed only if its reason goes into the conflict set; if not, $E = \emptyset$ can be reported on a satisfiable problem (notes).

The confusion, spelled out. A conflict set is a record of what has been proved so far about the values already closed. A frame that was just opened has closed nothing, so its set is empty because nothing has been learned, not because something was proved from no assumptions. Only when every value is closed does the conflict set become a statement about the frame's variable as a whole, and only then does "empty" mean "unconditionally".

An empty set below the top. A small case, computed with slides/viz/traces/u19b.py (uv run python -m slides.viz.traces.u19b prints the run): variables $p \in \{1, 2\}$, $x \in \{1\}$, $y \in \{1\}$, one constraint $x \ne y$, order $p, x, y$. CBJ descends $p = 1$ and $x = 1$, rejects $y = 1$ with reason $\{x\}$, jumps from $y$ to $x$ merging $\{x\} \setminus \{x\} = \emptyset$, and then $x$ is exhausted at depth 1 with an empty conflict set: unsatisfiable, after 5 steps, without ever trying $p = 2$. That is right: $x$ and $y$ conflict whatever $p$ is. Chronological search, which stops only when the top frame is exhausted, goes on to try $p = 2$ and takes 11 steps (the same module, in its chronological mode).

The relation to CDCL, precisely. In unit 20's example the learned clause $\lnot x_2 \lor \lnot x_3$ is the nogood $\{x_2 = 1, x_3 = 1\}$. Its deepest literal is at the current level (the first UIP); the backjump goes to the level of the next-deepest literal and asserts the negation of the UIP there, so the flipped variable moves up to a shallower level. CBJ's nogood $E$-as-values has the target $t$ as its deepest variable; CBJ returns to $t$'s own depth and closes $t$'s value there. So under this matching (the UIP plays $t$), CDCL lands at the level of the nogood's second-deepest variable and CBJ at its deepest. The two differ in where the flipped variable ends up, but both go back past decisions the nogood does not mention, and both certify unsatisfiability by the same object: a nogood with nothing left in it.

The fine print, in practice. Two kinds of pruning must be kept apart. Pruning done once, before search, that removes only values in no solution (or in no solution that a symmetry-breaking rule keeps) is harmless: the search then refutes a problem with the same solutions. Pruning done under the current assignment, as forward checking and maintained arc consistency do, is different: there is no fixed "pruned problem", and a value removed that way is a closed value like any other. It needs a reason, the variables whose values caused the removal, and that reason must go into the frame's conflict set. This is why Prosser's FC-CBJ adds the pruning variables to the conflict set, and what unit 21's explanations provide for general propagators.

Without it the stop rule is unsound. A three-variable case, computed with slides/viz/traces/u19b.py with each frame opening on its list pruned against the assignments so far (its fc option; uv run python -m slides.viz.traces.u19b prints the three runs): $a \in \{1, 2\}$, $b \in \{2\}$, $x \in \{1, 2\}$, constraints $a \ne x$ and $b \ne x$, order $a, b, x$. Under $a = 1$, $b = 2$ both values of $x$ are pruned, so $x$ opens with no values and an empty conflict set, and the search reports "unsatisfiable" at step 3. But $a = 2$, $b = 2$, $x = 1$ is a solution. With the pruning variables recorded, $x$'s conflict set is $\{a, b\}$ from the start: the search jumps to $b$, then to $a$, and finds that solution in 7 steps. A heuristic that silently drops values, for instance to keep a list short, breaks the stop rule in the same way.

19b · completeness

Learned nogoods

A nogood store starts empty.
A reason written out as values is an absolute nogood: it holds whatever the rest of the assignment is. So recording it is sound.
A store hit is one more way to close a value, with the nogood's other variables as the reason.
Forgetting is safe for soundness, and for termination of a single run: a store is only a source of closings. With restarts, see section 4. Only adding needs a proof, and the reason invariant is that proof.

The invariant and the measure of section 3 do not care where a closing came from, so a store changes neither soundness nor the termination of a single run.

what the run could record

Step 7 closes $c = 2$ with reason $\{a\}$ under $a = 1$: the nogood $\{a = 1, c = 2\}$.

Step 8 closes $a = 1$ with reason $\emptyset$: the nogood $\{a = 1\}$, so $a = 1$ is in no solution.

back to unit 19: decision against reason

The decision nogood at step 7 is $\{a = 1, b = 1, c = 2\}$: every choice above. It does not fire under $b = 2$. The reason form $\{a = 1, c = 2\}$ leaves $b$ out, so it does: the difference between the two is exactly the thrashing.

Why forgetting is free. Nothing in the completeness proof depends on a nogood staying in the store: each store hit is justified when it happens, by the nogood being absolute. Removing a nogood only removes future opportunities to close values early. That is why SAT solvers can delete learned clauses by LBD (unit 20), and why a CP solver can bound its store's memory however it likes. Unit 20's one exception, never deleting a clause that is currently the reason for an assignment, comes from CDCL storing its reasons as clauses; here the reason already lives in the conflict set, so the store can lose anything. (Termination across restarts is another matter: section 4.)

Why adding needs the proof. A stored nogood is used under assignments quite different from the one it was learned under, possibly after a restart. The reason names only some of the assigned variables. Read together with every decision above it, it would be the decision nogood, which is true but fires only on that exact path. Using $R$ with $x = v$ alone, with the others (here $b$) dropped, needs more: the reason invariant, through the combining lemma for reasons that come from jumps, shows that it is a nogood by itself, so it can be used under any assignment, even after a restart.

Decision against reason nogoods. Unit 19 called decision nogoods weak and said the strong form records why the branch failed. This is the smallest example of the difference. $\{a = 1, b = 1, c = 2\}$ is true, but it mentions $b$, so it prunes nothing under $b = 2$. $\{a = 1, c = 2\}$ is also true, and it is exactly what a search at $a = 1$, $b = 2$ would need to skip $c = 2$ at once. Unit 20's first-UIP clauses and unit 21's explanations are ways of computing such reason-based nogoods when propagation, not just a single constraint check, closed the values.

Plain CBJ, as in Prosser's paper, stores nothing between jumps: its conflict sets are forgotten when frames are discarded. Adding a store is what turns it into a learning search.

section 3

Termination

One number that goes down on every step.

19b · termination

The measure

$n$ variables, $D_{\max}$ the largest domain, radix $B = D_{\max} + 2$. For depth $d < n$, $r_d$ counts the values of its frame not yet closed, the current one included, or is $\top = B - 1$ if depth $d$ has no frame.

$$\mu = \sum_{d<n} r_d \, B^{\,n-1-d}$$

The termination measure: the counts read as one number in base $B$, shallowest digit first.

the CBJ run: $n = 4$, $D_{\max} = 2$, $B = 4$

after step	digits, depth 0 1 2 3	$\mu$
start	2 3 3 3	191
6	2 2 1 0	164
7, jump to $c$	2 2 0 3	163
8, jump to $a$	1 3 3 3	127
9, descend	1 2 3 3	111
16, jump to $a$	0 3 3 3	63

every step lowers $\mu$

Each step lowers the shallowest digit it changes, and changes only digits below it.

Reject: $r_d - 1$.
Descend: $r_{d+1}$ from $\top$ to $|\mathrm{dom}\,x| \le D_{\max}$.
Jump to $j$: $r_j - 1$, and every digit below $j$ reset to $\top$.
Stop ends it.

$$\sum_{d>j} (B-1)\,B^{\,n-1-d} = B^{\,n-1-j} - 1 < B^{\,n-1-j}$$

Lowering digit $j$ by one outweighs anything below it.

Why $+2$: $\top$ must exceed every open frame's $r_d \le D_{\max}$, so that opening a frame is a decrease.

full proof in the notes (N)

In the table, step 8: depths 1 and 2 go up (2 → 3, 0 → 3), but depth 0 drops from 2 to 1, which outweighs them: $163 \to 127$. Digits are per depth, not per variable.

explore interactively

Reading the table. At the start only $a$'s frame is open, with both values unclosed ($r_0 = 2$); depths 1 to 3 have no frame ($\top = 3$). So $\mu = 2 \cdot 64 + 3 \cdot 16 + 3 \cdot 4 + 3 = 191$. After step 6 the stack is $a, b, c, d$ with $r = (2, 2, 1, 0)$: $128 + 32 + 4 = 164$. The reference implementation, slides/viz/traces/u19b.py, asserts on every step that $\mu$ drops, and uv run python -m slides.viz.traces.u19b prints the digits and $\mu$ before and after every step of both runs: for the 16 steps before the stop it drops every time, and the stop leaves it at 63.

Full proof

First, the digits are digits. The stack holds at most $n$ frames, because each frame's variable is unassigned when the frame is opened (the heuristic's contract, section 4), so the variables on the stack are distinct. An open frame has $0 \le r_d \le |\mathrm{dom}\,x| \le D_{\max} < B - 1$, and an empty depth has $r_d = B - 1$. So every $r_d$ lies in $\{0, \dots, B - 1\}$, and $\mu$ is a natural number below $B^n$ whose base-$B$ digits are the $r_d$.

Second, comparing two such numbers: if they agree on digits $0, \dots, j - 1$ and the first has a smaller digit $j$, the first is smaller, because the second's digit $j$ is worth at least $B^{n-1-j}$ more, while the first's digits below $j$ are worth at most $\sum_{d>j} (B-1) B^{n-1-d} = B^{n-1-j} - 1$, a geometric sum.

Third, each step, where $d$ is the depth of the deepest frame:

Reject closes the current value of frame $d$: $r_d$ drops by 1, nothing else changes.
Descend opens a frame at depth $d + 1$, which had no frame: $r_{d+1}$ goes from $B - 1$ to $|\mathrm{dom}\,x| \le D_{\max} = B - 2$. Digits $0, \dots, d$ are unchanged: the current value of frame $d$ is not closed by descending.
Jump to depth $j < d$: $r_j$ drops by 1 (its current value is closed), digits above $j$ are unchanged, and the discarded depths $j+1, \dots, n-1$ become $B - 1$.
Stop (satisfiable or unsatisfiable) is the last step.

In each case the shallowest changed digit decreases, so $\mu$ decreases. The starting value is below $B^n$ and every step but the stop lowers $\mu$ by at least 1 without going below 0, so there are at most $\mu_0 \le B^n - 1$ such steps, plus the stop: the search stops after at most $B^n$ steps. For the example, $4^4 = 256$ (and $\mu_0 + 1 = 192$); the actual run takes 17.

A store hit is a closing, and counts as a reject here. The bound $B^n$ is far from tight, since it is larger than the whole search tree; its only job is to show that the search ends.

section 4

Choosing the variable at each level, and why the proofs survive it

Any variable, as long as it is a new one.

19b · variable ordering

The heuristic's contract

the contract

In dynamic variable ordering the heuristic is consulted each time a frame opens. It may read anything: the assignments so far, the conflict history, unit 19's weights, randomness. Its one obligation: return an unassigned variable whenever one exists.

what breaks it

Returning a variable already on the stack: the stack's variables stop being distinct, the depth is no longer bounded by $n$, and $\mu$ is undefined.

why neither proof looks at the choice

The measure counts values per depth. On a descend, the new frame's digit drops from $\top$ to at most $D_{\max}$, whichever variable it holds. On a jump, the digits that change sit below the one that decreases.

The invariant names variables, not depths: "above" means above on the stack as it is now.

full proof in the notes (N)

first-fail, with no propagation

Plain CBJ never shrinks a domain, so "smallest domain first" is a static order. A dynamic first-fail counts the live domain: the values still consistent with the assignments so far, computed only for choosing; the frame still opens on the full domain. dom/wdeg also reads the history, bumping the constraint that rejected a value: on the Mycielski graph (next slide) CBJ with dom/wdeg puts 10 different variables at depth 5 in one run, with dom 3. Both stop with the right answer.

explore interactively

The argument in one paragraph. Look at what each step does to the stack. A reject moves one cursor. A jump truncates the stack and moves one cursor. A descend pushes one frame, and only there does the heuristic act, choosing which variable the new frame gets. The measure reads each frame only through "how many values are left", and the domain size bounds that by $D_{\max}$ whichever variable was chosen; the jump resets discarded depths to $\top$ whichever variables they held. The invariant speaks of "variables of frames above", which is a property of the stack as it is, not of any global order. Neither argument ever needs the order to be the same across the run.

What the heuristic must guarantee. A variable that is not on the stack, returned whenever the stack is not complete. The engine only descends when fewer than $n$ frames are open, so an unassigned variable always exists when the heuristic is asked; the obligation is only not to return an assigned one. Everything else is allowed, including randomness, reading the weights that failures bump, or looking at the learned store.

Why first-fail needs the live domain here. In units 17 and 19 the engine propagates, so domains shrink under the assignments and "smallest domain" changes from branch to branch. A plain CBJ engine removes no value from any domain, so the domain sizes are the model's and "smallest domain first" is fixed before search starts. The live domain restores the dynamic behaviour without propagating: for each unassigned variable, the values consistent with every assigned neighbour, computed afresh each time a frame opens and used only to choose (live_domain in slides/viz/traces/u19b.py). It is the domain forward checking would leave, but the frame does not open on it: opening on it would need the pruning variables in the conflict set (the fine print in section 2). Reading the live domains, like reading anything else, is inside the contract.

dom/wdeg and history. Every constraint's weight starts at 1, and each rejection adds 1 to the weight of the constraint that rejected. The choice is therefore a function of the run so far, not of the current assignment alone. Within one run this cannot show at a single assignment, since no partial assignment is visited twice (the nodes a run visits form a tree); it shows across restarts with the weights kept, where the same partial assignment reached again can get a different next variable. The contract does not mind: at each descend the heuristic returns some unassigned variable, and that is all either proof uses. The counts on the slide are the distinct variables seen at each depth over one CBJ run, printed by uv run python -m slides.viz.traces.u19b: on the Mycielski graph, depths 0 to 9 see 1, 1, 1, 2, 2, 3, 3, 2, 2, 1 variables under dom, and depths 0 to 7 see 1, 3, 4, 7, 8, 10, 10, 5 under dom/wdeg.

Full proof

Let the heuristic be any procedure, with any state of its own (weights, a random generator, a store), that returns an unassigned variable whenever it is called with fewer than $n$ frames on the stack. We show that the search is sound and stops within $B^n$ steps.

Distinct variables. Only a descend adds a frame, and its variable is the heuristic's answer, which is unassigned, that is, not on the stack. Rejects and jumps add no frame. By induction the variables on the stack are always distinct, so there are at most $n$ frames, and with $n$ frames every variable is assigned: a passing value there stops the search with "satisfiable" instead of descending, so the heuristic is never asked with $n$ frames open.

Termination. The proof on the measure slide uses three facts about the stack: at most $n$ frames, so $\mu$ has $n$ digits; an open frame's count is at most its domain size, at most $D_{\max}$; a descend fills the first empty depth, and a jump truncates the stack at the target and closes the target's value. None of the three names the variable the heuristic chose, so the case analysis goes through word for word, and each run stops within $B^n$ steps.

Soundness. The proof on the invariant slide says "above" for "earlier on the current stack", and uses distinctness in two places. After a jump to $t$, the variable $x_t$, whose value changes, appears in no reason of $t$ or of a frame above $t$, because those reasons name only variables above their own frame and $x_t$ is on the stack once. And the descend case needs the new variable to appear in no existing reason: reasons name variables on the stack, and the new variable was not on it. The jump case needs the target to be the deepest frame whose variable is in $E$, by position on the stack; the pitfall slides show what happens otherwise. Nothing else in the proof refers to the order.

So soundness and termination hold for every heuristic that keeps the contract, and with them completeness. If the heuristic returns an assigned variable, the first part of the proof (distinct variables) fails: the stack has a variable twice, its depth can grow past $n$, and $\mu$ is not defined.

Checked. slides/viz/traces/u19b.py was run on 4 000 random binary CSPs (seed 19; 2 to 7 variables, domains of 1 to 3 values, each pair constrained with probability one half), each solved twice: with the static order, and with a dynamic order that picks a uniformly random unassigned variable at each descend. Of the 8 000 runs, 3 458 were unsatisfiable. Every answer matched brute force, every reported solution satisfied all constraints, and $\mu$ decreased on every step. uv run python -m slides.viz.traces.u19b repeats the check. A random order is a demanding test of the claim: it makes no attempt to be consistent from branch to branch.

19b · variable ordering

Ordering against backjumping, measured

five orders, ties by input order

input: as the model lists the variables.
max-cardinality (static): most already-ordered neighbours, ties by degree.
dom: smallest live domain.
dom/deg: live domain over the number of constraints to unassigned variables.
dom/wdeg: live domain over the summed weights of those constraints; weights start at 1, a rejection bumps the rejecting one.

All four instances are unsatisfiable. Every run checks $\mu$ at every step, and its answer against brute force.

steps	example		Mycielski		pigeons		random
steps	chron.	CBJ	chron.	CBJ	chron.	CBJ	chron.	CBJ
input	33	17	1888	1175	325	325	468	174
max-card.	15	15	1312	1050	325	325	164	91
dom	15	15	568	568	325	325	72	64
dom/deg	15	15	652	652	325	325	68	68
dom/wdeg	15	15	700	659	325	325	68	68

what the numbers say

Under the input order CBJ saves 16, 713 and 294 steps on the example, the Mycielski graph and the random CSP; under dom, 0, 0 and 8. Ordering and backjumping attack the same waste: a good order leaves CBJ less to save. On the pigeons there is nothing to skip under any order.

explore interactively

Where the numbers come from. slides/viz/traces/u19b.py; uv run python -m slides.viz.traces.u19b prints this table with rejections and jumps too. A step is a descend, a reject, a jump or the stop, as on the earlier slides. "Chronological" is the deck's chronological backtracking: back to the frame just above, stopping only when the top frame is exhausted. dom, dom/deg and dom/wdeg read the live domain of the previous slide; a variable with an empty live domain is chosen first whatever its degree, and under dom/deg and dom/wdeg a variable with no constraints to unassigned variables, whose ratio would divide by zero, comes last (on the example that is $b$). Chronological search and CBJ are separate runs, so under dom/wdeg they bump different weights and can choose differently.

The instances. The running example. The Mycielski graph of the 5-cycle (the Grötzsch graph): 11 vertices, 20 edges, chromatic number 4, so 3 colours are refuted; with 4 colours CBJ finds a colouring, which the module checks. Five pigeons in four holes, as a CSP: all-different of 5 variables over 4 values, that is, the complete graph on 5 vertices with 4 colours. One random binary CSP (seed 3, 10 variables with 3 values each, each pair constrained with probability 0.5, each value pair forbidden with probability 0.35), which has 22 constraints and, by brute force, no solution.

Rejections, chronological then CBJ, in the order of the table's rows. Example: 12, 6; then 6, 6 for every other order. Mycielski: 945, 574; 657, 524; 285, 285; 327, 327; 351, 331. Pigeons: 196, 196 for every order. Random CSP: 235, 84; 83, 42; 37, 32; 35, 35; 35, 35. CBJ's jumps: example 4 for every order; Mycielski 237, 225, 141, 162, 161; pigeons 64 for every order; random CSP 28, 18, 14, 16, 16.

The point. Thrashing is failure rediscovered under choices that played no part in it. Backjumping skips those choices after the fact; a good order avoids making them between the culprits in the first place. On the example the max-cardinality order is $a, c, d, b$: the triangle is refuted before $b$ is reached, $b$ is never opened, and both searches take the same 15 steps; so do the three dynamic orders. On the Mycielski graph and the random CSP, CBJ saves hundreds of steps under the input order and little or nothing under dom. On the pigeons every order gives the same 325 steps, and CBJ saves none of them: all-different constrains every pair of variables, so no choice is irrelevant to a failure. dom/wdeg is not the best order on the Mycielski graph (700 and 659 steps, against dom's 568), which is unit 19's point that heuristics are bets on an instance family.

The literature. Chen and van Beek (2001) give both sides of this interaction. In theory, there is a "perfect" dynamic variable ordering that makes CBJ redundant (the $a, c, d, b$ order is a small case of the same effect), and the more local consistency is maintained, the less backjumping can improve on it. In practice, they find that adding CBJ to a search that maintains generalized arc consistency can still give speedups of orders of magnitude.

19b · variable ordering

The jump-target pitfall

two readings of "the deepest frame whose variable is in E"

By stack position: the deepest frame on the stack whose variable is in $E$. This is the rule the proofs use.

By index: the variable of $E$ that comes last in the input order. A CBJ written to search in the input order may do this, since there index order is depth order.

the index rule under any other order

It can jump to a frame above the deepest frame in $E$. Then it merges $E$ minus the target's variable into the target's conflict set: a set naming a variable below the target, whose frame the jump discards. The invariant is broken.

trial 291, one random dynamic order

Stack $v_5, v_3, v_0, v_2, v_4, v_1$. Step 10: $v_1$ is exhausted with $E = \{v_0, v_3\}$.

Stack rule: jump to $v_0$, at depth 2. It finds a solution at step 30.
Index rule: jump to $v_3$, at depth 1, discarding $v_0$; $v_3$'s set becomes $\{v_0\}$. Step 11: $v_3$, which has one value, is exhausted. Dropping the discarded $v_0$ leaves $E = \emptyset$: "unsatisfiable".

index rule, 3 000 random CSPs	input order	random order
the jump looks for a discarded variable: crash	0	319
such variables dropped: a satisfiable instance called unsatisfiable	0	31

A static order is not enough: under max-cardinality, static but not the input order, 159 crashes and 12 wrong answers.

explore interactively

The experiment. pitfall_stats in slides/viz/traces/u19b.py; uv run python -m slides.viz.traces.u19b prints the counts and both runs of trial 291 step by step. 3 000 random binary CSPs (seed 7; 3 to 7 variables, domains of 1 to 3 values, each pair constrained with probability one half, each value pair allowed with probability 0.6), each solved with the static input order and with a random dynamic order seeded by the trial number. The index rule comes in two variants. As written, it looks up the frame of the chosen variable, and when that variable is no longer on the stack the lookup fails (319 dynamic runs): in the module that raises an error, in a real solver it might read a stale entry or jump somewhere arbitrary. That lookup is where the break shows, not where it happens: an earlier jump already left a conflict set naming a discarded variable, and the next slide counts those runs. The second variant avoids the crash the way a hurried fix would, by silently removing from $E$ every variable that is not on the stack. That turns the crash into a wrong answer: 31 satisfiable instances reported unsatisfiable, the first of them trial 291. Under the static input order neither variant ever goes wrong, in 3 000 runs each. What protects it is not that the order is static but that it is the input order: under max-cardinality, a static order that equals the input order on only 456 of the 3 000 instances, the index rule crashes in 159 runs and the dropping variant answers 12 satisfiable instances "unsatisfiable". The same module prints these counts.

Trial 291, step by step. The two runs agree for nine steps: $v_5 = 0$, $v_3 = 0$, $v_0 = 0$ descend; $v_2 = 0$ is rejected by the constraint $v_2 v_3$, reason $\{v_3\}$; $v_2 = 1$ and $v_4 = 0$ descend; the three values of $v_1$ are rejected with reasons $\{v_3\}$, $\{v_0\}$, $\{v_3\}$. At step 10 $E = \{v_0, v_3\}$. On the stack $v_0$ is deeper (depth 2) than $v_3$ (depth 1), but $v_3$ has the larger index. The index rule jumps to $v_3$, closes $v_3 = 0$, and records $\{v_0\}$ as the reason, although $v_0$'s frame has just been discarded. The stack rule jumps to $v_0$, closes $v_0 = 0$ with reason $\{v_3\}$, and goes on: it answers "satisfiable" at step 30 with $v_0 = 1$, $v_1 = 1$, $v_2 = 1$, $v_3 = 0$, $v_4 = 0$, $v_5 = 1$.

Where the proof breaks. The jump case of the completeness proof shows that no solution agrees with the assignment on $E$, and then needs $E \setminus \{x_t\}$ to consist of variables above the target $t$, so that it is a valid, upward-pointing reason for closing $t$'s value. That holds because $t$ is the deepest frame whose variable is in $E$. With $t = v_3$ it fails: $E \setminus \{v_3\} = \{v_0\}$ names a variable below $v_3$. The claim "no solution has $v_0 = 0$ and $v_3 = 0$" is true, but once $v_0$'s frame is gone it no longer says anything about $v_3 = 0$ alone; and the jump has discarded $v_0$'s untried value $v_0 = 1$ with no reason at all. Dropping $v_0$ from the set turns "no solution with $v_3 = 0$ and $v_0 = 0$" into "no solution with $v_3 = 0$", which is false: the solution has $v_3 = 0$.

19b · variable ordering

The correction: jump by stack position

the stack-position rule

Keep an array depth[$x$], set when $x$ is assigned and cleared when it is unassigned. The jump target is the frame at the largest depth[$x$] over $x \in E$.

a cheap invariant check

After each step, assert: every variable in a frame's conflict set is on the stack above that frame. On trial 291 the index rule trips it at step 10: $v_3$'s set, at depth 1, is $\{v_0\}$, and above it is only $v_5$.

the lesson

When the search follows the input order the two rules coincide, so tests that always search in index order cannot see the bug. Test with other orders, dynamic ones included, and assert the invariant.

3 000 random CSPs	input order	random order
index rule: the check fires	0	417
index rule, dropping: wrong answers	0	31
stack rule: wrong answers	0	0
stack rule: the check fires	0	0

With the check on, the index rule is stopped in all 31 wrong-answer runs before it answers. It also fires on runs that crash, and on runs that break the invariant and still, by luck, answer correctly.

explore interactively

The numbers. pitfall_stats in slides/viz/traces/u19b.py, on the same 3 000 instances and orders as the previous slide; uv run python -m slides.viz.traces.u19b prints them. The stack rule, with the check switched on, answered all 6 000 runs correctly and never tripped the check. The index rule with the check switched on trips it in no run under the input order and in 417 under the random order: a conflict set named a discarded variable. That is more than the 319 runs that crash without the check, and since the index rule without dropping gives no wrong answer, the rest broke the invariant and answered correctly anyway: a broken invariant does not always show in the answer, which is why asserting it catches more than comparing answers does. Every one of the 31 runs that the dropping variant answers wrongly is stopped by the check before an answer.

Implementing the rule. A depth array indexed by variable, written on assign and cleared on unassign, makes the target the maximum of depth[$x$] over $E$: one pass over the conflict set, the same cost as the index rule's maximum. Reading positions off the stack directly works too. Whichever is used, the rule is "deepest on the stack", never "last in the model".

The check. For each frame, every variable in its conflict set must be the variable of a frame above it. Checked after every step it costs a pass over the conflict sets, which a debug build can afford; checking only the frame a step touched (the exhausted frame before a jump, the target after it) is cheaper and catches the same breaks, because a step changes at most one conflict set: if the invariant held before the step, only that set can break it. It is the invariant of section 2, with "valid" left out and only "points upward" kept: the part a program can check.

The lesson, in general. A rule that is only correct under an assumption the tests always satisfy cannot be caught by those tests. Here the assumption is "index order is depth order", which holds whenever the search follows the input order, and fails under max-cardinality as it does under dynamic orders. Randomised dynamic orders on small random instances, with answers checked against brute force and the invariant asserted, found it in seconds.

19b · variable ordering

Restarts with a dynamic order

assumed

Each run starts from a fresh stack; a cutoff counts steps; the instance is fixed, so $n$ and $B$ never change. Dom/wdeg's weights, and a store of nogoods written out as values, may carry over.

the restart loop ends, and is complete

A restart sends $\mu$ back up. But a run that is not cut off stops within $B^n$ steps, whatever the order. The Luby sequence (unit 19) is unbounded, so some run gets a cutoff of at least $B^n$, and that run finishes. Every answer is sound, so the loop is complete.

full proof in the notes (N)

A triangle $a, c, d$, plus $a \ne b$ and $b \ne e$, all in $\{1, 2\}$; CBJ with dom/wdeg, weights kept, Luby cutoffs times 4. Each run restarts at $\mu = 767$. From run 4 the weights make the order open $c$ after $a$, not $b$; run 7 has room for its 15 steps.

explore interactively

The example. restart_example() and restart_runs in slides/viz/traces/u19b.py; uv run python -m slides.viz.traces.u19b prints every run step by step. Run $i$ has a cutoff of $4 \cdot$ luby($i$) steps (unit 19's sequence, 4, 4, 8, 4, 4, 8, 16, ...), CBJ with dom/wdeg, the weights kept from run to run, a fresh stack for every run, and no nogood store. What to read off it: a restart throws the stack away, so each run starts again at the same $\mu_0 = 767$, and $\mu$ says nothing about the loop as a whole; what carries over is the weights, and they change the order, not the proof. Runs 1 to 3 open $b$ right after $a$, as dom/wdeg's tie-break wants; the rejections by $a \ne c$, $a \ne d$ and $c \ne d$ raise those weights until, from run 4, $c$ comes after $a$ and $b$ is never opened again. Run 7 is the first whose cutoff, 16, is at least the 15 steps the search needs in that order, and it answers "unsatisfiable".

How crude the guarantee is. Here $B^n = 4^5 = 1024$, and the first run the proof promises will finish is run 511, the first with $4 \cdot$ luby($i$) $\ge 1024$. The loop ended at run 7. The same printout has the loop on the deck's other instances, with a unit of one step: the running example ends at run 31 (the proof promises run 511), Mycielski at run 2 046 (134 217 727), the pigeons at 1 023 (16 383), the random CSP at 126 (33 554 431).

Full proof

Assumptions. (1) Every run starts from a fresh stack: one frame, opened on the heuristic's first choice, nothing closed. (2) A run's cutoff $c$ is a number of steps: the run is abandoned once it has made $c$ steps without stopping. (3) The instance does not change between runs, so $n$ and $B = D_{\max} + 2$ are the same for every run. (Root-level pruning between runs would only lower $D_{\max}$, and the bound with it.) (4) The heuristic keeps the contract in every run; it may carry any state between runs. (5) A nogood store, if any, holds only reasons written out as values.

One run. Within a run, the measure proof applies as it is: the run starts at some $\mu_0 \le B^n - 1$, every step but the stop lowers $\mu$ by at least 1, so a run that is not cut off stops, with an answer, after at most $\mu_0 + 1 \le B^n$ steps. Carried-over weights, random state or store contents change which steps are taken, not this bound: the proof holds for any heuristic that keeps the contract, and a store hit is a closing like a reject.

Some run finishes. luby($2^{k+1} - 1$) $= 2^k$ for every $k$, so the sequence is unbounded, and there is a first index $i^*$ with luby($i^*$) $\ge B^n$. If the loop reaches run $i^*$, that run has a cutoff of at least $B^n$ steps, and it stops by itself within $B^n$ steps, so it is not cut off. So the loop makes at most $i^*$ runs, and it stops.

Every answer is sound. A run that answers does so by the rules of section 2, whose invariant holds within the run from its fresh stack. A store hit closes a value with a reason that is valid under any assignment, because a stored nogood is absolute; that is why nogoods may be kept across runs, while conflict sets, which are relative to the stack they were built on, may not. So the loop's answer is correct, and since it always gives one, the restart loop is complete.

Other cutoffs. A cutoff in failures or conflicts, as SAT solvers use, works the same way, because each step adds at most one failure: a run cut off after $c$ failures has made at least $c$ steps. What the argument needs is a schedule that is unbounded and a cutoff that a run of $B^n$ steps cannot exceed. With cutoffs that stay bounded it says nothing: see the next slide.

19b · scope

Where the argument stops

restarts with bounded cutoffs

If the cutoffs stay below $B^n$, no run need finish, and the last slide's argument says nothing. Termination then needs the learned nogoods: if every run adds one the store did not have and none is forgotten, there can be only finitely many runs.

dynamic backtracking (Ginsberg 1993)

Reordering the assigned variables is a different algorithm. It moves the culprit variable without erasing the assignments in between, so the per-depth counters no longer decrease. Its termination argument rests on the recorded nogoods instead.

what this deck proved, and for what

For any search that keeps a stack of frames, closes values only with valid reasons that point upwards, jumps by Prosser's rule, opens frames on unassigned variables in any order, and restarts, if at all, with unbounded cutoffs: sound in both answers, terminating, hence complete.

Restarts, with bounded cutoffs. The previous slide's argument needs one run with a cutoff of at least $B^n$ steps. A fixed cutoff, or a schedule that stops growing, never provides one, and then nothing in the measure prevents every run from being cut off forever.

Restarts, with nogoods. If every restart happens only after the run has stored at least one nogood not already in the store, and nothing is forgotten, then the number of restarts is bounded by the number of possible nogoods, which is finite. This is the classical argument for CDCL with restarts when learned clauses are kept. Solvers that delete clauses by LBD (unit 20) cannot use it as it stands, since forgetting could let the same nogood be learned again and again: they fall back on unbounded restart intervals, the argument above, or on limits on deletion, or they have no guarantee.

Dynamic backtracking. Ginsberg's algorithm, when a variable runs out of values, unassigns only the culprit variable and keeps every other assignment, including the ones made after the culprit. The stack is then no longer a stack of depths in the sense used here: the culprit leaves the middle of the order, and variables after it stay assigned. So $\mu$, which reads counts per depth, is the wrong measure. Ginsberg's paper proves termination from the nogoods it records (its eliminating explanations) instead. See the paper for the argument; this deck does not reproduce it.

The scope statement. The box on the slide is the precise claim of this addendum. Propagation fits inside it as long as every value it removes comes with a valid, upward-pointing reason that goes into the conflict set, which is exactly what unit 21's explanations provide (the fine print in section 2 shows what goes wrong without it).

19b · optimization

Optimization: the bound is one more constraint

branch and bound, as one search

To minimise $f$, keep the best solution found so far, the incumbent, and its cost $U$. Treat $f < U$ as a constraint that tightens each time a solution is found.

All variables assigned: record the incumbent, set $U$ to its cost, and close the last value, since it now breaks $f < U$.
A value breaks $f < U$: close it with reason the other variables $f$ depends on, as for any constraint.
$E = \emptyset$ at an exhausted frame means what it always meant: no solution of the constraints, now including $f < U$. With an incumbent, it is optimal; with none, $U = +\infty$ and this is plain unsatisfiable.

why the proofs carry over

A reason found under $f < U$ says: no solution agrees with $A$ on $R$, has $x = v$ and costs less than $U$. $U$ only goes down, so that stays true under every later bound. The invariant holds, and the combining lemma with it.

Recording a solution closes a value like any rejection, so the measure of section 3 still drops on every step.

what changes

Reasons from the bound, and nogoods learned from them, are true relative to the bound, not of the problem. They are safe to keep while $U$ only decreases; not if the objective or the question changes.

The same loop. Unit 07's branch and bound, written in this deck's terms, is CBJ on the problem "the constraints, plus $f < U$", where $U$ starts at $+\infty$ and drops each time a solution is found. The search does not restart after a solution: it closes the value that completed it and carries on from the leaf. Everything else is the loop of section 1.

Starting from a solution. A solution found beforehand, by a heuristic or a previous run, can seed the search: make it the incumbent and start with $U$ at its cost. Nothing else changes, and the search can only prune more. $E = \emptyset$ then proves that seed optimal if nothing better turns up, and "unsatisfiable" can no longer be the answer, since a solution exists. Starting from a bare guess $U$ with no solution behind it is different: an empty conflict set then only says "no solution costs less than $U$", which does not tell apart an unsatisfiable problem from one whose optimum is at least $U$.

The reason for closing after a solution. The new solution $A$ has $f(A) = U$, so it breaks $f < U$. The reason for closing the last frame's value $x = v$ is the set of variables of $f$ other than $x$: any assignment that agrees with $A$ on them and has $x = v$ has the same cost $U$. All of them are above the last frame, so the reason points upwards. If $f$ depends on every variable, this reason names everything, and the search backs up one frame at a time just after a solution. The more $f$ ignores, the further the jump.

Bounds before the leaf. A solver usually does not wait until $f$ is fully assigned: it computes a lower bound $L$ on $f$ from the assigned variables and closes a value when $L \ge U$. The reason is then the assigned variables that $L$ actually used. A bound that reads every assigned variable gives chronological backtracking, which is why explaining bounds well matters in practice (lazy clause generation does this for linear objectives: unit 21).

Proof that bound reasons stay valid. Say $R$ was a valid reason for closing $x = v$ when the bound was $U$: no solution agrees with $A$ on $R$, has $x = v$ and has $f < U$. Later the bound is $U' < U$. A solution with $f < U'$ also has $f < U$, so the same statement with $U'$ holds. The invariant of section 2 is therefore preserved when $U$ tightens, and every other step is the satisfaction case, with $f < U$ as one of the constraints. At the stop, $E = \emptyset$ says no solution has $f < U$; the incumbent has $f = U$, so it is optimal.

When the relative nogoods bite. To find all optimal solutions, the constraint has to be $f \le U$, and a nogood learned under $f < U$ would wrongly remove solutions that tie. The same goes for reusing a store on a changed objective, or for moving to the next objective in a lexicographic search: bound-derived nogoods must be dropped or kept apart from the absolute ones.

19b · optimization

Optimization on three variables

the instance

$a, b, c \in \{1, 2\}$, constraint $a \ne c$, minimise $f = a + 2c$. As in the running example, $b$ is in nothing. Static order $a, b, c$.

Optimum: $a = 2$, $c = 1$, cost $4$ (the only other option, $a = 1$, $c = 2$, costs $5$).

CBJ stops after 11 steps, chronological branch and bound after 21: it tries $b = 2$ under both values of $a$, and fails on $c$ in the same way each time.

Step 5: the closing of $a = 1$ carries reason $\emptyset$ under $U = 5$. It is still true at $U = 4$: there is no solution with $a = 1$ and cost below 4 either.

The tree chronological branch and bound visits. Green: a solution, which sets $U$ and closes its value with reason $\{a\}$. Red: rejected, by $a \ne c$ or by the bound. Once $c$ is exhausted (steps 5 and 10), CBJ jumps straight back to $a$ and never enters the shaded subtrees.

explore interactively

Following the bound. At step 4 the stack is $a = 1$, $b = 1$, $c = 2$, which satisfies $a \ne c$, and there is no bound yet, so this is the first incumbent, with $U = 5$. The full assignment now breaks $f < 5$; $f$ depends on $a$ and $c$, so the reason for closing $c = 2$ is $\{a\}$. Both values of $c$ are closed with reason $\{a\}$, so $E = \{a\}$ and the jump goes straight to $a$: the claim is "no solution with $a = 1$ costs less than 5", whatever $b$ is.

Under $a = 2$, $c = 1$ passes and costs $2 + 2 = 4 < 5$: a better incumbent, $U = 4$. Its closing again has reason $\{a\}$; $c = 2$ breaks $a \ne c$, reason $\{a\}$. The jump to $a$ closes $a = 2$ with reason $\emptyset$, both values of $a$ are closed with empty reasons, and the search stops: no solution costs less than 4, so the incumbent is optimal.

The chronological run. After step 4 it backs up to $b$, tries $b = 2$, and rejects $c = 1$ (by $a \ne c$) and $c = 2$ (now by the bound) again; the same happens under $a = 2$, with the two causes swapped. It finds the same optimum in 21 steps. Both counts use the deck's convention: every descend, rejection, recorded solution, jump or backtrack, and the final stop is one step. Both runs come from the reference implementation, which takes an objective and treats $f < U$ as one more constraint: opt_example() in slides/viz/traces/u19b.py, printed by uv run python -m slides.viz.traces.u19b.

19b · scope

CBJ with a nogood store, against CDCL

the same idea

Explain each dead end by a nogood that names only the choices that caused it; go back past the choices it does not name; an empty nogood proves there is no solution. Unit 20's learned clause is such a nogood, written as a clause.

when, and what is learned

CBJ jumps only when a frame is exhausted, and what it carries up is that frame's conflict set: the union of the reasons of its closed values.

CDCL analyses every conflict at once, resolving back through the implication graph to the first UIP: one literal of the current level, the rest from earlier levels.

where it lands

CBJ goes to the deepest variable of $E$ and closes its value; the next value is a new choice.

CDCL goes to the second-deepest level of the clause, where the clause is unit and forces the flipped UIP: an implication, not a decision.

how learned nogoods are used

This deck's store is checked when a value is tried: a hit closes that value. CDCL's clauses join unit propagation, so they fix values before anything is tried, and serve as reasons in later conflict analysis.

Around that, CDCL adds propagation at every node, VSIDS, frequent restarts that keep what was learned, and LBD deletion (unit 20). Unit 21 brings the whole engine back to CP.

Why the store is weaker than it looks. In this deck a stored nogood $\{a = 1, c = 2\}$ does nothing until the search sits at $a = 1$ and tries $c = 2$; then it closes that value. A CDCL solver holding the clause $\lnot[a = 1] \lor \lnot[c = 2]$ would, as soon as $a = 1$ is set, remove $c = 2$ by unit propagation, before $c$ is even chosen, and with a reason, so that the removal can take part in the next conflict analysis. That is the step from "checking nogoods" to "propagating nogoods", and in practice it accounts for much of the difference.

When and where. CBJ's unit of work is the frame: a value list with a cursor, and a jump only once every value of a frame is closed. CDCL has no value lists. Every conflict, that is every clause falsified by propagation, is analysed on the spot; the learned clause says exactly which assignment to undo, and the backjump lands at the level where that clause becomes unit, so the flipped literal is implied there rather than chosen. The box "unit 20's empty clause" in section 2 compares the two jump targets on unit 20's example.

What is learned. CBJ's conflict set is the union of reasons, one variable per rejection: the reasons are single constraints. With propagation the reason for a removed value is itself a set of assignments, and following reasons back until one literal of the current level is left is first-UIP resolution. So CBJ with forward checking's reasons (the fine print in section 2) already moves towards CDCL, and unit 21's explanations complete the step for general propagators.

The rest of the engine is heuristics, not correctness: the proofs of this deck cover any order and, with unbounded cutoffs, restarts; deletion by LBD is safe for soundness (section 2) and for termination only under the conditions on the previous slides.

19b · checkpoint

Done when

checkpoint

You can explain, on the example, why CBJ takes 17 steps and chronological search 33.
You can say why $E = \emptyset$ at an exhausted frame means unsatisfiable, and at a new frame nothing.
You can show $\mu$ decreasing on a jump, although the digits below it go up.
You can state the heuristic's contract, why neither proof looks at the choice, and why Luby restarts still end.
You can say why a jump target by index fails under any order but the input order, and how to test for it.

Reading

Prosser, "Hybrid algorithms for the constraint satisfaction problem", Computational Intelligence 9(3), 1993.
Ginsberg, "Dynamic backtracking", JAIR 1, 1993.
Dechter, Constraint Processing, 2003, the look-back chapter.
Chen & van Beek, "Conflict-directed backjumping revisited", JAIR 14, 2001.

20 SAT and CDCL: the same ideas with clauses, where the reasons come from unit propagation and the nogoods are kept.

The one-sentence version: every closed value carries a reason that points upwards, the reasons of an exhausted frame combine into a reason at the jump target, the deepest frame whose variable they name, an empty combined reason is a proof that there is no solution, and a base-$B$ counter over the depths shows that the whole process stops whatever order the variables are chosen in, provided each choice is a variable not yet on the stack and the jump target is found by its position on the stack. Luby restarts keep both properties, because some run is always allowed to finish.

For the reading: Prosser's paper introduces CBJ alongside hybrids with forward checking. Dechter's look-back chapter covers Gaschnig's, graph-based and conflict-directed backjumping together, with learning. Ginsberg's paper is the place for dynamic backtracking and its own termination argument. Chen and van Beek revisit CBJ in combination with look-ahead and variable ordering: in theory a perfect dynamic ordering, or enough local consistency, leaves backjumping little or nothing to save; in their experiments, adding CBJ to a search that maintains generalized arc consistency can still be faster by orders of magnitude.

Backjumping, and why search is complete

The route

Conflict sets and backjumping

Thrashing, on four variables

Frames, closed values, reasons, conflict sets

The backjump rule, and the example step by step

Completeness

Reasons are nogoods, and one invariant

Why an empty conflict set means "no solution"

Learned nogoods

Termination

The measure

Choosing the variable at each level, and why the proofs survive it

The heuristic's contract

Ordering against backjumping, measured

The jump-target pitfall

The correction: jump by stack position

Restarts with a dynamic order

Where the argument stops

Optimization: the bound is one more constraint

Optimization on three variables

CBJ with a nogood store, against CDCL

Done when

Reading

Next