Learn
Lesson · Intractability and Approximation · Exact answers, still exponential

A Bound the Search Can See

The first honest response to a hard problem is to solve it exactly and try to visit less of the search tree. The fractional relaxation from Greedy Algorithms and Matroids is an upper bound on anything a branch can still reach, so a branch whose bound is no better than the best answer so far can be cut. On the loaded instance that is 14 nodes against 44, with both searches returning 16 and an exhaustive check over 32 subsets agreeing.

at a node: value so far + fractional optimum of what remains = an upper bound bound ≤ best found so far ⟹ cut the branch; it cannot hold anything better 5 items, capacity 10: value 16 = the optimum, from 32 subsets 14 nodes with the bound, 44 without, 4 branches cut pruning changes the node count and never the answer
One search, two node counts, and one answer

Core ideas

What this lesson establishes

The first idea is what makes a bound usable at all. The second is that pruning is not a heuristic, which the lab checks rather than asserts. The third is the reading of the three numbers in the panel, one of which is not measured in the same units as the other two.

1

An upper bound on any completion licenses a cut

At a node the search has decided some prefix of the items and has a remaining capacity. Allow the remaining items to be taken fractionally and greed solves that relaxation exactly — it is the algorithm Greedy Algorithms and Matroids proved correct. Since allowing fractions can only help, the value so far plus the fractional optimum of the rest is at least as large as anything the branch can actually reach. If that number is no better than an answer already in hand, nothing in the branch can be, and the branch is cut without being explored.

2

Pruning is not a heuristic

This is the property to insist on, because the words sound like a trade. The bounded search and the unbounded search must return the same value, and both must agree with an exhaustive enumeration. The panel runs all three and prints the verdict: 16, 16, and 16 from 32 subsets. A bound that cut a branch containing the optimum would be a wrong algorithm rather than a fast one, and the check is what distinguishes the two.

3

Two of the three numbers are nodes and one is subsets

The panel reports 14 nodes with the bound, 44 without, and 32 for the whole tree. The third is 2^n, the number of subsets an exhaustive search examines — the leaves. The first two count nodes of a recursion, which includes every internal node on the way down. That is why the unbounded search can visit more nodes than there are subsets, and it is not a contradiction: 44 nodes produce fewer than 32 complete subsets between them, because most of them are partial.

The material

The bound, the cut, and the three counts

What the relaxation gives, why the cut is safe, what the lab measured on this instance, and what the same code does on two others.

Definition · Branch and bound

A search over partial solutions that, at each node, computes an upper bound on the best value any completion of that partial solution could have, and abandons the node when the bound does not beat the best complete solution found so far. The bound must never underestimate a completion, or the search may discard the optimum.

Theorem · The fractional relaxation is a valid bound

Let a node have decided items 1 through i - 1, with value v taken and capacity c remaining. Then every completion of that node has value at most v plus the fractional knapsack optimum of items i through n with capacity c.

A

n

y

c

o

m

p

l

e

t

i

o

n

c

h

o

o

s

e

s

a

s

u

b

s

e

t

o

f

t

h

e

r

e

m

a

i

n

i

n

g

i

t

e

m

s

f

i

t

t

i

n

g

i

n

`

c

`

.

T

h

a

t

s

u

b

s

e

t

i

s

a

l

s

o

a

f

e

a

s

i

b

l

e

f

r

a

c

t

i

o

n

a

l

s

o

l

u

t

i

o

n

&

m

d

a

s

h

;

t

a

k

e

e

a

c

h

c

h

o

s

e

n

i

t

e

m

i

n

f

u

l

l

&

m

d

a

s

h

;

s

o

i

t

s

v

a

l

u

e

i

s

a

t

m

o

s

t

t

h

e

f

r

a

c

t

i

o

n

a

l

o

p

t

i

m

u

m

o

f

t

h

e

s

a

m

e

s

u

b

-

i

n

s

t

a

n

c

e

.

A

d

d

i

n

g

`

v

`

t

o

b

o

t

h

s

i

d

e

s

g

i

v

e

s

t

h

e

c

l

a

i

m

.

T

h

e

f

r

a

c

t

i

o

n

a

l

o

p

t

i

m

u

m

i

s

c

o

m

p

u

t

e

d

e

x

a

c

t

l

y

b

y

t

a

k

i

n

g

i

t

e

m

s

i

n

d

e

c

r

e

a

s

i

n

g

v

a

l

u

e

p

e

r

u

n

i

t

w

e

i

g

h

t

a

n

d

s

p

l

i

t

t

i

n

g

t

h

e

o

n

e

t

h

a

t

d

o

e

s

n

o

t

f

i

t

,

w

h

i

c

h

i

s

t

h

e

g

r

e

e

d

y

r

u

l

e

p

r

o

v

e

d

c

o

r

r

e

c

t

i

n

G

r

e

e

d

y

A

l

g

o

r

i

t

h

m

s

a

n

d

M

a

t

r

o

i

d

s

.

□

What the lab measured

The instance is the five items 3:5, 4:6, 5:8, 2:3, 6:9 with a capacity of 10. The panel reports a value of 16, the same 16 from an exhaustive search over 32 subsets, both searches correct, 14 nodes with the bound, 44 without, 32 for the whole tree, and 4 branches cut with the bound against 0 without. The optimum takes items 3:5, 5:8 and 2:3, filling the sack exactly at weight 10 for value 16.

At the root the fractional bound is 16, which is exactly the integer optimum on this instance. That is a coincidence of the numbers rather than a general fact, and it is worth noticing because it is the best case for a bound: it knows the answer before the search starts, and still has to prove it by exploring until every alternative is cut.

Example · The same code, a different instance

Switch to the example with one heavy prize and a lot of filler — 9:30 and six copies of 1:2, capacity 9 — and the counts become 3 nodes with the bound against 135 without. The single item fills the sack, the fractional bound at the root is 30, taking it immediately gives 30, and everything else is cut at once. Switch to the example where every item has the same density and it is 30 against 86, with only 2 branches cut, even though the root bound of 26 is again exactly the optimum.

Those three instances are one algorithm and node counts of 14, 3 and 30. The ratio between bounded and unbounded search moves from about 3 to 45 to under 3, and the number of branches cut moves from 4 to 2 to 2 without tracking either. A tight bound at the root does not imply heavy pruning, because pruning compares the bound against the best answer found so far, and early in the search that is a poor answer.

What is therefore not established

Nothing about the worst case. Branch and bound on the knapsack is exponential in the worst case and the bound is not a complexity improvement — there are instances on which nothing is cut. What the node counts establish is what the bound bought on the instance in front of you, which is a real and useful fact about that instance and is the only kind of fact a counter can produce.

This is the first of the strategies and the one that gives up least: the answer is exact, the certificate is the search itself, and the cost is a number you can watch. Every later strategy trades some of that away — the guarantee of optimality, the range of instances, or the accuracy — and each trade is worth making only against a measurement like this one.

Interactive

Branch and bound: what the bound buys, in nodes

The fractional relaxation is an upper bound on any completion, so a branch that cannot beat the best found is cut — and the answer is still exactly the optimum

Branch and bound: what the bound buys, in nodesthe fractional relaxation is an upper bound on any completion, so a branch below the best so far can be cut
with the fractional boundwithout itthe whole tree, 2 to the n

Method

Adding a bound to a search, and proving you have not broken it

The bound is the easy part. Establishing that it cannot cut the optimum is the work.

1

Find a relaxation you can solve exactly

Drop a constraint until what is left is easy, and make sure dropping it can only improve the objective. Fractional items instead of whole ones is the standard move for the knapsack, and the relaxed problem has a greedy algorithm that was proved correct before this course started.

2

Prove the bound never underestimates a completion

Every completion must be a feasible solution of the relaxation, so that its value is at most the relaxed optimum. If some completion is not feasible in the relaxed problem, the bound is not a bound and the search will quietly discard optima.

3

Compare the bound against the best found so far, not against the final answer

The search prunes with the information it has at the time. This matters when reading a trace: a node whose bound is below the final answer was not necessarily cut, because the running best may have been worse when the node was reached. The lab's trace says only what the two numbers on the row settle, and the count of branches actually cut is a separate counter.

4

Run the search with the bound switched off and require the same answer

Two searches and an exhaustive enumeration, three answers, one number. This is the check that separates a correct pruning rule from a heuristic that happens to work on the instance loaded, and it costs one control on the panel.

5

Report the node counts as a measurement of this instance

Say what the bound bought here. Do not convert it into a factor and do not attach it to the algorithm: the same code on the panel's other examples gives 3 nodes and 30 nodes, and neither is the algorithm's number.

What a bound may and may not do

Score 0 / 0

Twelve bounded nodes on five items, and the four cuts

Items 3:5, 4:6, 5:8, 2:3, 6:9, capacity 10. Each row is a node where the bound was computed: the value decided so far, and the bound on any completion. The search ends with 16.

node depth value so far bound on any completion 1 0 0 16 2 1 5 16 3 2 11 79/5 4 3 11 31/2 5 4 14 31/2 6 4 11 31/2 7 2 5 16 8 3 13 16 9 4 16 16 10 4 13 16 11 3 5 31/2 12 1 0 31/2 best value 16 items 3:5 + 5:8 + 2:3 weight 10 of 10 nodes with the bound 14 branches cut 4 nodes without it 44 branches cut 0 the whole tree, 2 to the n 32 subsets exhaustive optimum over those 32 subsets 16

The node count is 14 and the table has 12 rows, and the difference is not an error. A node that has decided every item has nothing left to bound, so it is visited and counted without a bound being computed; there are two of those. Reading the table as the node count is the kind of small mismatch worth checking rather than assuming.

The bound of 31/2 at nodes 4, 5, 6, 11 and 12 is the fractional optimum doing its work: 15.5 is above 14 but below 16, so once 16 is in hand every one of those branches is refused, and before 16 is in hand they are not. Node 9 is where 16 first appears, and every cut in this run happens after it.

For a faded rehearsal, keep the items and raise the capacity to 14 before running it. The supplied first move: at capacity 14 the three heaviest items weigh 15, so the sack still cannot hold everything, and the fractional bound at the root will land between the best three-item value and the best four-item one. Predict whether the node count goes up or down, then check — and notice that whichever way it goes, it says nothing about capacity 15.

Common mistakes

1
Trusting a bound that has not been proved to be oneA quantity that usually exceeds the completions is not a bound. If any completion can beat it, the search will cut the branch holding the optimum and return a confident wrong answer with no symptom. The check on the panel — the same answer with the bound off, and again by exhaustive search — exists for this.
2
Reading a row of the trace as a branch that was prunedThe trace records the bound at each node and not the best value known at that moment, and the search prunes against the running best. So a row whose bound is below the final answer may or may not have been cut. The number of branches actually cut is the algorithm's own counter and it is in the panel above the table.
3
Turning the node counts into a claim about the algorithm14 against 44 is this instance. The bound is not an asymptotic improvement and no arrangement of these three numbers becomes one. Say what the bound bought here, and if you need a claim about all instances, prove one.

Check your understanding

✓
Finish when you can add a bound to a search and demonstrate that it cannot cut the optimum.

You should be able to choose a relaxation and say why relaxing can only improve the objective, prove the resulting quantity bounds every completion, explain why the comparison is against the running best rather than the final answer, and report node counts as measurements of the instance they were taken on.

Note: An exact search can also be made to depend on something other than the size of the input. “Parameterising the Budget” puts the exponential in a number the reader chooses, and prints 2^k · n beside 2^n with both computed.

Kept in this browser only, for you. Nothing is sent anywhere and nothing checks it. Carry it to another device.

Feedback and recommendations

Something to change about this lesson? Write it down, tick it off when it is handled, and export the list when you want to act on it. Kept in this browser only: nothing is sent anywhere, and nobody is notified.

No recommendations yet for this lesson.