Load raises the failure rate, the failure rate raises the amplification, and the amplification raises the load. Iterating that map from an offered load of 0.95 of capacity, with three retries and a ten per cent base failure rate, walks 0.95 → 1.06 → 1.11 → 1.17 → 1.24 and settles at 1.727 — above capacity, from a starting point that was comfortably inside it. The retries are not a response to the overload; they are the overload.
Core ideas
Two functions from the last two lessons, composed. Neither is new; the composition is, and so is the answer it lands on.
Above capacity the server cannot finish everything inside a client's patience, and the share it does finish is C/L, so the failure probability is base + (1 − base)·(L − C)/L. Below capacity it is just base. That is this lesson's model, stated rather than assumed, and the lab evaluates it rather than drawing a curve someone liked the shape of.
L′ = λ · attempts(p(L)). Feed a load in, get the failure rate it causes, get the amplification that failure rate produces, multiply the original offered load by it. The result is the load the service actually sees, and feeding it back in is one step of the iteration. Three steps by hand are enough to see which way it is going.
Starting at 0.95 of capacity with three retries and a 10% base failure rate, the iteration passes capacity on its first step and settles at 1.727. Nothing external changed: the offered load was serviceable throughout, and the policy meant to protect requests is what is drowning the service. Switching the retries off returns the load to 0.95.
The material
How the two earlier formulas compose into a feedback loop, what iterating it looks like, and why a bracket answers a question the iteration cannot afford to.
Write C for capacity, λ for the offered load before retries, base for the failure probability under no overload and r for the retry count. Define p(L) = base + (1 − base)·max(0, (L − C)/L) and L′ = λ · (1 − p(L)ʳ⁺¹)/(1 − p(L)). A fixed point is a load with L′ = L; the system's steady state is such a point, if one exists below the ceiling λ(r + 1).
Capacity is the unit: every load on this page is a multiple of C, so C = 1 and the offered load λ = 0.95 means ninety-five per cent of capacity. The base failure rate is what the service returns when it is not overloaded — a small tail of genuine errors, here ten per cent, which is high enough to make the iteration visible in five steps.
Work step two by hand and the loop stops being a metaphor. The load is 1.05545, so the overload share is (1.05545 − 1)/1.05545 = 0.05254, and the failure rate is 0.1 + 0.9 × 0.05254 = 0.14728. Four attempts at that rate cost 1 + 0.14728 + 0.02169 + 0.00319 = 1.17217, and 0.95 × 1.17217 is 1.11356. Every figure in the row came from the row above it.
The last column is worth a sentence of its own. The amplification carries the r-th power of the failure rate's denominator, so each step cubes it at r = 3: five digits, then thirteen, thirty-nine, a hundred and nineteen, three hundred and fifty-eight. By the seventh iterate the exact fraction has over three thousand digits, and the table stops and says so rather than silently rounding.
The iteration cannot reach the answer, so the lab does not use it to. The map is increasing in L and bounded above by λ(r + 1), so the function g(L) = L′ − L is non-negative at λ and non-positive at λ(r + 1), and a root is trapped between them. Bisection evaluates the map afresh at each probe, so nothing compounds and the denominators stay small.
The result is an enclosure rather than a rounded number: the fixed point is at least the lower bound and at most the upper one, and both ends are exact fractions. At this setting the two ends agree to five decimal places, which is why the readout prints the same figure twice.
0.95 of capacity was serviceable. With three retries the system settles at 1.727 of capacity and a 47.9% failure rate — and that failure rate is what sustains the amplification that produced the load. Turn the retries off in the lab and the load returns to 0.95 immediately. The overload has no external cause at all, which is exactly why it does not end when the original trigger does.
It is not gradual. At the same 10% base rate and three retries, an offered load of 0.80 settles at 0.8888 and one of 0.90 settles at 0.9999 — both below capacity and stable in a single step. 0.95 settles at 1.727. The system is fine, fine, and then eighty per cent over capacity, and the parameter that moved was five per cent of offered load.
Hold the offered load at 0.95 and vary only the policy: with no retries the fixed point is 0.95, with one retry 1.168, with two 1.439, with three 1.727. Every one of those is the same traffic against the same service. The policy is not a response to the overload in any of the last three cases; it is the whole of it.
Interactive
A feedback loop, iterated to its fixed point — which can sit above capacity
Method
Three of these are arithmetic you already have. The fourth is the question the arithmetic was for.
Write down what failure probability each load produces, and in what units capacity is measured. This lesson uses p(L) = base + (1 − base)(L − C)/L above capacity and base below it; a different model gives different numbers and the same shape of conclusion, but it has to be written down either way.
Failure rate from the load, amplification from the failure rate, load from the amplification times the original offered load. Three iterations are enough to see the direction, and doing them by hand is what makes the fourth believable.
The map is increasing and bounded by λ(r + 1), so bisect between the offered load and that ceiling. Iterating instead compounds the exact fractions until they are unusable, and rounding them is how a divergence gets reported as a convergence.
Above capacity is a storm; below it the policy is affordable. Then compare with the load that would have applied with no retries at all, because that difference is the part the system did to itself and the only part a policy change can remove.
A service running at ninety per cent of capacity with a five per cent base failure rate. The question on the table is whether to raise the retry count from two to three to improve a customer-visible error rate.
At ninety per cent offered load the extra retry is nearly free: the fixed point moves by one part in ten thousand and stays well inside capacity, because the load never crosses C and so the failure rate never leaves its base value. The feedback term is switched off entirely in this regime, which is why the policy looks harmless.
At ninety-eight per cent the same two policies both storm, and the one with the extra retry storms harder. The decision therefore is not about the retry count at all — it is about how close to capacity the service is allowed to run, because that is what decides whether the loop is engaged.
For a faded rehearsal, keep base = 0.05 and r = 2 and find the offered load at which the fixed point first exceeds capacity. The supplied first move is that while L ≤ C the map has the constant amplification 1.0525, so the loop only engages once λ × 1.0525 > 1. Solve that inequality by hand, then check the value against the lab's bracket by stepping the offered-load slider across it.
You should be able to state the failure model, apply the map by hand for three steps, bracket the fixed point between λ and λ(r + 1), and say how much of the resulting load is the system's own doing rather than its users'.
Something to change about this lesson? Write it down, tick it off when it is handled, and export the list when you want to act on it. Kept in this browser only: nothing is sent anywhere, and nobody is notified.
No recommendations yet for this lesson.