Exponential backoff decays the retry rate geometrically, which fixes the total. It does nothing at all to the peak: every client that failed at the same instant schedules its first retry at the same millisecond, so the retries arrive in waves at 1, 2, 4, 8, 16 and 32 seconds. Adding jitter to the delay spreads the same retries over the interval, and at 600 clients it cuts the peak from 3000 a second to 925 without changing the policy.
Core ideas
Two properties of a retry schedule that are usually named together and are controlled by different parts of it.
An outage is, by definition, a moment at which all of a service's clients fail at once. Each then schedules retry i at base × 2ⁱ⁻¹ after its own failure, and since the failures were simultaneous the retries are too: the first wave at one second, the next at two, then four, eight, sixteen and thirty-two. Doubling a constant leaves it constant.
Geometric delays mean a client makes six attempts in forty seconds rather than forty, which is a real reduction in total work. But all six of those attempts land in six instants, so the instantaneous rate at each instant is the full client population. The peak is what decides whether a recovering service stays up, and backoff alone does not touch it.
Drawing the delay uniformly from [d(1 − w/2), d(1 + w/2)) turns each wave into a smear over an interval proportional to the delay. With 600 clients, a one-second base and full-width jitter the peak falls from 3000 retries a second to 925 — a factor of 3.24 — with the same clients, the same policy and the same seed. Nothing was delayed longer on average; it was spread.
The material
What a synchronised retry schedule looks like, why the measurement resolution matters, and what jitter costs.
Under exponential backoff the delay before attempt i is base × 2ⁱ⁻¹. Under jitter of width w the delay is instead drawn uniformly from [d(1 − w/2), d(1 + w/2)), where d is that same nominal delay. Jitter of 100% therefore spreads attempt i uniformly over an interval of length d centred on d.
The lab draws its delays from a seeded linear congruential generator — multiplier 1 103 515 245, increment 12 345, modulus 2³¹ — so the histogram is the same every time the page is opened and a reader can check what they were told. Change the seed and the peak moves a little; change the jitter and it moves a lot.
The slot width is the whole measurement and the lab says so on its face. Without jitter every client in a wave retries in the same millisecond, so the true instantaneous spike is six hundred requests in one millisecond; any bucket wider than that understates it. At the 200 ms resolution the lab uses, the reported 3000 a second is a floor rather than a measurement.
Two things changed and they are worth separating. The peak fell by a factor of 3.24, which is what jitter is for. The total fell slightly too, from 3600 to 3323, which is an artefact of the outage boundary: a jittered attempt that lands after the service has recovered succeeds and the client stops, where its unjittered twin would have retried once more. Jitter is not a way to reduce total load; the small change here is a side effect.
A service coming back after a twenty-second outage has a first second in which it is either hit by 3000 requests or by about 900, depending on a single configuration parameter. If its capacity is 1500 a second, the first case fails again immediately — and every client that fails re-enters the backoff schedule, which is the previous lesson's loop with a synchronised phase. The second case is served.
Jitter does not increase the mean delay: the interval [d(1 − w/2), d(1 + w/2)) is centred on d, so half the draws are earlier than the nominal delay and half later. What it costs is predictability — a particular client may retry sooner or later than the schedule says — and that is almost never a cost anyone is actually paying.
Full-width jitter, w = 100%, spreads attempt i over [0.5d, 1.5d). Half-width leaves the waves partly visible: at 50% the same run peaks at 1240 a second rather than 925, because each wave is smeared over half the interval and neighbouring waves no longer overlap enough to fill the gaps. The lab's jitter slider makes that comparison directly.
Backoff without a cap reaches a 32-second delay by the sixth attempt and 512 seconds by the tenth, which is long enough that the client has effectively given up without saying so. Real policies cap the delay, and a capped backoff with jitter is the standard shape: exponential up to a ceiling, then uniform on a jittered interval around that ceiling. The waves this lesson is about are a property of the early attempts, which is exactly where the cap does not apply.
Interactive
Two seeded histograms of the same outage, one of them synchronised
Method
The third step is the one that is usually skipped, and skipping it makes the policy look better than it is by whatever factor the bucket is too wide.
That is what an outage is, and it is the case the schedule has to survive. A policy evaluated against independent, uniformly distributed failures is being evaluated against the case it was never needed for.
base × 2ⁱ⁻¹ for each attempt, until the delay exceeds the outage or the attempt budget runs out: 1, 2, 4, 8, 16, 32 seconds at a one-second base. Each of those instants receives the entire client population.
A one-second bucket turns six hundred simultaneous retries into a comfortable-looking six hundred a second. Use a slot narrow enough to resolve the spike, and report the slot width alongside the peak, because without jitter the true spike is instantaneous and every bucket understates it.
The total barely moves; the peak falls by roughly the factor the spreading achieves — 3.24 here at full width. Compare the jittered peak with the capacity the service has while it is recovering, which is generally less than its steady-state capacity.
A service with 2000 clients and a recovery capacity of 1200 requests a second comes back from a thirty-second outage. The question is whether it survives its own clients, and the answer is a comparison of two numbers.
Jitter cut the peak by a factor of 3.25 and the result is still nearly three times what the service can serve, because spreading 2000 clients over roughly a second is still thousands a second. This is the honest result: jitter is necessary and it is not sufficient, and the remaining gap has to be closed somewhere else — a larger base delay, a retry budget that lets only a fraction of clients retry at all, or refusing the excess at the door.
Note which number the design has to be compared against. Recovery capacity is lower than steady-state capacity: caches are cold, connection pools are empty and the first requests after an outage are the expensive ones. Sizing the first wave against the steady-state figure is how a service that should have survived does not.
For a faded rehearsal, keep the 2000 clients and the 1200 per second, and find the backoff base at which the jittered peak first fits inside capacity. The supplied first move is that full-width jitter spreads wave one over an interval about as long as the base, so the peak falls roughly as clients/base — which is a lower bound, because later waves overlap the first. Estimate the base from that, then find the true answer in the lab by stepping the base slider and reading the jittered peak, and say how far your estimate was out.
You should be able to say where the waves of an unjittered schedule land, measure a peak at a resolution that resolves them, predict the effect of jitter on the peak and on the total separately, and compare a first wave against a recovering service's capacity rather than its steady-state one.
Something to change about this lesson? Write it down, tick it off when it is handled, and export the list when you want to act on it. Kept in this browser only: nothing is sent anywhere, and nobody is notified.
No recommendations yet for this lesson.