Capacity does not arrive when it is ordered. During a ramp the fleet is permanently the boot window behind demand, and the requests lost to that gap are the area between two lines: a triangle while the boot and the ramp overlap, then a rectangle for the rest of it. The only thing that removes it is capacity that was already warm.
Core ideas
The geometry is easy. What is worth taking from the page is what the area means: not slow requests, but requests that were never served.
Demand rises at r a second; capacity responds to the demand of τ seconds ago, because that is when the instances now arriving were ordered. So once the ramp is under way the gap is r·τ and it stays there: 50 × 90 = 4 500 rps here, at every instant from the boot window until the ramp stops. It is not a transient that closes as the fleet grows.
While the boot window and the ramp overlap, the gap is growing and the region is a triangle of ½·r·τ² = 202 500 requests. After that the gap is constant and the region is a rectangle, r·τ·(D − τ) = 945 000. Added, they are r·τ·(D − τ/2) = 1 147 500 — the closed form, and the two routes must agree.
ρ here is λ/μ, the same definition “λ, μ and ρ” gives, with μ the capacity actually in service rather than the capacity ordered. At the worst instant λ = 9 500 against μ = 5 000, so ρ = 19/10. Above one there is no steady state at all: the backlog grows at λ − μ = 4 500 requests a second, and 1/(1 − ρ) is not a number.
The material
Where the trapezoid comes from, why the backlog never drains, and what a warm reserve is actually buying.
Let demand rise from a level the fleet already serves at a rate r requests per second per second, for a ramp of duration D. Let τ be the time between ordering an instance and it serving traffic. With no warm reserve, the shortfall — the number of requests demanded and not served — is
r·τ·(D − τ/2) for D ≥ τ,
which is the area between the demand line and the capacity line over the ramp.
The derivation is geometry. For the first τ seconds capacity does not move at all, so the gap grows linearly from zero to r·τ: a triangle of area ½·r·τ². After that, capacity rises at the same rate as demand but τ seconds later, so the gap is a constant r·τ for the remaining D − τ seconds: a rectangle of area r·τ·(D − τ).
Both routes give the same integer, which is the check. The closed form is the area of a trapezoid: the mean of the two gaps — zero at the start and r·τ from the boot window on — taken over the ramp, with the τ/2 recording that the gap spent the first boot window opening rather than open.
It is worth being exact about what 1 147 500 counts. It is not requests that were slow; it is demand that arrived while there was no capacity for it. Whether those requests queue, retry or fail is a property of the system in front of the fleet, and every one of those outcomes is bad in a different way.
Every row above ρ = 1 is a row with no steady state. The backlog grows at λ − μ for as long as the row lasts, which at the worst instant is 4 500 requests a second. This is the same ρ as “λ, μ and ρ”, and the only thing this page adds is that μ must be the capacity in service rather than the capacity requested — the difference between those two is exactly the subject of the lesson.
Look at the last row. When the ramp stops, capacity catches up and the two lines end level at 20 000 rps. Level means no spare: there is no surplus with which to work through what accumulated, so the backlog built during the ramp is still there when the ramp is over. It drains only when demand falls, or never.
That is why “autoscaling handles spikes” is the wrong model. Autoscaling handles growth: given enough time it tracks a rising load with a fixed lag and a bounded gap. A spike is precisely the case where there is not enough time, and the mechanism arrives after the event it was meant to absorb.
Hold r·τ = 4 500 rps warm and the shortfall is zero, not small. The reserve covers exactly the capacity the boot window costs: for the first τ seconds it absorbs the growing gap, and by the time it is exhausted the first ordered instances are in service. Set the reserve to 2 000 rps instead and the shortfall is reduced rather than removed — the lab shades what is left.
A reserve is idle capacity, which is the waste of “Utilisation and Waste” deliberately bought. That is the honest framing of the trade: the shortfall is priced in requests and the reserve is priced in instance-hours, and choosing between them is a decision rather than an optimisation.
Two knobs change r·τ without buying a reserve, and both are worth more than they sound. Halving τ — a smaller image, a warmer pool, a faster health check — halves the gap and quarters the triangle. Scaling on a leading signal rather than on the utilisation that results from it starts the boot earlier, which is the same saving bought with a prediction instead of with money.
Interactive
A boot window is a shortfall with an area, and a reserve is what removes it
Method
The gap first, because it is one multiplication and everything else is its area.
r·τ is how far behind the fleet runs for the whole of the ramp: 4 500 rps here. Quote it in the same units as the load, because it is directly comparable with the base load — 4 500 against 5 000 is a fleet running at nearly double what it can serve.
Triangle plus rectangle, then r·τ·(D − τ/2). They must agree to the request. Two routes to one figure is what stops a factor of two in the triangle going unnoticed, and a factor of two here is half a million requests.
λ/μ with μ the capacity in service. If it is above 1, say so in those terms: no steady state, backlog growing at λ − μ. A utilisation above one is not a busy system, and reporting it as 190% invites it to be read as one.
Compare capacity and demand at the end. Level means the backlog never drains on its own. This single comparison is what turns the shortfall from a transient into a permanent debt, and it is one subtraction.
r·τ of warm capacity removes the area exactly. That capacity is idle most of the time, so the decision is instance-hours against requests, and both sides of it now have numbers.
The base load is 5 000 rps and the fleet is exactly sized for it when the ramp begins.
The check line matters more than it looks. ½·r·τ² + r·τ·(D − τ) and r·τ·(D − τ/2) are the same quantity written two ways, and computing both is how you find out that you used D where you meant D − τ.
The nothing spare line is the one that changes what the shortfall means. If the fleet ended above demand, the backlog would clear and the shortfall would be a latency event. It ends level, so the shortfall is a permanent loss of just over a million requests, and no later behaviour of the autoscaler recovers it.
For a faded attempt, halve the boot window to 45 s and keep everything else. Predict before computing what happens to the triangle and to the total — one of them falls by a factor of four and the other does not — then read both off the lab and say what a warm reserve would now have to be.
You should be able to compute r·τ, produce the shortfall by both the two-region route and the closed form, evaluate ρ = λ/μ at the worst instant with μ the capacity in service, say whether the backlog drains, and name the reserve that removes the area.
Something to change about this lesson? Write it down, tick it off when it is handled, and export the list when you want to act on it. Kept in this browser only: nothing is sent anywhere, and nobody is notified.
No recommendations yet for this lesson.