Learn
Lesson · Queues and Utilisation · Variability and pooling

Variability: the Kingman Approximation

Two services with the same mean service time can wait very differently, and Kingman's approximation says by how much: the queueing time factorises into utilisation, variability and service time. Halve the variability term and you halve the wait. The arithmetic is exact; the model is an approximation, and the lab shows it missing.

W_q ≈ (ρ/(1 − ρ)) · ((c_a² + c_s²)/2) · S an approximation, not a theorem c = σ/mean c² = 0 for clockwork, 1 for memoryless c_a² = c_s² = 1 reproduces M/M/1's W_q exactly ρ = 4/5, S = 2: W_q = 8 at (1, 1); 4 at (1, 0) halved at the chain's own c²: Kingman 22/5, exact 4 about 10% high
Three factors, of which the middle one is a design choice

Core ideas

What this lesson establishes

The mean service time is not enough information to predict a wait, and this is the formula that says what else is needed.

1

Variability is a second parameter

Two queues can have identical λ, identical μ and identical ρ and wait differently, because one of them delivers its work in a steadier stream or finishes it in a more predictable time. The coefficient of variation c = σ/mean is the number that carries that difference, and it is dimensionless.

2

The formula factorises, which makes it a design tool

W_q is a utilisation factor ρ/(1 − ρ) times a variability factor (c_a² + c_s²)/2 times S. Each factor can be attacked separately: buy headroom, smooth the traffic, or make the work faster. Halving the variability term halves the wait, exactly and without buying a machine.

3

It is an approximation, and it is derived nowhere in this library

Kingman's result is a heavy-traffic limit — it becomes accurate as ρ approaches one and is read here at moderate load. The arithmetic on this page is exact for rational inputs, and the model is what is approximate. The lab makes that concrete by putting the formula's answer next to an exact one and letting them disagree.

The material

Utilisation × variability × service time

The coefficient of variation, the approximation it feeds, the M/M/1 case it reproduces, and the exact answer it misses.

Definition · Coefficient of variation

For a positive random quantity with mean m and standard deviation σ, the coefficient of variation is c = σ/m, and c² is what appears in the formula below. Clockwork — every gap or every service time identical — has c² = 0. A memoryless quantity has c² = 1. Anything burstier than memoryless has c² > 1. c_a describes the gaps between arrivals and c_s the service times; these are the only cs on this course and neither is ever a count of servers.

Kingman's approximation for a single server is then, for ρ < 1:

W_q ≈ (ρ/(1 − ρ)) × ((c_a² + c_s²)/2) × S utilisation variability the work itself ρ = 4/5, S = 2 slots c_a² = 1, c_s² = 1 (4)(1)(2) = 8 the M/M/1 answer c_a² = 1, c_s² = 0 (4)(1/2)(2) = 4 deterministic service

The first of those two lines is a useful check rather than a coincidence: at c_a² = c_s² = 1 the formula returns exactly ρS/(1 − ρ), which is the W_q = L_q/λ of the M/M/1 lesson. If it did not, the disagreement further down this page would be an arithmetic bug rather than a modelling fact.

The second line is the design lever. Making the service time deterministic — same work, same duration, every time — takes c_s² from 1 to 0 and halves the queueing time, at the same utilisation and the same mean service time. Nothing was bought. An M/D/1 queue waits half of what an M/M/1 queue waits, and the formula says so in one factor.

Where it misses, and by how much

The slotted chain of “Why Queues Form at ρ < 1” has coefficients of its own, and they are not free parameters: Bernoulli arrivals leave geometric gaps with c_a² = 1 − p, and geometric service has c_s² = 1 − q. At p = 2/5 and q = 1/2 that is c_a² = 3/5 and c_s² = 1/2. Feeding the formula its own model's coefficients is the fairest test available.

p = 2/5, q = 1/2, ρ = 4/5, S = 1/q = 2 slots chain's own c_a² = 1 − p = 3/5 c_s² = 1 − q = 1/2 Kingman (4)((3/5 + 1/2)/2)(2) = 22/5 = 4.4 slots exact Geo/Geo/1 L_q/p = (8/5)/(2/5) = 4 slots about 10% high, in the direction it is always high at moderate ρ

The formula is not wrong in the sense of a mistake; it is an approximation being read outside the regime it was derived for, and the error it makes is of the size approximations make. The exact answer here belongs to the discrete chain; the Pollaczek–Khinchine formula that would give the continuous-time answer for general service times is not derived anywhere in this library either, and this course says so rather than implying otherwise.

The lab prints four bars: the formula at whatever coefficients you set, the formula at the chain's own coefficients, the exact chain, and a seeded simulation of the same chain. The simulation is a sample and moves with the seed; the two exact bars do not move; and the gap between Kingman's bar and the exact one is the point of the page.

Used as a sensitivity tool the approximation is excellent, and that is how to use it. It answers “what would halving the variance of this stage be worth?” with a factor, and factors survive an approximate model far better than absolute numbers do. Used as a promise it is a number with an unstated error bar.

  • Lower c_s²: make the work uniform — fixed batch sizes, a bounded unit of work, the same code path for every request.
  • Lower c_a²: smooth the arrivals — a token bucket in front, jittered retries, client-side pacing rather than synchronised polling.
  • Raise neither by accident: unbounded retries and synchronised timeouts are variability injected into c_a², and this formula prices them.

Interactive

Variability, and what it costs

Kingman's formula against the exact chain and against a run

Kingman's approximationexact arithmetic on an approximate model, held against the exact chain and a run
Kingmanexact Geo/Geo/1simulatedthe disagreement

Method

Pricing a variability change

Work in factors, not in absolute milliseconds, and the approximation earns its keep.

1

Get ρ and S first

The utilisation factor ρ/(1 − ρ) and the service time are shared by every scenario you are about to compare, so compute them once.

2

Estimate the two coefficients

c² = 0 for anything clockwork, 1 for anything memoryless, and above 1 for anything burstier. A measured σ/m from a trace is better, and for a first pass the three anchors are usually enough.

3

Compute the wait, and then compute it again with one factor changed

The ratio between the two answers is the part of this that survives the approximation. Report the ratio, and report the absolute number only with the model named beside it.

4

Check the ratio against something exact where you can

At c_a² = c_s² = 1 the formula must return the M/M/1 answer, and the lab puts the exact discrete chain next to it at the chain's own coefficients. A formula that agrees where it should and misses by 10% where it should is behaving exactly as an approximation behaves.

Three factors

Score 0 / 0

What deterministic service is worth at ρ = 4/5

One utilisation, one mean service time, and the variability factor moved from memoryless to clockwork — then the same formula held against an exact answer.

ρ = 4/5 S = 2 slots ρ/(1 − ρ) = 4 memoryless service c_a² = 1, c_s² = 1 W_q = 4 × (1 + 1)/2 × 2 = 8 slots and this is exactly M/M/1's W_q at ρ = 4/5, S = 2 clockwork service c_a² = 1, c_s² = 0 W_q = 4 × (1 + 0)/2 × 2 = 4 slots the saving 8 → 4, a factor of 2, with no machine bought now the fair test: the slotted chain's own coefficients p = 2/5, q = 1/2 → c_a² = 1 − p = 3/5, c_s² = 1 − q = 1/2 Kingman 4 × (3/5 + 1/2)/2 × 2 = 22/5 = 4.4 slots exact chain W_q = 4 slots

The factor of two in the middle block is the lesson's design claim, and it is the one part of this page an approximation is reliable for: both answers carry the same modelling error, so the ratio between them is far better determined than either number alone.

The last block is the honesty. Given the chain's true coefficients the formula still says 22/5 where the exact answer is 4. That is an approximation behaving normally, and it is why this course states Kingman's result rather than proving it and never quotes it as an exact wait.

For a faded rehearsal, keep ρ = 4/5 and S = 2 and set c_a² = 2 with c_s² = 1 — arrivals burstier than memoryless. The supplied first move is that the variability factor becomes 3/2. Predict W_q and say what multiple of the M/M/1 answer it is.

Common mistakes

1
Believing only the mean service time mattersTwo stages with identical means and different spreads produce different waits, and the ratio between them can be a factor of two or more. Any latency investigation that has only measured means has not yet measured the quantity this formula is a function of.
2
Treating an exact calculation as an exact answerEvery arithmetic step on this page is a rational operation, and the result is still approximate, because the model is. Fed the slotted chain's own coefficients the formula says 22/5 against an exact 4. Quote factors, quote the model, and do not promise the absolute number.
3
Reading c as a count of serversc_a and c_s are coefficients of variation, dimensionless ratios of a standard deviation to a mean. The server count on this course is s, and the queue with several servers is written M/M/s. A c on this page is never a number of machines.

Check your understanding

✓
Finish when you can say what halving the variance of a stage is worth, as a factor.

You should be able to compute W_q from ρ, two coefficients of variation and S, show that the formula reproduces M/M/1 at c² = 1, say where its answer sits relative to an exact one, and name it as an approximation every time you quote it.

Note: Variability was the first of the four controls. “Many Servers: Pooling, and Erlang C” is the second and the least intuitive: at an identical utilisation, one queue in front of several servers waits a fraction of what the same servers wait behind separate queues.

Kept in this browser only, for you. Nothing is sent anywhere and nothing checks it. Carry it to another device.

Feedback and recommendations

Something to change about this lesson? Write it down, tick it off when it is handled, and export the list when you want to act on it. Kept in this browser only: nothing is sent anywhere, and nobody is notified.

No recommendations yet for this lesson.