Split traffic evenly across four servers, each with its own queue, and the utilisation is identical to pooling them behind one queue — but the waiting is not. At λ = 3, μ = 1 and s = 4 the pooled queue waits 27/53 of a second where a separate queue waits 3. Erlang C is the instrument that prices the difference.
Core ideas
The design idea is the whole lesson and the formula is only the instrument that puts a number on it.
That sentence is the entire mechanism. Four queues of one server each will, sooner or later, have a server with nothing to do while somebody waits in another line, and every such moment is capacity that was paid for and not used. One queue in front of four servers makes that state impossible.
Splitting λ four ways across four servers gives each ρ = λ/(4μ), exactly the pooled ρ = a/s. So this is not a comparison between a busy system and a quiet one: the same work, the same servers, the same fraction of time busy, and a different amount of waiting.
The formula below comes out of a birth–death chain whose service rate is min(n, s)μ, and that derivation is Operations Research, markov-chains-decisions-and-queues/multiple-servers-and-erlang-c. What is owned here is the design conclusion, and the formula is used as an instrument for reaching it.
The material
The offered load in servers, the Erlang C formula as stated, the worked comparison, and what pooling costs in practice.
M/M/s is the M/M/1 model with s identical servers behind one queue. The offered load a = λ/μ is measured in servers: it is how many servers the work would keep permanently busy. The per-server utilisation is ρ = a/s, and the system settles when ρ < 1. The server count on this path is always s.
The probability that an arrival has to wait at all is C(s,a) = [a^s/(s!(1 − ρ))] ÷ [Σ_(n<s) aⁿ/n! + a^s/(s!(1 − ρ))].
From it, L_q = C(s,a)·ρ/(1 − ρ) and W_q = L_q/λ by Little's Law. The derivation is not given here: it is Operations Research, markov-chains-decisions-and-queues/multiple-servers-and-erlang-c, and this course's list of what it does not cover says the same.
At s = 1 the formula collapses to C = ρ and the single-server results of “The M/M/1 Queue”, which is the check worth doing first on any implementation of it.
Nearly six times the wait, for the same machines under the same load. The split system is not overloaded and is not misconfigured; it is simply wasting the moments when one of its servers is free and one of its queues is not.
Pooling saves more seconds as the system gets busier and a larger multiple of the wait when it is quiet, and both halves of that are worth carrying. At ρ = 1/2 the pooled queue waits 2/23 of a second against 1 second separate — a ratio of 23/2 but a saving of under a second. At ρ = 3/4 the ratio is 53/9 and the saving is about two and a half seconds. At ρ = 9/10 the pooled wait is 4374/2221 against 9, a ratio of about 4.6 and a saving of about seven seconds.
So the headline “pooling matters most under load” is true of the seconds and false of the ratio, and it is the seconds that show up in an incident. The lab prints both rows at whatever λ, μ and s you set, so the two readings can be compared directly rather than argued about.
Pooling is not free, and the costs are real ones rather than rounding. A shared queue needs a dispatcher, which is a component that can itself queue or fail. Work that is not interchangeable — a session pinned to a server, a cache warmed for one shard — loses locality when it is pooled, and that shows up as a larger S, which the previous lesson says is multiplied by everything. The question is whether the factor gained on waiting exceeds the factor lost on service.
The same argument explains why a few large machines usually beat many small ones at the same total capacity, why a shared thread pool beats a pool per endpoint, and why sharding a workload that does not need to be sharded is a latency decision as well as a data one — which is where Partitioning and Load Balancing picks it up.
Interactive
One queue of s servers against s queues of one
Method
The comparison is only honest if both arms carry the same total load on the same servers.
a = λ/μ. This is the number of servers the work would keep permanently busy, and it is the input Erlang C actually takes.
ρ = a/s < 1, or nothing settles in either arm. Both arms have the same ρ by construction, which is what makes the comparison mean anything.
C(s,a), then L_q = C·ρ/(1 − ρ), then W_q = L_q/λ. The lab prints all three exactly, and s = 1 should reproduce M/M/1.
It is an M/M/1 queue at the same ρ, so L_q = ρ²/(1 − ρ) and W_q = L_q/(λ/s), the closed forms of “The M/M/1 Queue”.
The ratio is largest at light load and the absolute saving is largest at heavy load, and a decision usually turns on the second. Say which you are quoting.
Three requests a second into servers that manage one a second each. The two arrangements are indistinguishable on a utilisation dashboard.
Half of the pooled arrivals do not wait at all — C(4,3) ≈ 0.51 is the chance of waiting, and the other half find a free server. In the split arrangement an arrival joins whichever line it was routed to and waits behind whatever is in it, with three idle-capable servers elsewhere.
For a faded rehearsal, keep μ = 1 and s = 4 and drop the load to λ = 2. The supplied first move is a = 2 and ρ = 1/2. Compute the pooled W_q, compare it with the separate 1 second, and notice that the ratio went up while the seconds saved went down.
You should be able to compute a, ρ and the pooled W_q from Erlang C, compute a separate queue's W_q from the M/M/1 forms, report both the ratio and the seconds, and say what would have to be true of the work for pooling to be a bad trade.
Something to change about this lesson? Write it down, tick it off when it is handled, and export the list when you want to act on it. Kept in this browser only: nothing is sent anywhere, and nobody is notified.
No recommendations yet for this lesson.