Three symbols carry the whole course. λ is how fast work arrives, μ is how fast one server finishes it, and ρ = λ/μ is the fraction of the time that server is busy. The trap is the middle one: μ is a rate and not a duration, so a service taking 2 ms has μ = 500 per second.
Core ideas
One of these three symbols is a rate written as the reciprocal of a duration, and most of the confusion on this course starts there.
A request that takes S = 1 ms is served at μ = 1000 per second, not at μ = 1. The two are reciprocals, μ = 1/S, and they are never interchangeable. When S is quoted in milliseconds the conversion carries a thousand with it: μ = 1000/S.
ρ = λ/μ divides a rate by a rate, so the seconds cancel and what is left is a pure number: the fraction of time the one server is busy. At ρ = 4/5 it works four seconds in every five and is idle the fifth, and that idle fifth is the only thing a burst has to land in.
Every closed form later in this course divides by 1 − ρ, and at ρ ≥ 1 that is zero or negative because there is no settled queue length to report. What there is instead is a slope: work accumulates at λ − μ a second and keeps doing it until something changes.
The material
What each symbol is, what it is measured in, and what changes at ρ = 1.
The arrival rate λ is the mean number of arrivals per unit time. The mean service time S is how long one server takes over one job. The service rate μ = 1/S is how many jobs that server can finish per unit time when it never waits for work.
Both λ and μ are rates and S is a duration. Keeping that straight is the whole of this lesson, because the two quantities a designer is handed are usually a rate (requests per second, from a dashboard) and a duration (a service time in milliseconds, from a trace), and they cannot be divided until one of them has been turned into the other's kind.
S = 1 ms is 1/1000 of a second, so μ = 1000 per second. Then ρ = 800/1000 = 4/5: the server is busy four fifths of the time. It is not 800 times too slow and it is not running at 800; it has a fifth of itself spare.
The utilisation ρ = λ/μ is the long-run fraction of time a single server is busy. It is dimensionless: a rate divided by a rate. Equivalently it is the offered load measured in servers — ρ = 4/5 says the work offered would keep four fifths of one server occupied.
Doubling the service time halves μ, and ρ doubles with it. This is the cheapest sensitivity on the course and the one most often missed: nothing about the offered traffic has to change for a system to cross ρ = 1. A slower dependency, a larger payload, a colder cache — each of them moves μ and therefore moves ρ.
S = 2 ms gives μ = 500 per second, so ρ = 800/500 = 8/5. The load did not change. The service time did, and that was enough to put the system above one.
At ρ ≥ 1 work arrives at least as fast as it can be finished, so the amount waiting has no long-run average at all. It grows at λ − μ per second for as long as the condition lasts. That is a straight line, not a queue, and quoting a queue length for it is the first mistake this course can catch.
Below one the queue does settle, and the rest of this course is about what it settles at. The one thing to carry forward from here is that ρ is the input to all of it: every later formula is a function of ρ alone, and the closer ρ is to one the more violently that function moves.
Interactive
ρ = λ/μ is a ratio of two rates, and ρ ≥ 1 is a straight line
Method
Four lines of arithmetic, of which the second is the one that goes wrong.
S = 2 ms, not S = 2. Half the errors below are unit errors, and they are invisible once the unit has been dropped.
μ = 1/S. In milliseconds that is μ = 1000/S per second: S = 2 ms gives μ = 500 per second, S = 40 ms gives μ = 25 per second.
ρ = λ/μ is a pure number. If the answer has seconds or milliseconds attached to it, the division was not the one intended.
Below one, the queue settles and every formula on this course applies. At or above one, report λ − μ per second and the backlog after the interval you care about, and stop.
The same offered load twice, with only the service time changed. The point is that the second column is not a small degradation of the first.
Nothing about the traffic changed between the two columns. A service time of 1 ms leaves a fifth of a server spare; a service time of 2 ms asks for eight fifths of one, and the shortfall of 300 a second does not average away — it accumulates.
For a faded rehearsal, keep the 1 ms service and raise the load to 1200 a second instead. The supplied first move is that μ is unchanged at 1000. Work out ρ, say whether it settles, and if it does not give the backlog after five minutes before opening the quiz.
You should be able to take a service time in milliseconds to μ, take λ and μ to ρ, say at a glance whether a system settles at all, and when it does not give the growth rate rather than a queue length.
Something to change about this lesson? Write it down, tick it off when it is handled, and export the list when you want to act on it. Kept in this browser only: nothing is sent anywhere, and nobody is notified.
No recommendations yet for this lesson.