1
Compute what parallelism buys and where it stops
Amdahl’s speedup, its ceiling read off the asymptote, the n that reaches ninety per cent of it, and the Universal Scalability Law’s peak — after which more machines are worse.
Course · System Design
Scaling Laws and Cost
Adding a machine buys less than a machine’s worth, and sometimes less than nothing; paying for capacity buys more than you use, by a ratio you can compute. The course fits Amdahl’s ceiling and the Universal Scalability Law’s peak, prices batching, reserved capacity, autoscaling lag and storage tiers, and closes with two transfer break-evens.
S(n) = 1/((1−p) + p/n) → 1/(1−p) 5% serial caps at 20×
C(N) = N/(1 + α(N−1) + βN(N−1)) peaks, then falls
waste = 1 − mean·ρ_target/peak
u* = reserved price / on-demand price
shortfall = r·τ·(D − τ/2) unless a reserve of r·τ is held
What this course is about
- Subject
- System Design
- Lessons
- 11
- Format
- Interactive
Overview
What you will be able to do
By the end you can say what the next machine buys, locate the point past which it buys nothing, and compute the five break-evens that decide how capacity is bought rather than how much of it there is.
2
Price a batch and a machine shape
Per-item cost against added latency for a batch size, and horizontal against vertical capacity with the break-even found by evaluation and the single box’s missing parallel path named.
3
Compute what capacity costs when you pay for the peak
Waste from a peak-to-mean profile, the reserved against on-demand break-even utilisation and the cost-minimising split, and the shortfall an autoscaling lag leaves.
4
Compute the three transfer break-evens
The accesses per month at which a cold tier stops paying, the bandwidth at which compressing stops paying, and the result-to-input ratio that decides whether to move the data or the compute.
Lessons
11 lessons
What another machine buys first, then what a batch and a machine shape cost, then the three ways capacity is paid for, and last the three break-evens about moving bytes.
Amdahl's LawCompute the speedup n machines give, the ceiling it never passes, and the n that reaches a stated fraction of that ceiling.Not marked
The Universal Scalability LawLocate the machine count at which throughput peaks, and show that adding machines past it makes throughput fall.Not marked
Batching: Cost and LatencyChoose a batch size from where the per-item cost curve flattens, and report the wait that size costs.Not marked
Vertical vs HorizontalCost a target capacity as many small nodes and as one large machine, and locate every capacity at which the cheaper side changes.Not marked
Utilisation and WasteCompute the fraction of purchased capacity a load profile leaves unused, and show what cutting to the mean would do.Not marked
Reserved vs On-demandCompute the break-even utilisation of a reserved instance and the reserved level that minimises the bill for a profile.Not marked
Autoscaling LagCompute the requests a boot window costs during a ramp, and the warm reserve that removes the shortfall entirely.Not marked
Cost per RequestDecompose a monthly bill into cost per request and name the term that dominates it.Not marked
Storage Tiers and the Access Break-evenCompute the access rate at which a cold tier stops paying, and the age at which a decaying access profile reaches it.Not marked
Compress or NotTime a transfer with and without compression, and compute the link speed at which compressing stops paying.Not marked
Move the Data or the ComputeTime both placements of a computation and compute the result-to-input ratio at which they tie.Not marked
Recommended background
Capacity estimation, queues and utilisation, partitioning and load balancing and storage engines and indexes, plus recursion trees and amortised analysis from discrete mathematics, and graphs and asymptotes, maximum and minimum problems and compound interest and continuous growth from algebra.
Practice suggestions
Read Amdahl’s ceiling off the asymptote rather than from a table. The interesting claim is that a ceiling exists at all, and the asymptote is where you see it.
Scan the Universal Scalability Law rather than differentiating it. The peak is found by evaluating at integer N, and the closed form is checked against the scan, not used in place of it.
Do every break-even as an equality before you interpret it. Each one on this course is a single equation set to zero, and the interpretation is easier once the algebra is already done.
What it does not cover
Vendor pricing, discount structures and FinOps process. Prices are inputs.
Fitting α and β for the Universal Scalability Law. That is regression; here they are given and the peak is computed.
Capacity planning under uncertainty beyond the ranges of “Capacity Estimation”.