A simulation's output is a random variable. Averaging n draws estimates the expectation, and the estimator's own variance is σ²/n, so its standard deviation — the standard error — falls like 1/√n. Four times the runs buy half the width and no more, and the estimate without that number beside it is a number with no claim attached.
Core ideas
Three ideas, and the middle one is the arithmetic. The first and the third are about what the arithmetic does and does not promise, which is where this course differs from every other one on the path.
Run it again with a different seed and it changes. So the thing to report is not the number but the number together with a statement about the spread of the process that produced it — and that statement is computable from the same run, which is the whole of Monte Carlo output analysis.
Var(X̄ₙ) = σ²/n exactly, for independent draws. Discrete Probability proves it in “Variance and Standard Deviation”: the variance of a sum of independent variables adds, and dividing by n scales a variance by 1/n². The standard error is the square root of that, and it is the one quantity on this page that is not a fraction.
Halving the width of an interval takes four times the runs; getting one more decimal digit takes a hundred times. That is not a defect of the method to be engineered around — it is the rate, it is proved, and the only way to beat it is to change the estimator rather than to buy draws, which is what the second half of this course does.
The material
What the sample mean estimates, how far it can be from the thing it estimates, and how fast that distance shrinks.
Draw X₁, …, Xₙ independently from the distribution of interest and report X̄ₙ = (X₁ + ⋯ + Xₙ)/n. It is unbiased: E[X̄ₙ] = μ for every n, so it is aimed at the right number from the first draw.
The standard error of an estimator is the standard deviation of the estimator itself, not of the data. For the sample mean it is √(σ²/n) = σ/√n. Where σ² is unknown it is estimated by the sample variance s² = Σ(Xᵢ − X̄)²/(n − 1), and the estimate of the standard error is √(s²/n).
The n − 1 is not a fudge. The sample mean has already been fitted to the same data, so one degree of freedom has been spent, and dividing by n would make s² systematically too small. On four hundred draws the difference is a fraction of a percent; on four draws it is a quarter, which is why the convention is worth keeping even where it looks like pedantry.
For independent draws with variance σ², Var(X̄ₙ) = σ²/n. Hence the standard error is σ/√n, and multiplying n by four divides it by exactly two.
That is proved in Discrete Probability and it is not reproved here; what this page adds is the measurement. Squaring removes both roots, so the ratio of the standard error at n to the standard error at 4n is √((σ²/n)·(4n/σ²)) = 2 whatever σ² is. The panel prints that ratio as an exact 2 at every step, which is the difference between checking a claim and asserting it.
The sample mean is a fraction. The sample variance is a fraction. The standard error is the square root of a fraction, and a square root of a rational is either rational or irrational with nothing in between. So it is carried as an exact surd and printed twice: the surd, then a decimal with the word rounded attached. Everything else that prints as a decimal on these pages says “exact, shown to N places” instead, and the distinction is load-bearing — a reader who has seen “rounded” used loosely cannot tell which figures really are.
On the table 1, 4, 6, 4, 1 over 16, from the standard stream at seed 1: at n = 100 the sample mean is 21/10 and the sample variance is 113/99, giving a standard error of (1/330)√1243 ≈ 0.10684; at n = 400, 413/200 and 43031/39900, giving (1/79800)√17169369 ≈ 0.05192; at n = 1600, 3231/1600 and 2528639/2558400, giving (1/2558400)√4043293761 ≈ 0.02485. The three decimals are rounded and the six fractions are not.
Beside those sit the theoretical standard errors, which on this table are 1/10, 1/20 and 1/40 exactly, because σ² = 1 and 100, 400 and 1600 are squares. The estimates track them without matching them, which is the correct behaviour: s² is itself an estimate, and an estimate of a standard error has a standard error.
More draws do not converge to the answer. They shrink the spread of a random quantity, which is a different promise: a single long run can and does land further from the truth than a short one, and at n = 400 this stream is 13/200 out while the same length at seed 4 is 3/400 out and at seed 0 is 3/40 out. The estimator is unbiased at every n; it is the width that improves.
And a running mean that has stopped moving has not converged. A settled-looking trace is exactly what a small sample of a slow process produces, and the flat stretch is evidence of nothing at all. The interval is the evidence; the picture is a picture.
The same 1/√n turns up wherever a quantity is measured rather than computed. A thousand requests cannot separate a success rate of 99.9% from one of 99.8%, for precisely the reason four times the runs buy twice the precision here — and the arithmetic that settles the simulation question settles the measurement one unchanged.
Interactive
The sample mean has variance σ²/n, so its error falls like one over root n and not faster
Method
Four steps. The last one is the one that turns a number into a result.
X̄ₙ = (ΣXᵢ)/n. On a seeded stream this is a ratio of whole numbers and there is no reason to lose that; 413/200 is checkable and 2.065 is a claim about a checkable thing.
s² = Σ(Xᵢ − X̄)²/(n − 1). Also exact. This is the estimate of the spread of a single draw, and it is not yet anything about the estimate.
SE = √(s²/n). This is the only irrational quantity produced so far, and the only honest way to print it is the exact form followed by a decimal with the rounding named.
The width you need fixes n through n = σ²/SE². Halving the target width multiplies the budget by four; an extra decimal digit multiplies it by a hundred. Discovering that after the runs have been bought is the expensive order to do it in.
2.065 on its own is not a result. 2.065 with a standard error of about 0.052 from four hundred draws of a stated stream is one, because a reader can tell whether it answers their question.
The table has μ = 2 and σ² = 1 exactly, so the theoretical standard errors are fractions and the halving can be checked rather than believed. Everything below is the panel's opening state at seed 1.
The ratio column is the point of running the same stream at three lengths. Neither 1/10 nor 1/20 needs a root on this table, but the ratio is exactly 2 for any table, because (σ²/n) divided by (σ²/4n) is 4 and the roots cancel. A claim that survives not knowing σ² is worth more than one measured on a convenient example.
The three sample standard errors are 0.10684, 0.05192 and 0.02485. Their ratios are about 2.058 and 2.089, not 2. That is not a failure of the theorem: s² is an estimate and it moves, so the ratio of two estimated standard errors is itself an estimate. The exact 2 lives in the theoretical column, and the difference between the two columns is the difference between a fact and a measurement of it.
For a faded rehearsal, switch the panel's table to the skewed one — 1/2 on the first outcome and the rest falling away. The supplied first move is that its exact variance is 367/256, which is not a square over 400, so the theoretical standard error becomes (1/320)√367 ≈ 0.05987 and is itself rounded for printing. Say what happens to the ratio column, and why, before the panel shows you.
You should be able to compute a sample mean and a sample variance as exact fractions, say why the denominator is n − 1, form the standard error and print it in exact form with its rounding named, convert a target width into a number of draws, and state in one sentence what more draws do and do not promise.
Something to change about this lesson? Write it down, tick it off when it is handled, and export the list when you want to act on it. Kept in this browser only: nothing is sent anywhere, and nobody is notified.
No recommendations yet for this lesson.