1
A reference class is the group whose rate is used
To say that a person’s chance of an outcome is 1 in 5 is to say that 1 in 5 in some group the person belongs to has the outcome.
Lesson · Knowledge and Evidence · Evidence and belief
Reference Classes and Statistical Evidence
A statistic is a rate in a group, and it reaches an individual only through the group chosen. A person belongs to many groups with different rates, and a pooled rate is the rate of one more group. The lab prints the rate in each class and in their union from the same counts.
a rate belongs to a class, not a person
one person sits in many classes
pooled rate: add the counts, then divide
the lab divides; the class is chosen
Whose rate is it
Core ideas
What this lesson establishes
Evidence about a population is used for an individual by reading the individual into a class. The step is easy to take without noticing.
2
An individual belongs to many classes
Smokers, cyclists, people of sixty, smokers who cycle: each has its own rate, and the data do not choose among them.
3
A pooled rate is a rate for a class too
Adding the counts of two groups gives the rate of their union. It depends on how many are in each group, and it is the answer to a question about the union.
The material
A smoker who cycles
One person, several rates, and counts that make each of them exact.
A man of sixty smokes and cycles daily. Asked how likely he is to die of heart disease before seventy-five, a doctor can consult tables. The counts below are invented, so that every rate is a clean fraction.
class died total rate
---------------------------------------------
smokers 200 1000 1/5
who cycle 5 100 1/20
who do not 195 900 13/60
non-smokers 30 1000 3/100
who cycle 6 400 3/200
who do not 24 600 1/25
He is a smoker, so 1/5 applies. He is a smoker who cycles, so 1/20 applies. He is a person who cycles, a person of sixty, a person who lives in his city, and each of those has a rate of its own. All of these are statements about groups he belongs to, and the two rates above differ by a factor of four.
Definition · Reference class
A reference class is the group whose rate of an outcome is taken as the probability of that outcome for one case. The reference class problem is that a case belongs to many such groups, with different rates, and the counts do not say which to use.
A practical rule is to use the narrowest class that has enough data. The lab shows the cost on both sides. The class of smokers who cycle gives 1/20, but it rests on 100 people and 5 deaths, and the lab, which treats counts as exact rates and estimates nothing, cannot tell how far that figure can be trusted. The class of smokers gives 1/5 on ten times as many people and ignores the cycling.
The pooled rate
The rate for smokers, 1/5, is the pooled rate of the two groups beneath it: add the deaths, 5 + 195 = 200, add the totals, 100 + 900 = 1000, and divide. It is not the average of 1/20 and 13/60, which would ignore that nine in ten smokers here do not cycle. The pooled rate sits close to the larger group’s rate because the larger group supplies most of the counts.
Nothing makes the pooled rate “the” probability. It is the probability for a smoker chosen at random from this population, which is the right answer to a question about that, and the wrong one to a question about a man who cycles.
When the pooled rate points the wrong way
The effect can be larger than a difference in size. In a well-known set of counts, two treatments for kidney stones are compared by stone size. The open operation has a higher success rate than the keyhole procedure for small stones, 81/87 against 234/270, and for large stones, 192/263 against 55/80. Pooled, it is 273/350 against 289/350, and the keyhole procedure is ahead.
A patient with a large stone asks which treatment to choose. The pooled figures answer a question about patients whose stones might be of either size, and he is not one of them. The reason the pooled comparison reverses, which is that the two treatments were given to different mixes of patients, belongs to Science, Induction and Causation, where it is called Simpson’s paradox; here it only shows that the choice of class can change which treatment looks better.
A rate in a class is a conditional credence given membership, in the sense of the earlier lesson. Learning that he also has a family history puts him in a narrower class and changes the number again. The probability for an individual is relative to what is known about him, and the lab only divides.
Interactive
Rates within groups and pooled
the same counts, compared group by group and all together
Method
From counts to a rate for one person
The order that makes the choice of class visible.
1
List the classes the person belongs to
Everything you know about him that you have a count for. Each is a candidate.
2
Write each rate as a fraction
Cases with the outcome over cases in the class. Keep the counts beside the fraction so you can see how much data each rests on.
3
Pool by adding counts
To combine two classes, add numerators and denominators. Averaging the rates gives the wrong answer unless the classes are the same size.
4
Compare the classes
If the rates differ, the data have not chosen between them. Say what reason you have for the one you use.
5
State the class with the rate
“1 in 20 of smokers who cycle”, not “his chance is 1 in 20”. The sentence should carry its class.
Which class, and what rate
Score 0 / 0Choose the answer you can justify.
A smoker who cycles
From the counts above, the rates that apply to him.
smokers who cycle: 5 / 100 = 1/20
smokers who do not: 195 / 900 = 13/60
all smokers (pooled): 200 / 1000 = 1/5
non-smokers (pooled): 30 / 1000 = 3/100
his classes give: 1/20 and 1/5
Two of these rates are about classes he is in, and they differ by a factor of four. The pooled 1/5 is not wrong; it is the rate for smokers whether or not they cycle, and he is a smoker who does.
Common mistakes
1
Treating the pooled rate as the probabilityThe pooled rate for smokers is 1/5 and it is a true statement about the 1,000 of them. It is the rate for a smoker whose cycling is unknown. For this man, whose cycling is known, the class that counts it gives 1/20, and the pooled figure would be the natural one only for someone about whom nothing else is known.
2
Averaging two rates instead of adding countsThe two groups of smokers have 100 and 900 people. The average of their rates, 2/15, weights them equally and describes no class. Adding the counts gives 1/5, and changing a group’s size changes it.
3
Thinking the data can choose the classThe counts print a rate for each class and are silent on which class is the right one. Choosing is a judgement about relevance, about how much data each class has, and about what else is known about the case. A lab or a table of statistics can compute any of them and cannot say which to use.
Check your understanding
✓
Finish when you can say which class each rate is a rate of.
You should be able to compute the rate of an outcome in two classes and in their union from counts, say why the pooled rate is not the average of the rates, and state for a given individual which classes apply and why the figure for one differs from the figure for another.
Note: All counts here are exact and no estimate is attempted. Deciding how far a rate from a small class can be trusted is a question for statistics, which this course does not cover.