1
A witness has two likelihoods
How probable a yes is if the claim is true, and how probable it is if the claim is false. A witness of reliability 9/10 says yes with probability 9/10 if it is true and 1/10 if it is false.
Lesson · Knowledge and Evidence · Credence
Testimony and Independent Witnesses
A witness who is right 9 times in 10 does not make the claim 9 in 10 likely; the prior still counts. Each independent witness contributes its own Bayes factor and the factors multiply. Hume’s argument about miracles compares two likelihoods: that the witness is mistaken, and that the event happened.
reliability: P(yes | true), not P(true | yes)
a witness: one Bayes factor, r / (1 − r)
independent witnesses: factors multiply
Hume: compare a false report with a miracle
A witness is a test
Core ideas
What this lesson establishes
A witness who says yes is evidence in the sense of the previous lesson, and nothing more. The lesson is to read the witness’s reliability as a likelihood.
2
Independence is relative to the truth
Two witnesses are independent when, once you know whether the claim is true, what one says tells you nothing about what the other says. Witnesses who share a source are not independent.
3
A miracle report compares two improbable things
One is that the event happened, and the other is that the witness is wrong. Which is more probable decides what the report is worth.
The material
One witness, then two
A claim at 1 in 100, a witness at 9 in 10, and the arithmetic on paper.
A claim has prior credence 1/100. A witness who is reliable in the sense that she says yes with probability 9/10 when the claim is true, and says no with probability 9/10 when it is false, says yes. So the likelihood of a yes is 9/10 if the claim is true and 1/10 if it is false.
claim prior P(yes | claim) product posterior
------------------------------------------------------
true 1/100 9/10 9/1000 1/12
false 99/100 1/10 99/1000 11/12
The posterior is 1/12, about 8%. The witness raised the claim from 1% to 8% and nowhere near 90%. Her Bayes factor is (9/10)/(1/10) = 9, the prior odds are 1 to 99, and the posterior odds are 9 to 99, which is 1 to 11.
This is the error of the previous lessons in a different setting. Her reliability, 9/10, is the probability of a yes given that the claim is true, and what is wanted is the probability that the claim is true given a yes. They differ by the same factor as before, and a claim that was improbable to start with stays improbable unless the witness is very good.
Definition · Independent witnesses
Two witnesses are independent when, given whether the claim is true, the probability of what one says does not depend on what the other says. Then the likelihood of both answers is the product of the two likelihoods, and so is the Bayes factor.
A second witness of the same reliability also says yes. The likelihood of two yeses is (9/10)·(9/10) = 81/100 if the claim is true and (1/10)·(1/10) = 1/100 if it is false: a factor of 81, which is 9 times 9.
witnesses Bayes factor posterior odds posterior
-------------------------------------------------------
none 1 1/99 1/100
one 9 1/11 1/12
two 81 9/11 9/20
Two independent witnesses take the claim to 9/20, a little under a half. The second is worth as much as the first in the sense that matters, a factor of 9, and worth more in credence, because the same factor applied at higher odds moves the credence further.
Independence is an assumption the lab does not check. Two newspapers that both reprint the same agency report are one witness heard twice, and multiplying their factors would count that witness’s possible error as if it had been confirmed.
Hume and the miracle
Hume’s argument about miracles can be put as a comparison of two likelihoods. Suppose the prior for an extraordinary event is 1/1000000, and a witness reports it who is wrong only 1 time in 1,000. Of the ways the report could come about, one is that the event happened and the witness is right, with weight (1/1000000)·(999/1000); the other is that it did not and the witness is wrong, with weight (999999/1000000)·(1/1000). The second is 1,001 times the first, and the posterior is 1/1002.
The conclusion is a comparison and not a ban. A report is worth believing when its Bayes factor outweighs the odds against the event. One witness at 999 in 1,000 has a factor of 999 against odds of about a million, and three independent such witnesses have a factor near a thousand million. The lab puts the claim after three at about 99.9%. What the argument requires is that the witnesses really are independent and that their reliability is as stated, and those are the premises worth disputing.
Interactive
Exact Bayesian updating
prior times likelihood, normalised; the Bayes factor is the weight of the data
Method
Weighing a witness
Five moves, in the order that keeps the prior in view.
1
Write the prior for the claim
Before the witness spoke. A claim that was unlikely stays unlikely unless the evidence is strong.
2
Write the witness’s two likelihoods
The chance of a yes if the claim is true and the chance of a yes if it is false. Reliability gives both when the witness is symmetric.
3
Form the Bayes factor
The first over the second. A witness at 9 in 10 has a factor of 9; at 999 in 1,000 a factor of 999.
4
Multiply for independent witnesses
Check independence first, by asking whether they could have got the claim from one source. Then multiply the factors.
5
Convert to a posterior
Multiply the prior odds by the total factor and turn the odds back into a credence.
Witnesses and miracles
Score 0 / 0Choose the answer you can justify.
The claim at 1 in 100, two witnesses at 9 in 10
Each witness says yes.
prior odds: 1 : 99
one witness: factor 9 odds 9 : 99 = 1 : 11
posterior 1/12
second witness: factor 9 odds 81 : 99 = 9 : 11
posterior 9/20
Each witness is right 9 times in 10, and two of them still leave the claim below even odds because it began at 1 in 100. A claim that began at 1 in 2 would stand at 81/82.
Common mistakes
1
Thinking a 90% reliable witness makes the claim 90% likelyHer reliability is P(yes | true), and what is wanted is P(true | yes). For a claim at 1/100 the posterior after her yes is 1/12: nine true yeses for every ninety-nine false alarms. The reliability only becomes the posterior when the prior is even odds.
2
Multiplying the factors of witnesses who share a sourceTwo witnesses who read the same notice are one witness. The lab multiplies the factors because it is told the witnesses are independent, and a lesson that finds 9/20 for two witnesses has assumed it. If the second witness only repeats the first, the posterior stays at 1/12.
3
Reading Hume as saying that no testimony could support a miracleThe comparison is between the odds against the event and the factor the testimony supplies. One witness at 999 in 1,000 falls short for an event at 1 in a million, and three independent ones do not. The argument sets a bar and invites the reader to ask whether the evidence clears it.
Check your understanding
✓
Finish when you can say what a witness is worth in a number.
You should be able to turn a stated reliability into two likelihoods and a Bayes factor, update a prior on one or several independent witnesses, and state Hume’s argument as a comparison of two weights, saying which premises it depends on.
Note: A symmetric reliability is the simplest case. Real witnesses are more likely to say yes when the claim is true than to say no when it is false, or the reverse; the lab takes any pair of likelihoods you type.