1
Confirmation is raising probability
E confirms H when P(H | E) > P(H), disconfirms it when the inequality reverses, and is neutral when the two are equal. It is a comparison of two numbers, not a property of E alone.
Lesson · Science, Induction and Causation · Confirmation
Confirmation and the Weight of Evidence
Evidence confirms a hypothesis when the posterior exceeds the prior, and its weight is the Bayes factor, the ratio of the probability of the evidence under the hypothesis to its probability under the alternative. Evidence that is merely consistent with a hypothesis need not confirm it, and surprising evidence confirms more than expected evidence.
E confirms H ⟺ P(H | E) > P(H)
weight: BF = P(E | H) / P(E | ¬H)
BF > 1 confirms; BF = 1 neutral; BF < 1 not
surprising E weighs more than expected E
Confirmation and its weight
Core ideas
What this lesson establishes
Confirmation is a relation between evidence and a hypothesis. This lesson defines it by probability and measures it by a ratio.
2
Weight is the Bayes factor
P(E | H) / P(E | ¬H) says how much more probable the evidence is if the hypothesis is true than if it is false. It is the same whatever the prior.
3
Surprise carries weight
Evidence that was improbable on the alternative and probable on the hypothesis has a large factor. Evidence expected either way has one near 1.
The material
Probability up, and by how much
A definition, a measure, and three cases that differ only in one likelihood.
“Updating on Evidence” gave the procedure for revising a probability on new evidence. This lesson asks what to call the result. A reasonable rule is that the evidence supports the hypothesis if, and only if, the hypothesis is more probable afterwards than before.
Definition · Confirmation
E confirms H when P(H | E) > P(H). It disconfirms H when P(H | E) < P(H) and is neutral when they are equal.
That is a qualitative verdict. To compare how strongly two pieces of evidence confirm, the usual measure is the factor by which they multiply the odds in favour of the hypothesis.
Theorem · Odds form of Bayes' rule
posterior odds = BF · prior odds, where BF = P(E | H) / P(E | ¬H). So E confirms H exactly when BF > 1, and the factor does not depend on the prior.
Three cases share a prior of 1/2 on H and the same likelihood for the evidence under H, namely P(E | H) = 1. They differ only in how probable E is if H is false.
case P(E | H) P(E | not H) Bayes factor P(H | E)
-------------------------------------------------------------
surprising 1 1/10 10 10/11
expected 1 9/10 10/9 10/19
neutral 1 1 1 1/2
In every row E is perfectly consistent with H, because H makes it certain. Yet the verdicts are different. In the first row the evidence would rarely occur unless H were true, so seeing it is strong support. In the second it would very probably have occurred anyway, so it supports H only slightly. In the third it would have occurred whatever, and it supports nothing.
The third row is the one that corrects an easy mistake. That a hypothesis predicts the evidence is not the same as the evidence confirming the hypothesis. “The sun rose this morning” is predicted by every theory of the heavens anyone takes seriously, and so confirms none of them against the others.
The limit of the lab is in the second likelihood. P(E | ¬H) is an average over every way H could be false, weighted by their priors, and it is a number you chose. Change it and the same evidence changes weight, which is why two scientists can agree on the observation and disagree on what it shows.
Interactive
Exact Bayesian updating
prior times likelihood, normalised; the Bayes factor is the weight of the data
Method
Weighing a piece of evidence
Four steps, in this order.
1
State the hypothesis and its negation
Name H and everything that is not H. The alternative is part of the question; evidence has weight only against it.
2
Find both likelihoods
Write P(E | H) and P(E | ¬H). The second is a mixture over the ways H could fail, weighted by their priors.
3
Take the ratio
Divide the first by the second. A result above 1 confirms, 1 is neutral and below 1 disconfirms.
4
Apply it to the prior odds
Multiply the prior odds by the factor to get the posterior odds, and convert back to a probability if you need one.
Does it confirm, and how much
Score 0 / 0Choose the answer you can justify.
The same prior, two kinds of evidence
Prior 1/2 on H, and P(E | H) = 1. The only difference is P(E | ¬H).
surprising: BF = 1 / (1/10) = 10
prior odds 1 to 1, posterior 10 to 1, P(H | E) = 10/11
expected: BF = 1 / (9/10) = 10/9
prior odds 1 to 1, posterior 10 to 9, P(H | E) = 10/19
neutral: BF = 1 / 1 = 1
posterior = prior = 1/2
The evidence is the same sentence in all three, and H predicts it with certainty in all three. A factor of 10 and a factor of 10/9 differ in weight by the ratio 9 to 1, and that difference comes from the alternative alone.
Common mistakes
1
Thinking evidence consistent with a hypothesis confirms itIn the neutral case H predicts E with certainty and E is observed, and the posterior stays exactly at the prior 1/2, because the alternative predicted E with certainty too. Consistency says only that the likelihood is not 0; confirmation compares it with the alternative's.
2
Measuring confirmation without an alternativeP(E | H) alone settles nothing. The same value of 1 gives a factor of 10, 10/9 or 1 as P(E | ¬H) moves from 1/10 to 9/10 to 1.
3
Treating the Bayes factor as the probability of the hypothesisA factor of 10 means the odds are multiplied by 10. From a prior of 1/2 that gives 10/11; from a prior of 1/100 it gives 10/109, nowhere near certainty.
Check your understanding
✓
Finish when you can classify a piece of evidence and give its weight.
Given a prior and two likelihoods you should be able to compute the Bayes factor, say whether the evidence confirms, disconfirms or is neutral, and compute the posterior from the prior odds.
Note: Probability-raising is one of several measures of confirmation, and the Bayes factor is one of several ways to weigh it. They agree on the verdict and can disagree on the comparison between two cases with different priors; the cases here share a prior.