1
It is a property of the probabilities
Not of causation, not of physical separation. Two events in the same experiment can be independent and two in different rooms need not be.
Lesson · Discrete Probability · Conditioning
Independence
Independence is an equation, not a story about causation. Two events are independent when conditioning on one does not change the other — and checking that means checking the equation.
A, B independent ⟺ P(A ∩ B) = P(A)·P(B)
equivalently P(A | B) = P(A) when P(B) > 0
DISJOINT is nearly the opposite of independent
pairwise independent ⇏ mutually independent
One equation
Core ideas
What this lesson establishes
Independence is defined by an equation so that it can be checked. Intuition about it is unreliable.
2
Disjoint is almost the opposite
If A and B are disjoint with positive probability, then P(A|B) = 0 ≠ P(A): knowing B occurred tells you A did not.
3
Pairwise is weaker than mutual
Three events can be independent in every pair and still fail the three-way condition, which is why mutual independence requires every subset.
The material
Independence
The definition, the equivalent form, and the two standard confusions.
Definition · Independent events
A and B are independent when P(A ∩ B) = P(A)P(B). Equivalently, when P(B) > 0, P(A|B) = P(A): conditioning on B changes nothing.
The second form is the meaning and the first is the test. The first is preferred as the definition because it is symmetric and does not require P(B) > 0.
Example · Independent within one experiment
Roll two dice. A = "first die is even", B = "second die is 5". P(A) = 1/2, P(B) = 1/6, P(A ∩ B) = 3/36 = 1/12 = (1/2)(1/6). Independent, and unsurprisingly so.
Example · Independent, surprisingly
Same two dice. A = "first die is even", B = "the sum is 7". P(A) = 1/2, P(B) = 1/6, and A ∩ B is {(2,5),(4,3),(6,1)} with probability 3/36 = 1/12. Independent — even though the sum obviously depends on the first die.
That second example is why the definition is an equation. Nothing in the physical description suggests independence; the numbers satisfy the equation, and that is what independence means.
Theorem · Complements preserve independence
If A and B are independent, so are A and B̄, Ā and B, and Ā and B̄.
P(A ∩ B̄) = P(A) − P(A ∩ B) = P(A) − P(A)P(B) = P(A)(1 − P(B)) = P(A)P(B̄). The other cases follow by symmetry and by applying this twice.
□
Disjoint is not independent
If A ∩ B = ∅ and both have positive probability, then P(A ∩ B) = 0 while P(A)P(B) > 0, so they are dependent. Intuitively: knowing B occurred tells you A did not, which is a large change in your information. The only way to be both disjoint and independent is for one of them to have probability 0.
Nor is the union of independent events the sum: P(A ∪ B) = P(A) + P(B) − P(A)P(B), by “The Probability Axioms”’s inclusion and exclusion with the product put in for the intersection. "Independent, so add" and "disjoint, so multiply" are the two halves of one confusion, and both come from treating the two properties as one.
Mutual independence
Definition · Mutually independent
Events A₁, …, A_n are mutually independent when for every subset of them the probability of the intersection is the product of the probabilities. Pairwise independence is only the two-element case.
Example · Pairwise but not mutually
Flip two fair coins. A = "first is heads", B = "second is heads", C = "the two agree". Each has probability 1/2, and each pair is independent: P(A ∩ C) = P(HH) = 1/4 = (1/2)(1/2). But P(A ∩ B ∩ C) = P(HH) = 1/4, while P(A)P(B)P(C) = 1/8. Any two of these events determine the third, so they cannot be mutually independent.
The example is worth keeping. Checking pairs and declaring mutual independence is a real and common error, and the failure here is total: any two of the three events determine the third completely.
Interactive
Sample space, event, condition
Exact fractions, every outcome listed
Method
Checking independence
Compute three numbers and compare two of them.
1
Compute P(A), P(B) and P(A ∩ B)
All three, exactly. Estimating any of them defeats the check.
2
Compare P(A ∩ B) with P(A)P(B)
Equal means independent. This is the definition and there is no other test.
3
Do not reason from the story
Physical separation does not imply independence and shared causes do not preclude it. The example with the sum of two dice settles that.
4
For several events, check every subset
Pairwise independence does not give mutual independence. All 2ⁿ − n − 1 conditions are required.
Independence
Score 0 / 0Choose the answer you can justify.
Three events, checked properly
Two fair coins. A = first heads, B = second heads, C = the two agree.
S = {HH, HT, TH, TT}, each 1/4.
A = {HH, HT} P(A) = 1/2
B = {HH, TH} P(B) = 1/2
C = {HH, TT} P(C) = 1/2
PAIRWISE:
P(A∩B) = P(HH) = 1/4 = (1/2)(1/2) independent
P(A∩C) = P(HH) = 1/4 = (1/2)(1/2) independent
P(B∩C) = P(HH) = 1/4 = (1/2)(1/2) independent
TRIPLE:
P(A∩B∩C) = P(HH) = 1/4
P(A)P(B)P(C) = 1/8 NOT equal
Pairwise independent, NOT mutually independent.
Any two of these events determine the third: if the first is heads and they agree, the second is heads. Mutual independence would mean no combination of the events carries information about another, and here two of them carry all of it.
Common mistakes
1
Confusing independent with disjointThey are close to opposite. Disjoint events with positive probability are strongly dependent.
2
Inferring independence from the descriptionPhysical intuition is unreliable. Compute the three probabilities and compare.
3
Checking pairs and stoppingMutual independence needs every subset. Pairwise is a strictly weaker condition, and the gap is not a technicality.
Check your understanding
✓
Finish when you check the equation rather than the story.
Take three events in the two-dice experiment and determine which pairs are independent, computing all three probabilities each time. At least one answer will not be the one you expected.
Note: Independence of random variables (“Random Variables”) is the same idea applied to every pair of values at once: X and Y are independent when P(X = a, Y = b) = P(X = a)P(Y = b) for all a and b. “Linearity of Expectation” shows that expectation adds without needing it.