1
The continuation probability
After every round the game goes on with probability δ, so a payoff n rounds away is worth δ to the power n times as much now. A larger δ means the future weighs more.
Lesson · Games and the Social Contract · Repeated games
The Shadow of the Future
Suppose each round is followed by another with probability delta. Against a partner who punishes defection, cooperating forever beats defecting once exactly when delta is above a threshold fixed by the four payoffs. The threshold needs an uncertain end rather than a long game: a last round that is known unravels the whole thing.
δ: the chance there is a next round
GRIM holds if δ ≥ (T − R) / (T − P)
TFT holds if δ ≥ (T − R) / (R − S) as well
a known last round unravels, backwards
How much the future must count
Core ideas
What this lesson establishes
The previous lesson ran matches of fixed length. The question here is when a player with a reason to defect has a reason, larger still, to hold back.
2
A punishment makes defection costly later
A player who defects gains the temptation now, and loses whatever the partner's reply costs in the rounds that follow. Cooperation is stable when the loss outweighs the gain.
3
A known last round has no future
In the final round there is nothing to protect, so defecting dominates; with that round settled, the one before has no future either. The argument runs backwards to the first round.
The material
When the future outweighs the temptation
Two comparisons, one threshold from each, and the argument for why the end must not be known.
Take the standard payoffs, R = 3, S = 0, T = 5 and P = 1, and let the game continue after each round with probability δ. A payoff n rounds away is then worth δ^n times as much now, so getting x every round is worth x·(1 + δ + δ^2 + …). The sum in the brackets is 1 / (1 − δ) whenever δ is below 1: multiply it by δ and subtract, and every term but the first cancels. So the value of x every round is x / (1 − δ), and the expected number of rounds, which is the same sum with x = 1, is 1 / (1 − δ). “The St Petersburg Game” summed one tail of this series; this is the whole of it.
Against a player who never forgives
GRIM cooperates until it is defected against once and defects forever after. A player who meets GRIM can cooperate throughout, or defect and be punished. Cooperating throughout earns R / (1 − δ). Defecting at once earns T this round and then, with GRIM defecting for good, P in every later round, worth δ·P / (1 − δ) altogether. Cooperation is at least as good when
R / (1 − δ) ≥ T + δ·P / (1 − δ)
R ≥ T·(1 − δ) + δ·P
δ·(T − P) ≥ T − R
δ ≥ (T − R) / (T − P)
With the standard payoffs that is 2 / 4, or 1/2.
Against a player who forgives
TFT punishes only for as long as the other defects. The best deviation against it is one of two. Defect forever, which is as bad as it is against GRIM and needs the same δ. Or defect once and go back to cooperating, which earns T now, S next round while TFT retaliates and R afterwards. Comparing the two streams round by round, the first two rounds of cooperating pay R + δ·R and the two rounds of the deviation pay T + δ·S, with everything later equal. Cooperation is at least as good when
R + δ·R ≥ T + δ·S
δ·(R − S) ≥ T − R
δ ≥ (T − R) / (R − S)
With the standard payoffs this is 2 / 3. The lab reports the larger of the two thresholds for TFT, since both deviations must be unprofitable, and the first alone for GRIM. With these payoffs the milder punishment needs the more patient player, because a deviator who returns to cooperation loses only one round to the retaliation.
The end that must not be known
Suppose the game lasts exactly one hundred rounds and both players know it. In round one hundred there is no later round to punish in, so defecting dominates, as in the one-shot dilemma. Both defect then whatever happened before. So in round ninety-nine nothing the players do changes round one hundred, the same argument applies, and so on back to the first round. A hundred rounds is not a long game in this sense; it is a game with a last round. A game that ends after each round with probability one half lasts two rounds on average, and with these payoffs it just sustains cooperation against GRIM.
Interactive
The repeated prisoner's dilemma
one match, a tournament of seven strategies, and the discount that sustains cooperation
Method
Finding the discount factor
The same five lines for any partner who punishes.
1
Write the value of cooperating forever
It is the reward every round: R / (1 − δ).
2
Write the value of the best deviation
The temptation now, then whatever the partner's reply yields in the rounds after.
3
Set cooperation at least as large and cancel
Multiply through by 1 − δ to clear the denominator and collect the terms in δ.
4
Solve for the smallest delta
The result is a ratio of two gaps between payoffs: the gain from defecting over the gain of cooperation, against how much the reply takes away.
5
Compare with the delta you have
If it is above the threshold the deviation does not pay. If it is below, it does.
Thresholds and the last round
Score 0 / 0Choose the answer you can justify.
The thresholds with the standard payoffs
R = 3, S = 0, T = 5, P = 1, and delta = 9/10.
GRIM: (T - R)/(T - P) = 2/4 = 1/2
TFT: (T - R)/(R - S) = 2/3, and 2/3 > 1/2
Cooperate for good against GRIM: 3/(1/10) = 30
Defect at once: 5 + (9/10)·1/(1/10) = 5 + 9 = 14
9/10 clears 1/2 and 2/3: cooperation is stable
Thirty against fourteen is not close. At δ = 1/4 the comparison reverses: cooperating for good is worth 4 and defecting at once is worth about 5.33, and the lab says the future does not count enough.
Common mistakes
1
Thinking cooperation needs a long game when it needs an uncertain endA game of exactly one hundred rounds unravels from the last round back, while a game that continues with probability one half, two rounds on average, sustains cooperation against GRIM with the standard payoffs. What counts is whether a next round is possible at every point, not how many there will be.
2
Assuming a mild punishment sustains cooperation as easily as a harsh oneGRIM needs a continuation probability of 1/2 and TFT needs 2/3. A deviator against TFT who goes back to cooperating loses only one round to the reply, so the shadow of the future has to be heavier.
3
Reading the threshold as a prediction that players will cooperateIt says cooperation is stable, that nobody gains by deviating. Both players always defecting is also stable in the repeated game, so the threshold says what is possible, not what will happen.
Check your understanding
✓
Finish when you can derive a threshold and use it.
Given the four payoffs, compute the continuation probability above which cooperation is stable against GRIM and against TFT, say whether a stated delta clears each, and explain why a known final round removes the threshold.
Note: The lab refuses payoffs that are not a dilemma, and refuses 2R below T + S as well, since then taking turns to exploit beats cooperating and the derivation above is not the right comparison. That is why the high-temptation preset is the standard game scaled by two, with the temptation raised to eleven.