Lesson · Games and the Social Contract · Repeated games

Repeated Games and Reciprocity

When the prisoner's dilemma is played for many rounds, a strategy is a rule that turns the history so far into the next move. The lab plays two named rules against each other round by round and then runs a round robin of seven. The winner is not the strategy that beats its partners but the one that does well with all of them.

Open the interactive lesson Practice

a strategy is a rule: history in, move out TFT: cooperate first, then copy the last move a match goes on points, a tournament on sums never beating a partner can still win

Rules, matches, totals

Core ideas

What this lesson establishes

Three words need fixing before the lab means anything: strategy, match, and total.

A strategy is a rule over histories

In a repeated game a strategy does not name a move; it says what to play given everything that has happened. The lab's seven are all simple: always cooperate, always defect, copy the other's last move, and four more.

A match is a fixed number of rounds

Two strategies meet for a stated number of rounds, and each earns the sum of its round payoffs. The record of moves is the evidence for any claim about what a strategy did.

A tournament sums over partners

In the round robin every strategy meets every strategy, itself included, and its total is the sum of its scores. One strategy outscoring another in a match is not what the total counts.

The material

Playing the dilemma again

Seven rules, one match read round by round, and a tournament.

The previous two lessons gave the dilemma one round. Here the same four payoffs, R = 3, S = 0, T = 5 and P = 1, are played over a stated number of rounds by two strategies, and each player's score is the sum of what the rounds paid.

Definition · Strategy in a repeated game

A strategy is a rule that gives a move, C or D, for every possible history of play. It may depend on what the other player has done, on what the player has done, and on how many rounds have passed.

ALLC always cooperates and ALLD always defects.
TFT (tit for tat) cooperates in the first round and afterwards copies the other player's previous move.
GRIM cooperates until the other player defects once, and defects ever after.
PAVLOV cooperates first, then repeats its move if the last round paid R or T, and switches otherwise; equivalently it cooperates exactly when both players made the same move last round.
TF2T (tit for two tats) defects only after two defections in a row.
STFT (suspicious tit for tat) is TFT with a defection in the first round.

Take TFT against ALLD for ten rounds. In the first round TFT cooperates and ALLD defects, so TFT gets 0 and ALLD gets 5. From then on TFT copies the defection and both defect for nine rounds, one point each round. The lab prints 9 for TFT and 14 for ALLD.

TFT lost that match, and it can never do better than tie any match: it defects only after being defected against, so it cannot score more than its partner. ALLD is the reverse. It scores at least as much as its partner in every match, since it can always choose the strictly better move in a round. A natural inference is that ALLD is the strategy to beat.

The round robin refutes it. ALLD takes 50 from ALLC and 30 from PAVLOV, but only 14 from TFT and from GRIM, 18 from TF2T, and 10 from itself and from STFT. Meanwhile TFT, GRIM, PAVLOV, ALLC and TF2T cooperate in all ten rounds with one another and take 30 each time. Over the whole field TF2T scores 185, TFT 184 and ALLD 146.

Two matches that never defect

TFT against TFT cooperates throughout and each scores 30; so do GRIM and PAVLOV against each other, because neither is ever defected against and so neither ever has cause to respond. Reciprocity is a rule that is silent when met by cooperation, which is why it does not cost anything against a cooperator.

Two limits belong here. The field is seven rules chosen for the lab, and a different field would rank them differently. And the winner at ten rounds is only one point ahead of the next, so the order is not stable: change the number of rounds in the lab and watch it move.

Interactive

The repeated prisoner's dilemma

one match, a tournament of seven strategies, and the discount that sustains cooperation

Pick two strategies and a number of rounds

The table shows the first twelve moves of the match and every strategy's round-robin total at the same number of rounds. Change the rounds and watch the order of the totals.

Worked example

Payoffs R S T P

Player A

Player B

Rounds10

Continuation probability δ

A’s total—

B’s total—

Round-robin winner—

Cooperation needs—

Method

Reading a match and a tournament

Match first, then the field.

Write each strategy as a rule

State what it plays in round one and what it plays given the history. If you cannot, you do not yet know what the lab ran.

Play the first few rounds by hand

Round one needs no history. Round two uses round one's moves, and so on.

Add each side's payoffs

Use R, S, T and P for the four move pairs. The score of a match is a sum.

Compare the totals within the match

This says who won that match. It says nothing yet about the tournament.

Read the round robin as sums over partners

A strategy's total adds its scores against all seven, including itself. Find which partners it gained from and which it lost to.

Matches and totals

Score 0 / 0

Choose the answer you can justify.

Tit for tat against always defect

Ten rounds, with R = 3, S = 0, T = 5, P = 1.

Round 1: TFT plays C, ALLD plays D -> 0 and 5 Rounds 2 to 10: both play D, nine rounds -> 9 and 9 TFT total: 0 + 9 = 9 ALLD total: 5 + 9 = 14 Round robin: TF2T 185, TFT 184, ALLD 146

TFT loses the match by five and finishes one point behind the winner of the round robin. ALLD beats or ties every partner it meets and finishes sixth of seven. The two facts are consistent because the tournament adds points and does not count victories.

Common mistakes

Thinking the strategy that beats every opponent wins the tournamentALLD scores at least as much as its partner in every match and still finishes sixth of seven at ten rounds, on 146 points against 185 for the winner. TFT never beats a partner and finishes one point behind. A tournament adds points across partners, and strategies that cooperate with one another earn 30 a match where mutual defection earns 10.

Reading the winner as the best strategyThe winner is the best of these seven against these seven at this number of rounds. At one round ALLD and STFT win; at five, TF2T; at eight, TFT and TF2T tie. No claim about the best strategy in general follows.

Treating reciprocity as a feelingTFT is a rule with no attitude in it: copy the last move. That it earns well is a fact about the payoffs and the partners, which is what the next lesson turns into a threshold.

Check your understanding

✓

Finish when you can read a match round by round and say why the totals came out as they did.

Given two of the seven strategies and a number of rounds, write out the moves of the first few rounds, sum the payoffs, and say which of the two scored more and by how much.

Note: The seven strategies and the round-robin rules are fixed by the lab; the payoffs and the number of rounds are yours. Raising the number of rounds to fifty puts TF2T further ahead, and the winner at a single round is a defector. The last tile, the continuation probability that cooperation needs, belongs to “The Shadow of the Future”; nothing in this lesson depends on it.