Setup and payoffs

In the classic presentation of the Prisoner’s Dilemma, you and your fellow bank robber have been arrested and imprisoned. You cannot communicate with each other. You are facing a prison sentence of one year each. Both of you have been offered a chance to betray the other (Defect); someone who Defects gets one year off their own prison sentence, but adds two years onto the other person’s prison sentence. Alternatively, you can Cooperate with the other prisoner by remaining silent.

So:

• If you both Cooperate (refuse to testify), you each get 1 year in prison.

• If one Defects and the other Cooperates, they go free and the other gets 3 years in prison.

• If you both Defect (testify), you each get 2 years in prison.

Or in the form of an outcome matrix where \((o_1, o_2)\) is the outcome for Player 1 and Player 2 respectively:

As usual, we assume:

• Both you and the other agent are strictly selfish, and don’t care at all what happens to the other.

• You also don’t care about honor or reputation.

• There’s no mob boss to kill anyone who testifies, and you have no other means of enforcement.

• Your utility function is strictly linear in years of prison time avoided.

(For scenarios that would reproduce the resulting ideal structure with more realistic human motives and situations, see true prisoners dilemma.)

Then we can rewrite the Prisoner’s Dilemma as a game with moves \(D\) and \(C,\) and positive payoffs where $X denotes “X utility”:

Significance

In the Prisoner’s Dilemma, each player is individually better off Defecting, regardless of what the other player does. However, both players prefer the outcome of mutual Cooperation to the outcome from mutual Defection; that is, the game’s only Nash equilibrium is not Pareto optimal. The Prisoner’s Dilemma is therefore an archetypal example of a coordination problem.

The Prisoner’s Dilemma provoked an enormous amount of debate, mainly due to the tension between those who accepted that it was reasonable or ‘rational’ to Defect in the Prisoner’s Dilemma, and those who found it hard to believe that two reasonable or ‘rational’ agents would have no choice except to helplessly Defect against each other.

The Iterated Prisoner’s Dilemma (IPD) was another important development in the debate—instead of two agents playing the Prisoner’s Dilemma once, we can suppose that they play the PD against each other 100 times in a row. Another development was ‘tournaments’, run on a computer, in which many programmed strategies play the Prisoner’s Dilemma against every other program. Combined, these yield an IPD tournament, and almost every IPD tournament—whatever the variations—has been won by some variant or another of Tit for Tat, a strategy which Cooperates on the first round and on each successive round just plays whatever the opponent played previously.

Examining such tournaments has yielded the conclusion that strategies should be ‘nice’ (not be the first to Defect, i.e., not play Defect for the opponent has played Defect), ‘retaliatory’ (Cooperate less when the opponent Defects) and ‘forgiving’ (not go on Defecting forever after the opponent Defects once).

The strategy in Tit for Tat stands in contrast to the conclusion that it is reasonable to Defect in the oneshot Prisoner’s Dilemma. Indeed, it stands in contrast to the supposedly ‘rational’ (on some views) strategy in the Iterated Prisoner’s Dilemma. If the game is to be played 100 times, then clearly it is ‘rational’ to play Defect on the last and 100th round. But if both players are ‘rational’ and know that the other is ‘rational’, they both know the other player will reason this way and Defect on the 100th round. Then since play in the 100th round is insensitive to play on the 99th round, both agents reason that they should Defect on the 99th round, and so by induction they both Defect on the 1st and every successive round.

This conclusion has been challenged from many directions, on both the oneshot and iterated Prisoner’s Dilemma. Douglas Hofstadter observed that two rational agents should both realize that there is only one ‘rational’ conclusion, whatever that conclusion is; Hofstadter proposed ‘superrationality’ as rationality taking into account that superrational agents facing similar problems must arrive at similar conclusions. Logical decision theory, which says that the principle of rational choice is to decide as if choosing the logical output of your decision algorithm, can be seen as generalizing this viewpoint. Logical decision theorists have also shown that if the agents in the Prisoner’s Dilemma have common knowledge of each other’s algorithms, they can end up cooperating (and this works even if the two agents are not identical).