A Game Theoretic View of the Atonement

The Prisoner’s Dilemma came up in the comments to a post of mine from about a month ago. I outlined my thoughts very briefly there (see comment #12), but I’d like to return to them in more depth today.

The Prisoner’s Dilemma is perhaps the most important scenario studied in game theory, and “it shows why two individuals might not cooperate, even if it appears that it is in their best interests to do so.” To understand the analysis, however, I’ll need to back up and give a very brief game theory primer.

 

In game theory, a game is a situation where two players each face two or more options in pursuit of goals which are at least partially in conflict, and where the outcome of the situation depends on the choices that each player makes. This interdependence is what game theory from more general decision theory. In traditional parlance, games are won and lost, but in game theory game are solved when you understand exactly what decisions the players will take when each takes into account the actions of every other player This arrangement of complementary player actions is called an equilibrium.

There are many kind of equilbria, but the most important is the Nash equilibrium. A Nash equilibrium is a set of of player actions such that each player has no incentive to change his or her action in response to the actions chosen by other players.

Now we’re ready to see how the concept of Nash equilibrium applies to an example of the Prisoner’s Dilemma. According to the initial setup, two prisoner (Prisoner A and Prisoner B) have been arrested. They are currently being held in separate interrogation rooms, and each faces a simple option: rat out their fellow prisoner (“defect”) or keep mum (“cooperate”). The results are of these decisions are illustrated in the table below. (This format is called a payoff matrix, if you’re curious.)

Intuitively, it seems like the best solution is for the two prisoners to cooperate. In that case, as the payoff matrix describes, each serves a short, 1-year prison sentence. Unfortunately, this arrangement is not a Nash equilibrium. To see why, just ask whether there’s anything Prisoner A would prefer, given that Prisoner B is choosing to cooperate. There is.

If Prisoner A knew that Prisoner B was going to cooperate and stay mum, then Prisoner A would prefer to defect by ratting out Prisoner B. In that case, Prisoner B gets a harsh 3-year sentence, but Prisoner A gets off with no jail time at all. So the “cooperate/cooperate” response is not a solution to the game. Neither, of course, is “cooperate/defect”. In that case, If Prisoner B knew that Prisoner A was going to defect, Prisoner B would also want to defect to reduce his or her jail time from 3 years to 2 years.

So we’ve ruled out “cooperate/cooperate” along with “cooperate/defect”, and we see that “defect/cooperate” is no better. What about “defect/defect”? Well, in that case, nothing that either Prisoner can do unilaterally will increase their payoff. If either player shifts, alone, from “defect” to “cooperate”, that player will end up serving 3 years instead of 2. Well, why don’t they just both switch to cooperate?

The answer is that if you believe that they are rational and that their payoffs are fully reflected in the table, they can’t. By far one of the biggest stumbling blocks to understanding game theory is trusting the payoff matrix. Intuitively, we understand that if we ratted out our compatriot we would feel guilty and if we cooperated together we would feel a sense of vindicated trust. When we imagine outcomes like this, we’re basically stating that the payoffs in the payoff matrix aren’t correct. That’s fine as far as it goes. You can draw your own payoffs for the story about two prisoners who have been captured and factor in things like guilt and friendship and easily jury-rig a game where “cooperate/cooperate” is a Nash equilibrium and the story has a happy ending (unless you’re the cops, I guess). So far so good, but you haven’t solved this game , you’ve solved some other game. You’re no longer talking about the Prisoner’s Dilemma (capital P, capital D). In that game, the total payoffs–including all factors like guilt, friendship, honor, shame, and so forth–have been incorporated into the table above. Your prisoners might feel so bad about turning in their fellow conspirator that it outweighs the benefits of turning states evidence, but these prisoners do not. In our case, Player B knows that if he chose to cooperate, Player A would rather defect and vice versa. Therefore they must each choose “defect” as a matter of self-preservation.

Of course the particular story about prisoners being interrogated and choosing whether to cooperate or defect is pretty specific, but what makes the Prisoner’s Dilemma so important is that the overall structure can be used to represent a wide variety of important, real-world problems. That very partial list includes things like doping in professional sports, how much a company should spend on advertising, climate change, and international arms races. The basic question is simply this: how do you get people to cooperate when there’s a benefit from exploiting one another’s attempt to cooperate? In that sense, I believe it’s the fundamental practical ethical question humanity faces.

In that context, the non-existence of a Nash equilibrium to support “cooperate/cooperate” is disheartening, to say the least.

But all is not necessarily lost. If the Prisoner’s Dilemma is a model for the core moral consideration in human interaction, then it’s obviously not played in a vacuum. Instead, many rounds are played, sometimes with new players and sometimes with the same players many times in a row. How many times do you encounter a situation that could be modeled as the Prisoner’s Dilemma with your coworkers? Friends? Family? Spouse? Many, many times every single day. The quest is not to find the optimal strategy for a single, isolated instance of the Prisoner’s Dilemma (we already know that is “defect/defect”), but rather to find a strategy for how to play the Prisoner’s Dilemma an indefinite number of times with an indefinite number of other players.

We’re now in the context of the iterated Prisoner’s Dilemma. The bad news is that the techniques for determining equilibria in iterated games are substantially more complex than in stand-alone games. The good news is that in iterated games, “cooperate/cooperate” can be supported over time as a Nash equilibrium (or one of the more sophisticated equilibria like subgame perfect Nash equilbria or Bayesian Nash equilibria that are refinements of the basic Nash equilibrium concept). The unfortunate thing about the solutions to the iterated Prisoner’s Dilemma is that they rely on threats. In fact, much of what makes analysis of iterated games complicated is trying to determine which threats are credible. As a general rule, the harsher the available threat, the easier it is to achieve cooperation.

The biggest breakthrough in the iterated Prisoner’s Dilemma came in the early 1980s when political scientist Robert Axelrod sponsored a series of Prisoner’s Dilemma tournaments in the early 1980s. The setup was very simple: a bunch of computer programs were to be matched against each other in a series of repeated Prisoner’s Dilemma games, and the winner would be the program that garnered the most total points over the span of all the games. A wide variety of incredibly sophisticated and complex strategies were submitted, many of which relied on attempting to learn the strategy of the other computer programs to subsequently exploit it. What stunned the researchers, however, was that the very simplest programs (comprising just 4 lines of code) was also the most successful. The program was called simply tit-for-tat. All it did was this: start out by cooperating and then, on every subsequent game, simply repeat whatever the opponent had played in the previous game. The explanation for exactly why this simple strategy was so successful is complex, and it basically launched the study of evolutionary cooperation, but it boils down to this: be nice but provocable.

My own belief is that, in moral terms, tit-for-tat looks a whole lot like “eye-for-an-eye”. It is, in essence, an implementation of justice and retribution. It’s highly successful relative to other strategies, but it requires infliction of pain to work. Is there something better?

I believe so, but before I get to my theories and close the post, a couple of qualifiers. First: the tit-for-tat strategy is considered the most robust strategy over all, but in any given scenario the actual best strategy depends on the composition of the other players. If a single tit-for-tat player gets dropped into a pool of players who play defect all the time, then the tit-for-tat player will lose. Furthermore, the best strategy also depends on the specific rules of the contest and especially the number of rounds and the method for matching the players for each round. Trying to decide which set of rules offers a good model for real life is a complex issue in and of itself.

Secondly: you might be wondering why anyone bothers trying to explain human interaction using sophisticated mathematical models at all. To that I can only offer this explanation: although obviously mathematical models such as these cannot hope to capture the full complexity of human interactions, game theoretic analysis has led to crucial insights into real-world problems in the past. As an example, I would suggest Thomas Schelling and his work in books like The Strategy of Conflict. I don’t claim game theory is the only way to approach these issues–not remotely!–but I do believe it offers unique and important insights.

So, what can we learn from applying game theory to humanity?

First, I think we can learn to recognize how difficult the problem of building Zion truly is. Placing ourselves at risk goes against our rational self-interest, at least in the short-run. In addition–and unlike the computer programs in Axelrods tournament–humans make mistakes. This means sometimes we defect when meant to cooperate, or cooperate when we meant to defect. These mixed signals dramatically complicate our attempts to grapple with the practical and ethical problems of learning to cooperate. (The connection to Zion is simply this: I imagine a society where everyone chooses “cooperate” all the time is a partial glimpse of Zion).

Second, I think we should be careful about how we apply “be nice, but provocable” to human nature. In the context of the original Prisoner’s Dilemma and Axelrod’s tournament, the negative effects of defection had to come from the other player. But if sin brings with it a natural consequence, then the provocation may not be required from the nice agents. It’s built into the world. On the other hand, if the negative effects are delayed or disguised by noise, additional chastisement might be required to alert the players of the true payoffs. This is a technically crucial point, because if I’m going to cite game theory to get me this far, I have to rely on it to solve the problems I’ve raised. And the only way out of the Prisoner’s Dilemma, even the iterated version, is threats. A Zion where cooperation is maintained by threat hardly seems like a Zion at all. If the punishment is external, however, and doesn’t rely on retaliation, then we’re changing the payoffs and thus the nature of the game. And I believe that indeed part of what the Gospel does is change our perceptions of the payoffs.

Third, there’s a lot to learn from the way that the programs interacted in Axelrod’s tournament. Specifically: the “nice” programs tended to significantly outperform the exploitative programs when each dealt with similar programs. (Neighborhoods of nice AIs do much better than neighborhoods of exploitative AIs or mixed neighborhoods.) In addition, there’s a very slight improvement on tit-for-tat that can be used to get a marginal increase in performance called tit-for-tat-with-forgiveness. This strategy is the same as tit-for-tat, except that it sometimes (1-5% of the time) replies to a defect by cooperating instead of defecting. Of course, the term “forgiveness” is loaded, and I don’t think it should be read in the usual, religious sense because the whole concept of animosity or resentment is outside the scope of this model. So, instead of forgiveness, I think that sometimes departing from tit-for-tat to proffer cooperation despite an opponent’s betrayal in a previous game can be seen as a kind of investment that serves two purposes. First, it helps to identify “nice” players to each other, which enables them to collect in neighborhoods. (It also allows nice players to overcome the problems represented by their imperfections).

This introduces the notion of signalling (an economics term for reliable communication, as opposed to cheap talk). In this sense, deliberately sacrificing well-being is a kind of call to others. It’s capacity for use as a communications medium was demonstrated in a 20th anniversary replay of the Axelrod tournament. In that case, the winners submitted multiple versions of the same program. These programs were designed to perform a specific pattern of 10 moves to identify each other in the game. If two of the same AI found each other, they then choose “cooperate/defect” to maximize the value to one of the players, but if the two AI found other opponents, they would always choose “defect” to minimize the benefit to the other player. At the end of the competition, these AI filled most of the very top and very bottom spots in the ranking. This illustrates the power of being able to communicate with like-minded agents to circumvent the traditional assumptions.

But I think that signalling can do more than just attract those are already “nice.” Returning to the idea that beating the game involves changing the payoffs, I think that communities of “nice” agents can serve as an example that cooperation has additional benefits when you consider the aggregate payoff instead of only the individual payoffs. This is especially true if, over time, the benefits from communities of nice agents can be reinvested to create even better payoffs in the future. This creates a kind of “city on a hill” effect, but there’s another possible role for the idea of sacrifice.

If the basic premise is correct, if deliberately choosing to cooperate as a deviation from tit-for-tat (from justice) works as a signal because it is costly (because it renders the person who chooses this path vulnerable), then the strength of the signal corresponds to the depth of the sacrifice. If this is true, then the strength of the signal depends in a sense on the cost of the sacrifice to send it. An infinite cost begets infinite credibility, and becomes an eternal beacon with a simple message: cooperate. That’s my game-theoretic perspective on the Atonement.

I don’t believe that this perspective is in any sense the exclusively “right” way to look at the Atonement. What I believe is that the Prisoner’s Dilemma represents a deeply important problem of practical ethics. What I’ve outlined here is an informal sketch of a re-conceptualization of the iterated Prisoner’s Dilemma in the context of a complex system. The most important change, both philosophically and technically, from the canonical example is that I’ve introduced the idea of learning. Specifically, in the model I have in mind, we can learn that cooperation leads to increasing returns as networks of cooperating players experience benefits from repeated cooperative interaction. If we can learn, then there is the possibility of changing our perceptions of the payoffs and therefore changing the game. And that’s the part of the impact that I believe Christ had: His sacrifice showed us that the game can be changed.