The Ultimate Strategy
I just watched this most recent Veritasium video and thought I’d dedicate a short blog post to Game Theory and the Prisoner’s Dilemma. It’s a topic that my dad and I talk about quite often because it seems to have far-reaching implications as well as practical applications to our daily lives.
This won’t be a super comprehensive take on the subject since there are tons of great explainer posts/videos out there on the basics. In fact, I think the full video does a good job of covering those, and I’ll link to some other resources at the end of the post for the curious reader. This post will focus more on the details that stood out to me and I also plan on writing more about this topic in the future so stay tuned. This means I’ll be assuming some level of understanding of the Prisoner’s Dilemma, the concepts of collaboration and defection, Nash Equilibria and how to find them, and the concept of finite/infinite iterative games.
Background
The main focus of this video was on the early research done by Professor Robert Axelrod (whom the main character of the show Billions is named after). It discusses a series of computer tournaments, where different strategies for the Prisoner’s Dilemma were pitted against each other. So for example, some players would always defect on the other player, some always cooperated, others had a mix, and one was even completely random.
The surprising winner was a simple strategy called “Tit for Tat”, which cooperates on the first move and then does whatever the other player did on the previous move.
But what might be surprising about this whole thing is that the “nice” strategies typically outperformed the “nasty” strategies, because the benefits of cooperating with other nice strategies outweighed the costs of nasty strategies defecting on each other.
Here are some of the interesting conclusions that came from this research.
Successful Strategies Shared 4 Common Traits
The first trait was that all the top strategies were “nice.” In this context, it simply meant that they all started by attempting to collaborate with their opponent. This avoided triggering defections from other strategies and usually led to long collaborative stretches throughout the game.
The second trait was that the best strategies were provocable. This means that they wouldn’t just let themselves be taken advantage of, they would respond to their opponents' defections.
The third trait was that the strategies had to be forgiving. This means that after a defection, if the opponent offered to collaborate again, the relationship could be repaired and they could both go back to collaboration.
And the final trait is that a player’s strategy needs to be “clear.” This wasn’t obvious to me at first, but it makes sense because understanding which strategy an opponent is playing makes it easier to assess how to respond. Sneakiness is almost seen then as a form of defection inherently.
These qualities are similar to the principles of fairness and reciprocity that are often found in human societies, and more broadly in the natural world with communal organisms. And this makes sense that our values would emerge out of these more basic principles.
Noise and Generosity
One thing I liked about Axelrod’s approach to his research was that he started simple and started adding complexity over time, and as he did so the simulation seemed to approach the real world more and more.
One important aspect of the game is the number of turns that you play for. You can imagine that the longer you need to play against an opponent will influence which type of strategy you adopt. If it’s a short-term one you might benefit from defecting since the relationship is ending and for a long-term one you would collaborate more since you’ll be playing against the opponent more. Knowing when the game ends precisely (in the experiment it’s 200 turns) can cause certain strategies to work sub-optimally because they start working backward based on the idea that they’ll defect at the very end to get ahead on the last turn. This is similar to the idea of Minimaxing.
To correct for this the researchers added a random element to the number of rounds in the simulation. That uncertainty helped avoid a kind of overfitting of strategies to the duration of the game that was creating a lot of weirdness.
Another way they added complexity was to introduce errors and noise. Basically strategies misconstruing the other’s actions as defections when they were actually collaborations. In the real world this mapped to situations like when the USSR mistook radar readings as an American missile launch.
It turned out that the solution to this was to actually add a slight forgiveness bias. One issue that arose in tit-for-tat strategies was that they’d occasionally enter into this “echo” of alternating collaboration and defection if one side ever defected (either due to some randomness that was introduced into the model or some noise that caused a misunderstanding). The only way to break out of this was for one side to be just a little extra forgiving which would “repair” the relationship and cause both sides to go back to collaborating.
Ecological and Evolutionary Simulation
One cool simulation they also ran was an ecological simulation, where they looked at a population of players, each running one of the strategies. Over time they’d play each other, and depending on their success their populations would either grow or shrink. This was done to test how the effectivness of strategies as the ratio of the kinds of other strategies they encountered changed.
So you could imagine a population full of pure collaborators being overrun and wiped out by a small group of defectors over time.
But it turns out if you run all the strategies together, the nice ones tend to find a stable ratio and the nasty ones tend to go extinct. Once again, the cost of defection outweighs the benefits of collaboration.
What surprised me about this though was that small pockets of “nice” strategies can overwhelm a population of nasty strategies by collaborating with each other and eventually spreading. But thinking about this a bit more it makes sense. Remember “nice” doesn’t mean being a pushover. These strategies still need to be able to retaliate when presented with defectors.
What is Winning Really??
One thing that has made me hopeful for the past few years is knowing that “nasty” strategies tend to wipe themselves out. Over the long run, this seems to lead to more collaboration and less conflict. This seems to be supported by the simulations, but more importantly by real-world data too. The world by most metrics seems to be getting safer and less violent, fortunately.
It reminds me of a quote from the Bible’s Sermon on the Mount in the Gospel of Matthew:
“The meek shall inherit the earth.”
It suggests that those who are humble, gentle, and patient will ultimately be rewarded or inherit the Earth in a spiritual or metaphorical sense.
I liked the way the video concluded things because it shows a very counterintuitive point about what “winning” means.
Nasty strategies can only ever win or tie their matches.
Nice strategies can only lose or tie.
And yet the nice ones perform better, how can that be?
It comes down to how you think about “winning” itself. One way to look at the game is zero-sum, and that you are trying to take from your opponent. In a lot of games we play that is definitely the case.
But another way to look at the game is that you are taking from the banker, or whatever is doing the payout structure. You could look at this as being the world or the Universe itself. It provides, and if you collaborate with your “opponent” you can both reap the rewards together. In this sense, the game becomes a positive sum game where everyone can win.
I try to cultivate this mindset as much as possible and focus more on being a generative force in the world rather than someone who defects and acts on their vindictive impulses. And what I loved most about this video is that it also showed that even small pockets of “niceness” can spread and take over even a completely “nasty” population.
Conclusion
There’s a lot more depth to the topic of Game Theory and I’m sure there will be more to say in future blog posts. But I think the practical life advice you can take away from this is to always start new relationships by collaborating, don’t be a pushover, be able to forgive others, and be clear about what your intentions are. Do that and you’ll prove that:
Nice guys really do finish first.
