On Replicator Dynamics and Evolutionary Games

Transcription

1 Explorations On Replicator Dynamics and Evolutionary Games Joseph D. Krenicky Mathematics Faculty Mentor: Dr. Jan Rychtar Abstract We study the replicator dynamics of two player games. We summarize the development of the dynamics and we present known classifications of the games. The summary is based on the outcomes of the dynamics. We explain the dynamics on the classical prisoner s dilemma game. We also study the asymptotic behavior of the dynamics. This is a preparatory work for more intense study and a development of new models of evolutionary dynamics. Introduction The foundations of game theory applied to biology were done by Maynard Smith and Price [9] and Maynard Smith [8]. Evolutionary dynamics has many applications in mathematics, biology, economics, and many other sciences, see Cressman [6] or Samuelson [10] for examples. A basic knowledge of games lends itself to a greater understanding of dynamic evolutionary theory. Originally conceived in earlier biological research, games try to explain the interactions within a population. When the interactions between two species are likened to a rational game and two strategies (e.g., behavior encoded in DNA) are available to the population of otherwise identical individuals, then three classes of symmetric normal form games can result: the Prisoner s Dilemma, Coordination, and the Hawk-Dove. The goal of this note is to introduce the replicator dynamics of two player games, to classify games according to the outcome of the dynamics, and to identify which strategies, if any, are evolutionary stable. An evolutionary stable strategy is a strategy such that if all members of a population adopt it, then no other strategy could invade the population under the influence of natural selection, Maynard Smith [8]. Mathematical analysis of games Here we introduce the replicator dynamics for symmetric games with two players and two strategies in a similar manner as done in Cressman [6]; this seems to be the most accessible construction. For alternative approaches, see Samuelson [10], Hofbauer and Sigmund [7], Binmore [3], or Taylor and Jonker [13]. It is assumed that the population consists of two types of individuals: players adopting strategy A and players adopting strategy B. During a given time period (a mating season, for example) each individual interacts with one randomly chosen opponent. The player receives a payoff based on the payoff matrix: 1

2 B Mathematics A a b B c d The payoff to an A player who interacts with a player A (B) is a (b). The payoff to player B who interacts with a player A (B) player is c (d), see Stahl [12] for a more detailed exposition of this type of game. Assume there are N A individuals adopting strategy A (respectively, N B individuals adopting strategy B). The average payoffs (or average fitness), f A and f B B for each player A and each player B, are given by the formulas: f A = a N A /( N A + N B ) + b N B /( N A + N B ) f B B = c NA/( N A + N B ) + d N B /( N A + N B ) where factor N A /( N A + N B ) B represents the chance of meeting a player A and factor NBB/( N A + N B ) represents the chance of meeting a player B. The standard exponential growth model (for an example, see Anton et al. [1]) is used to model the growth of subpopulations of A (or B) players: d/dt N A = f A N A d/dt N B = f B N B Adding these equations together provides an equation for the total size N = N A + N B of the population: d/dt N = d/dt (N A + N B )= f A N A + f B N B Combining all of the above equations yields the replicator dynamic that describes the evolution of p A (=N A /N), the proportion of the population that adopts a strategy A: d/dt p A = d/dt N A /N = p A (1-p A ) ((a-c)p A +(b-d)(1-p A )). Note that it is enough to study the dynamics for p A (the fraction of A individuals in the population) since the dynamics for p B B is determined by the equation pa + p BB =1. In the next section we will analyze the above dynamics. Analysis of the replicator dynamics It is beneficial to think about the replicator dynamic as the equation: where: d/dt p A = F(p A ) F(x) = x(1-x)((a-c)x+(b-d)(1-x)). We see that the replicator dynamic is a first order differential equation, and using the standard tools from an introductory differential equation course (for example, see Brauer and 2

3 Explorations Nohel [5]), we can investigate the solutions of the dynamics without explicitly finding them. In particular, we can immediately see that the equation has exactly three constant solutions: p A = 0 p A = 1 p A = (b-d)/((b-d)- (a-c)) as each of them is a root of F(x) = 0. The third point is of a special importance as it is the only possible nontrivial root. The constant solutions are called rest points of the dynamic. The replicator dynamic distinguishes three types of games (as classified in Cressman [6] or Weibull [14]). Depending on the constants a, b, c, and d, the third root may or may not be in the interval [0, 1], the only interval where the quantity p A makes sense. If the third root is not in the unit interval, then the dynamics has only two rest points. If it is in the interval, it may or may not be asymptotically stable. A constant solution p A = p 0 is called asymptotically stable if all solutions that start sufficiently close to p 0 must eventually approach p 0 as t (see Boyce and DiPrima [4], p. 497, for more details). In our setting, asymptotical stability is the same as evolutionary stability. The stability of a point p 0 depends on the sign of F (p 0 ). If F (p 0 ) is negative, then the point is stable. If F (p 0 ) is positive, then it is not stable. Indeed, if F (p 0 )<0, F is decreasing at p 0 and thus if p A starts slightly above p 0, p A has to decrease as it solves d/dt p A = F(p A ) < F( p 0 )=0. Thus p A stays above p 0 but it is getting closer to p 0 and therefore has to converge to p 0 if chosen sufficiently close to p 0. Similar arguments work if p A starts slightly below p 0 or if F (p 0 )>0. Based on the above mathematical analysis, we have three classes of games. Each class is named after the most popular game characterizing it. The Prisoner s Dilemma class results if the nontrivial root is not in the interval, i.e. if the payoffs satisfy: (a-c)(d-b)<0. The dynamics then has two rest points. Only one point is stable. After renaming the strategies, we may assume that the strategy B is the stable one. The classical example from this class is the Prisoners Dilemma game (Axelrod [2]), which is a game that arises from the following payoff matrix: A 3 0 B 5 1 In the most famous variant of the Prisoner s Dilemma game, there are two strategies available for the population: either to defect or to cooperate. The payoffs are described in the following instances. If the two competing players cooperate, they both receive a Reward (R). Alternatively, when both competing players defect, they both receive a Punishment (P). In the third case, one player cooperates and one player defects. The defecting player earns a bonus, Temptation (T) and the cooperating player earns nothing, the Sucker s payoff (S). What makes the game to be the prisoner s dilemma is the relative magnitude of the payoffs: T>R>P>S. 3

4 Mathematics It is obviously in the best interest of the population for the players to cooperate. However, the population will always evolve to a state where all players defect. Here lies the crux of the situation. If the population consists of cooperating individuals only, a defecting player (that can appear in the population either by mutation or by migration, etc.) will have a big advantage against cooperating players and thus the number of defecting players will grow. If the population consists of a mixture of cooperating and defecting players, the defecting players will, on average, do better than cooperating ones (notice that the payoffs for B are higher than the payoffs for A regardless of the opponent). Since defecting players do better, their numbers will grow. If the population consists of defecting individuals only, the cooperating individual cannot survive (because if it appears, it performs poorly in competition with defectors). In the language of replicator dynamics, the population of cooperators is a rest point. Since the defectors have a larger fitness in any population, it is not stable. Once a single defector appears, it will have a bigger fitness than the cooperators. So, the proportion of defectors will start to increase until the population reaches the other rest point of defectors only. The Coordination class results if the nontrivial root is in [0,1], but it is not stable, i.e. if the payoffs satisfy: a>c and d>b. The dynamics then has three rest points; the two corner ones are stable. The classical example of this game is the Stag hunt game (see Skyrms [11]): A 4 0 B 3 3 This class of games is characterized by individuals cooperating with others in the population adopting the same strategy. The population eventually evolves towards one where every individual adopts the identical strategy. Which of the two strategies adopted will depend on the initial population profile and it is not determined a priori by the payoff matrix. The Hawk-Dove class results if the nontrivial root is in [0,1] and it is stable, i.e. if the payoffs satisfy: a<c and d<b. The dynamics then has three rest points; only the middle one is stable. The classical example of the Hawk-Dove game was introduced by Maynard Smith [8]. It is represented by the matrix: A -1 6 B 0 3 A typical Hawk-Dove game has an aggressive strategy and a passive strategy. A large proportion of aggressive individuals is not stable because they make cohabitation difficult. On the other hand, a large proportion of passive individuals is not stable because the presence of an aggressive strategy would wreak havoc on the population. Therefore, the population evolves to coexistence. 4

5 Explorations Asymptotic behavior Since each root r of F has a multiplicity of one, one locally has: F(x) = F (r)(x-r) which means that if p A is almost r, it behaves like a solution of: i.e. like an exponential function: d/dt p A = F (r)( p A -r), p A (t) = p A (0) exp(f (r)t)+r. The above formula provides an asymptotic behavior around constant solutions and a behavior of the solutions for large, almost infinite, time intervals. As a consequence we have that p A converges to stable rest points exponentially fast but it will never reach them. On the example of the Prisoner s dilemma game it means that the population will always evolve to the population where almost all are defectors, but there will always be a tiny fraction of cooperators. In the coordination class, the population will always evolve to one where almost all behave in the same way, but there will always be a tiny fraction of individuals that will behave differently. Future Plans We plan to modify the dynamics to incorporate the effects of mutations explicitly in the model, i.e. what would happen if we assume that a small portion of A players mutates to become B players and visa versa. This will modify the exponential growth model introduced here and in turn, the whole replicator dynamics. Acknowledgements The research was supported by the UNCG New Faculty Grant (awarded to Jan Rychtar) and UNCG Office of Sponsored Programs. The authors would like to thank the anonymous referee panel for their valuable comments and suggestions. Joseph Krenicky worked as Jan Rychtar s undergraduate research assistant and presented this paper at the North Carolina Undergraduate Research Symposium, Raleigh, November 12, 2005 as a poster, No way out? Prisoner s Dilemma. 5

6 Mathematics REFERENCES H. Anton, I. Bivens, and S. Davis, 2005, Calculus 8 th ed., John Wiley & Sons, Inc., State (2005). R. Axelrod, The Evolution of Cooperation, Basic Books, New York (1984). K. Binmore, Fun and Games, D.C. Heath, Lexington, MA (1992). W.E. Boyce and R.C. DiPrima, Elementary Differential Equations, 8 th ed., John Wiley & Sons, Inc., State (2005). F. Brauer and J.A. Nohel, Ordinary Differential Equations: A FirstCourse, W.A. Benjamin, New York (1967). R. Cressman, Evolutionary Dynamics and Extensive Form Games, MIT Press, Cambridge, MA, (2003). J. Hofbauer, and K. Sigmund, The Theory of Evolution and Dynamical Systems, Cambridge University Press, Cambridge (1998). J. Maynard Smith, Evolution and the Theory of Games, Cambridge University Press (1982). J. Maynard Smith and G.R. Price, The Logic of Animal Conflict. Nature 246 (1973), L. Samuelson, Evolutionary Games and Equilibrium Selection, MIT Press, Cambridge, MA (1997). B. Skyrms, The Stag Hunt and Evolution of Social Structure Cambridge, Cambridge University Press (2003). S. Stahl, A Gentle Introduction to Game Theory, American Mathematical Society, Providence, R.I. (1999). P.D. Taylor and L.B. Jonker, Evolutionary Stable Strategies and Game Dynamics, Mathematical Biosciences 40 (1978), J. Weibull, Evolutionary Game Theory, MIT Press, Cambridge, MA (1995). 6