Repeated Games. Econ 400. University of Notre Dame. Econ 400 (ND) Repeated Games 1 / 48

Repeated Games Econ 400 University of Notre Dame Econ 400 (ND) Repeated Games 1 / 48

Relationships and Long-Lived Institutions Business (and personal) relationships: Being caught cheating leads to punishment or exclusion Econ 400 (ND) Repeated Games 2 / 48

Relationships and Long-Lived Institutions Business (and personal) relationships: Being caught cheating leads to punishment or exclusion Government: We are willing to give up resources now in the expectation that we will be paid back later (Fiat money, Social Security) Econ 400 (ND) Repeated Games 2 / 48

Relationships and Long-Lived Institutions Is cooperation always good? Econ 400 (ND) Repeated Games 3 / 48

Relationships and Long-Lived Institutions Is cooperation always good? Is anonymity always bad? Econ 400 (ND) Repeated Games 3 / 48

Relationships and Long-Lived Institutions Is cooperation always good? Is anonymity always bad? Do we value relationships because of the relationship per se, or for the stream of benefits it provides? Is that bad? Econ 400 (ND) Repeated Games 3 / 48

Repeated Prisoners Dilemma Suppose two players are going to play prisoners dilemma t = 1,2,...,T times, where the payoffs are given by S C S 1,1-1,2 C 2,-1 0,0 Econ 400 (ND) Repeated Games 4 / 48

Repeated Prisoners Dilemma Suppose two players are going to play prisoners dilemma t = 1,2,...,T times, where the payoffs are given by S C S 1,1-1,2 C 2,-1 0,0 What are the subgame perfect Nash equilibria of the game? Econ 400 (ND) Repeated Games 4 / 48

Repeated Prisoners Dilemma But what if the other players was known to be a nice guy/gal, who plays S as long as you have played S in all previous periods, then chooses C for all future periods once you have chosen C i.e., they play nice until you cheat them. Maybe now there will be cooperation for some number of periods? Consider the payoff from choosing S for the first T 1 periods, then choosing C in the final period to get the 2. Then the payoff from the strategy (S,S,...,S,C) is: 1+δ +δ 2 +...+δ T 1 +2δ T = 1 δt 1 δ +2δT Econ 400 (ND) Repeated Games 5 / 48

Repeated Prisoners Dilemma But then comparing confessing first at T 1 to confessing first at time T gives ( 1 δ T 1 ( 1 δ +2δ ) T 1 T ) ( ) 1+δ 1 δ 1 δ +2δT = δ T 1 1 δ +2 δ > 0 So it looks like the unravelling is going to occur, even if one player is willing to cooperate. Econ 400 (ND) Repeated Games 6 / 48

Repeated Prisoners Dilemma But let s look at the difference in payoffs between confessing for the first time in period T 1 and confessing for the first time in period T again: ( ) 1+δ δ T 1 1 δ +2 δ As T gets large, δ T 1 becomes small, and cooperation is almost an equilibrium of the repeated game early on. But as the end of the game approaches, the unravelling motive will kick in. Econ 400 (ND) Repeated Games 7 / 48

Repeated Games Definition A repeated game is (i) a terminal date, T, giving the number of times the players interact, where T = 1,2,3,... The calendar date is given by t = 1,2,...,T. (ii) a discount factor, 0 δ 1, that represents both how patient the players are and how likely the game is to continue. (iii) a stage game of finite length: A specification of the players, actions, payoffs, and timing, which is usually independent of the calendar date. Econ 400 (ND) Repeated Games 8 / 48

Histories Presumably, players will want to keep track of how their opponents have behaved in previous periods; this allows them to choose strategies that reward or punish other players for good or bad behavior. But how do we keep track of what has happened in repeated games? In prisoners dilemma, all of the possible outcomes from one repetition of the game are Σ = {(S,S),(S,C),(C,S),(C,C)} When the game goes two periods, however, it becomes Σ 2 = {[(S,S),(S,S)],[(S,C),(S,S)],[(C,S),(S,S)],[(C,C),(S,S)], [(S,S),(S,C)],[(S,C),(S,C)],[(C,S),(S,C)],[(C,C),(S,C)], [(S,S),(C,S)],[(S,C),(C,S)],[(C,S),(C,S)],[(C,C),(C,S)], [(S,S),(C,C)],[(S,C),(C,C)],[(C,S),(C,C)],[(C,C),(C,C)]} since we need to keep track of what happened in the first period, and the second period. Econ 400 (ND) Repeated Games 9 / 48

Histories Let Σ be the set of all the strategy profiles for the stage game. (For example, in prisoners dilemma, Σ = {(S,S),(S,C),(C,S),(C,C)}). Econ 400 (ND) Repeated Games 10 / 48

Histories Let Σ be the set of all the strategy profiles for the stage game. (For example, in prisoners dilemma, Σ = {(S,S),(S,C),(C,S),(C,C)}). If we want to keep track of the outcomes of a repeated game, we re interested in sequences of observations from Σ. For two periods, Σ Σ = Σ 2 is the set of all possible outcomes for two repetitions of the game. For three periods, Σ Σ Σ = Σ 3 is the set of all possible outcomes for three repetitions of the game, and so on. Econ 400 (ND) Repeated Games 10 / 48

Histories Definition Let the set of all strategy profiles for the stage game be Σ. Then the set Σ t = Σ Σ... Σ contains all lists of the possible outcomes in terms of what strategies the players have used in each of the t periods. Then any element h t of the set Σ t = H t is a history at time t. Econ 400 (ND) Repeated Games 11 / 48

Equilibria in Repeated Games Definition A set of strategies is a Subgame Perfect Nash Equilibrium of a repeated game if, for any t-period history h t, there is no subgame in which any player has a profitable deviation. Note that no player can have a profitable deviation for any history, even though, given the strategies, only one history actually occurs. But it is precisely because the players know the consequences of their actions that the equilibrium history arises. Econ 400 (ND) Repeated Games 12 / 48

Equilibria in Repeated Games Suppose the stage game has an equilibrium s = (s1,...,s N ). Then the strategy for the repeated game where each player i plays si after every history is a Subgame Perfect Nash Equilibrium of the repeated game. Econ 400 (ND) Repeated Games 13 / 48

Prisoners Dilemma Consider the following behavior strategy for an infinitely repeated prisoners dilemma: If the history at time t is {(S,S),(S,S),...,(S,S)}, play S. For any other history at time t, player C. Econ 400 (ND) Repeated Games 14 / 48

Prisoners Dilemma: Equilibrium Analysis Well, we have to check all the possible histories to see if there are any profitable deviations. There s really just two cases: and anything else. {(S,S),(S,S),...,(S,S)} Econ 400 (ND) Repeated Games 15 / 48

Prisoners Dilemma: Equilibrium Analysis Well, we have to check all the possible histories to see if there are any profitable deviations. There s really just two cases: {(S,S),(S,S),...,(S,S)} and anything else. Let s start with anything else : Suppose the history at time t is not {(S,S),(S,S),...,(S,S)}. Then today and in all future periods, all my opponents will choose C. Then I should choose C, since it maximizes my discounted payoff. So there are no profitable deviations from the proposed strategies for these histories. Econ 400 (ND) Repeated Games 15 / 48

Prisoners Dilemma: Equilibrium Analysis Suppose the history at time t is {(S,S),(S,S),...,(S,S)}. Is playing S an optimal strategy, given the behavior strategies of the players? If I choose S this period, the next period s history is {(S,S),(S,S),...,(S,S),(S,S)}, and given the strategies, the players will choose S from then on forever. The payoff from that is 1+δ +δ 2 +δ 3 +... = 1 1 δ If I deviate this period, next period s history is {(S,S),(S,S),...,(S,S),(C,S)} {(S,S),(S,S),...,(S,S),(S,S)} so all players will confess, generating a sequence of histories where players confess forever. Econ 400 (ND) Repeated Games 16 / 48

Prisoners Dilemma: Equilibrium Analysis So it all comes down to whether it s better to cooperate than cheat in any period, or 1 1 δ 2 δ 1 2 Econ 400 (ND) Repeated Games 17 / 48

Prisoners Dilemma: Equilibrium If δ > 1 2, both players using the strategy If the history at time t is {(S,S),(S,S),...,(S,S)}, play S. For any other history at time t, play C. is a Subgame Perfect Nash Equilibrium of the infinitely repeated prisoners dilemma. Econ 400 (ND) Repeated Games 18 / 48

Collusion and Bertrand In the Bertrand pricing game, there are two firms 1 and 2 who have the same marginal costs c and compete in prices, choosing p 1,p 2 = {0,1,...,c,...,8,...,10}. Fix demand at 1 for all prices less than or equal to 10. If p 1 < p 2, all the consumers go to firm 1 and firm 2 gets no business. If p 1 = p 2, the firms split the market equally. If p 2 > p 1, all consumers go to firm 2 and firm 1 gets no business. Econ 400 (ND) Repeated Games 19 / 48

Collusion and Bertrand Consider the repeated game: T = Discount factor: 0 < δ < 1 Stage game: Bertrand Competition Econ 400 (ND) Repeated Games 20 / 48

Collusion and Bertrand Consider the repeated game: T = Discount factor: 0 < δ < 1 Stage game: Bertrand Competition Notice that if there are 11 price increments, there are (22) t possible outcomes we might observe by time t. If t = 5, there are 5,153,632 histories. Even on a very good computer, computing the extensive form and payoffs would take a lot of time. Econ 400 (ND) Repeated Games 20 / 48

Bertrand: Stage Game Equilibrium and Trigger Strategies We know from the first part of the class that p1 = p 2 = c is a Nash equilibrium of the stage game. Let s use this as the punishment for a breakdown in cooperation, but otherwise have the players use p 1 = p 2 = 10. Consider the strategies: If the history is {(10,10),(10,10),(10,10),...,(10,10)}, play 10 this period. For any other history, play c this period. Econ 400 (ND) Repeated Games 21 / 48

Bertrand: The Optimal Deviation If your opponent adopts these strategies and plays nice up to some date t = 0,1,..., what is the best way to stab him in the back? Econ 400 (ND) Repeated Games 22 / 48

Bertrand: The Optimal Deviation If your opponent adopts these strategies and plays nice up to some date t = 0,1,..., what is the best way to stab him in the back? (If you re going to ruin a relationship, at least do it optimally) By charging the price just below the cutoff point, 9 1, the deviator captures the whole market and only losses a dollar on each unit: π d = (1)(9 1 c) Econ 400 (ND) Repeated Games 22 / 48

Bertrand: Equilibrium Analysis Once a deviation has occurred, the players use p1 = p 2 = c in all future periods. In any of these scenarios, there are no profitable deviations, because if your opponent is playing c in these periods, your best response is c. The payoff to cooperating is 10 c 2 +δ 10 c 2 +δ 210 c 2 The payoff to optimally deviating is +... = 10 c 2 1 1 δ (9 c)+δ0+δ 2 0+... = 9 c Econ 400 (ND) Repeated Games 23 / 48

Bertrand: Equilibrium Analysis Then cooperating is better than deviating if or 10 c 2 1 1 δ 9 c δ 8 c 19 2c Econ 400 (ND) Repeated Games 24 / 48

Bertrand: Equilibrium Analysis Then cooperating is better than deviating if or 10 c 2 1 1 δ 9 c δ 8 c 19 2c For example, if c = 2, we have δ 6/15. But if c = 6, we have δ = 2/7. Econ 400 (ND) Repeated Games 24 / 48

Bertrand: Equilibrium Then as long as the strategies δ 8 c 19 2c If the history is {(10,10),(10,10),(10,10),...,(10,10)}, play 10 this period. For any other history, play c this period. are a Subgame Perfect Nash Equilibrium of the infinitely repeated Bertrand game. So collusive is possible in the infinite-horizon version of the repeated game. (What about the finite version of the repeated game?) Econ 400 (ND) Repeated Games 25 / 48

The Pattern Solve for all of the equilibria of the stage game. (Competitive Play) Econ 400 (ND) Repeated Games 26 / 48

The Pattern Solve for all of the equilibria of the stage game. (Competitive Play) Find a strategy profile that gives all the players a higher payoff. (Cooperative Play) Econ 400 (ND) Repeated Games 26 / 48

The Pattern Solve for all of the equilibria of the stage game. (Competitive Play) Find a strategy profile that gives all the players a higher payoff. (Cooperative Play) Enforce cooperation through trigger strategies: If all players have previously cooperated, continue cooperating. If any player has previously defected, play competitively. Econ 400 (ND) Repeated Games 26 / 48

Examples of Equilibria in Repeated Games For any game, playing the equilibrium of the stage game forever is a Subgame Perfect Nash Equilibrium. In prisoners dilemma, as long as the players didn t discount their payoffs too much (δ >.5), one Subgame Perfect Nash Equilibrium of the game was to be silent in all periods unless your opponent had previously confessed at some point, and then to confess forever. In Cournot, as long as the players didn t discount their payoffs too much (δ >.377), one Subgame Perfect Nash Equilibrium of the game was to collude in all periods unless your opponent had previously played competitively, and then to play the Cournot quantity forever Econ 400 (ND) Repeated Games 27 / 48

The Nash Threats Folk Theorem Theorem Consider any N-player infinitely repeated game with a stage game equilibrium s = (s 1,s 2,...,s N ) yielding payoffs u = (u 1,u 2,...,u N ). Suppose there is another strategy profile ŝ = (ŝ 1,ŝ 2,...,ŝ N ) yielding payoffs û = (û 1,û 2,...,û N ), where, for every player i, û i u i If the players are sufficiently patient, then there is a Subgame Perfect Nash Equilibrium in which the players use ŝ in every period of the infinitely repeated game. Econ 400 (ND) Repeated Games 28 / 48

The Folk Theorem Let s fill in the blanks for the prisoners dilemma game: Theorem Consider prisoners dilemma with a stage game equilibrium s = (C,C) yielding payoffs u = (0,0). Suppose there is another strategy profile ŝ = (S,S) yielding payoffs û = (1,1), where, for every player i, 1 0 If the players are sufficiently patient, then there is a Subgame Perfect Nash Equilibrium in which the players use (S,S) in every period of the infinitely repeated game. Econ 400 (ND) Repeated Games 29 / 48

The Folk Theorem: Trigger Strategies Consider the following trigger strategy for player i: If the history at t is h t = (ŝ,ŝ,...,ŝ), play ŝ i in period t. For any other history at time t, play s i in period t. This is called a trigger strategy because it starts in cooperative mode, but after any defection by any player, it switches to punishment or competitive mode, and they play the stage game strategies forever. Econ 400 (ND) Repeated Games 30 / 48

The Folk Theorem: Optimal Deviations Since û is presumably not a Nash equilibrium of the stage game, there are at least some players for whom u d j > û j u j i.e., while they prefer cooperating to the equilibrium of the stage game, they prefer defection to cooperation. Econ 400 (ND) Repeated Games 31 / 48

The Folk Theorem: Cooperating and Deviating The payoff to cooperating to player j is û j +δû j +δ 2 û j +... = 1 1 δûj Econ 400 (ND) Repeated Games 32 / 48

The Folk Theorem: Cooperating and Deviating The payoff to cooperating to player j is û j +δû j +δ 2 û j +... = 1 The payoff to deviating to player j is 1 δûj û d j +δu j +δu j +... = û d j +u j δ(1+δ +δ2 +...) = û d j +u j Then cooperating is better than deviating for player j if or 1 1 δ j û j û d j +u j δ j ud j û j u d j u j But 1 δ j, by the work on the previous slide. δ j 1 δ j δ 1 δ Econ 400 (ND) Repeated Games 32 / 48

The Folk Theorem: Equilibrium: Let δ = {δ 1,δ 2,...,δ N } so we have selected the highest discount factor for which cooperating is better than deviating, for all the players. Econ 400 (ND) Repeated Games 33 / 48

The Folk Theorem: Equilibrium: Let δ = {δ 1,δ 2,...,δ N } so we have selected the highest discount factor for which cooperating is better than deviating, for all the players. Then if all players are sufficiently patient, in the sense that each of their discount factors are greater than δ, then the trigger strategies are a subgame perfect Nash equilibrium of the infinitely repeated game, and they will play ŝ in every period. Econ 400 (ND) Repeated Games 33 / 48

The Folk Theorem: Geometry Econ 400 (ND) Repeated Games 34 / 48

Solving for Equilibria in Repeated Games 1. Solve for all equilibria of the stage game. (Competition) Econ 400 (ND) Repeated Games 35 / 48

Solving for Equilibria in Repeated Games 1. Solve for all equilibria of the stage game. (Competition) 2. Find a strategy profile (equilibrium or not) where all the players do at least as well as in the stage game. (Cooperation) 3. Design trigger strategies that support cooperation and punish with competition. 4. Find the optimal deviation for any player. Compute the minimum discount factor for which cooperating is an equilibrium. 5. Conclude that the trigger strategies are an equilibrium of the infinitely repeated game as long as all the players are sufficiently patient. Econ 400 (ND) Repeated Games 35 / 48

Trade and Bonuses/Tips Why do people tip for services or firms pay bonuses to workers? Econ 400 (ND) Repeated Games 36 / 48

Equilibria with Forgiveness The grim trigger strategies of the Nash Threats Folk Theorem are pretty harsh: Mess up once, and cooperation is cut off forever. What if punish by playing the stage game equilibrium K rounds and then return to cooperative mode, instead? Then for Prisoners Dilemma, cooperating is better than deviating if 1+δ +δ 2 +... 2+δ0+δ 2 0+...+δ K 0+δ K+1 +δ K+2 +... Econ 400 (ND) Repeated Games 37 / 48

Equilibria with Forgiveness Then cooperating is better than deviating if 2δ 1+δ K+1 or K log(2δ 1) log(δ) 1 Econ 400 (ND) Repeated Games 38 / 48

Equilibria with Forgiveness Then cooperating is better than deviating if or K 2δ 1+δ K+1 log(2δ 1) log(δ) If you compute the limit as δ 1 (use L Hopital s rule twice), you get that the minimal punishment period is 0. So players that are sufficiently patient will never cheat on each other. 1 Econ 400 (ND) Repeated Games 38 / 48

Collusion and Cournot Recall the Cournot game: Two firms simultaneously choose quantities q 1,q 2 > 0, where the market price is p = A q 1 q 2 and the firms have no costs. (Step 1) The Nash equilibrium of the stage game is q 1 = q 2 = A 3 giving profits ) 2 π 1 = π 2 = ( A 3 Econ 400 (ND) Repeated Games 39 / 48

Collusion and Cournot Summing the firms profits, we get π 1 +π 2 = (A q 1 q 2 )q 1 +(A q 1 q 2 )q 2 = (A q 1 q 2 )(q 1 +q 2 ) or Π = (A Q)Q Econ 400 (ND) Repeated Games 40 / 48

Collusion and Cournot Summing the firms profits, we get π 1 +π 2 = (A q 1 q 2 )q 1 +(A q 1 q 2 )q 2 = (A q 1 q 2 )(q 1 +q 2 ) or maximizing gives Π = (A Q)Q Q = A 2, Π = ( ) A 2 2 Since Π = A2 4 > 2A2 9 = π 1 +π 2, collusion is potentially profitable. Econ 400 (ND) Repeated Games 40 / 48

Collusion and Cournot (Step 2) Suppose the two firms are playing the Cournot game an infinite number of times, and they share a discount factor δ. Let ˆq = A 4, ˆπ = ( A 4 ) 2 This is half the monopoly quantity, A/2, and strictly less than the Cournot equilibrium quantity A/3. (Step 3) Consider the strategy: If the two firms have both used ˆq in all previous periods, use ˆq = A/4 this period. If either firm ever did anything besides ˆq, play the stage Cournot quantity q = A/3. Is this a subgame perfect Nash equilibrium of the infinitely repeated game? Econ 400 (ND) Repeated Games 41 / 48

Equilibrium Analysis: The Optimal Deviation To decide if this is an equilibrium, we need to know the payoff from optimally stabbing your partner in the back (if we re going to break up a relationship, we might as well do it optimally, right?). (Step 4) Then we need to solve max(a ˆq q )q = max (A A4 ) q q q q Maximizing gives q = 3A 8, π = ( ) 3A 2 8 Econ 400 (ND) Repeated Games 42 / 48

Equilibrium Analysis: Cooperation Cooperating in some period t after a history in which all the players previously cooperated keeps the industry in collusive mode (remember, given the proposed strategies), giving a payoff of ˆπ +δˆπ +δ 2ˆπ +... = ˆπ 1 1 δ which equals ( ) A 2 1 4 1 δ Econ 400 (ND) Repeated Games 43 / 48

Equilibrium Analysis: Deviation Deviating optimally in some period t after a history in which all the previously players cooperated switches the game from collusion to competition (remember, given the proposed strategies), giving a payoff of π +δπ +δ 2 π +... = ( ) 3A 2 +δ 8 ( ) A 2 ( 1+δ +δ 2 +... ) 3 Which is ( ) 3A 2 +δ 8 ( ) A 2 1 3 1 δ Econ 400 (ND) Repeated Games 44 / 48

Equilibrium Analysis Then cooperating is better than deviating if or or ( ) A 2 1 4 1 δ ( ) 3A 2 +δ 8 1 1 161 δ 9 16 + 1 δ 91 δ δ 17 45 ( ) A 2 1 3 1 δ So if δ 17 45, there are no profitable deviations as long as the players have previous cooperated. Econ 400 (ND) Repeated Games 45 / 48

Cournot and Collusion (Step 5) So if the players are sufficiently patient, or then the strategies δ 17 45 If the two firms have both used ˆq in all previous periods, use ˆq = A/4 this period. If either firm ever did anything besides ˆq, play the stage Cournot quantity q = A/3. are a subgame perfect Nash equilibrium of the infinitely repeated Cournot game. Econ 400 (ND) Repeated Games 46 / 48

Money as a long-lived institution Suppose each calendar date t, a generation of new people are born, and a previous generation die. Each generation lives for two periods. In the first period, they receive a wage of 4 during their working years, but receive no wage in period two of their lives. Their utility function over consumption pairs is u(c 1,c 2 ) = c 1 + c 2, where c 1 is the amount consumed in period 1 and c 2 is the amount consumed in period 2. Econ 400 (ND) Repeated Games 47 / 48

Money as a long-lived institution Suppose that there is a dollar that each generation hands to the next one. The dollar obligates the young to give the old y units of their wealth so that the old have consumption in old age. Suppose the players adopt the following strategies: If all previous generations have accepted the dollar and paid y to the old, do so this period. If any previous generation refused to pay, do not pay. Suppose the economy only lasts for T periods. For what values of y is it an equilibrium for the young to honor the dollar? Econ 400 (ND) Repeated Games 48 / 48