Prerequisites Almost essential Game Theory: Dynamic REPEATED GAMES MICROECONOMICS Principles and Analysis Frank Cowell April 2018 1
Overview Repeated Games Basic structure Embedding the game in context Equilibrium issues Applications April 2018 2
Introduction Another examination of the role of time Dynamic analysis can be difficult more than a few stages can lead to complicated analysis of equilibrium We need an alternative approach one that preserves basic insights of dynamic games for example, subgame-perfect equilibrium Build on the idea of dynamic games introduce a jump move from the case of comparatively few stages to the case of arbitrarily many April 2018 3
Repeated games The alternative approach take a series of the same game embed it within a time-line structure Basic idea is simple connect multiple instances of an atemporal game model a repeated encounter between the players in the same situation of economic conflict Raises important questions how does this structure differ from an atemporal model? how does the repetition of a game differ from a single play? how does it differ from a collection of unrelated games of identical structure with identical players? April 2018 4
History Why is the time-line different from a collection of unrelated games? The key is history consider history at any point on the timeline contains information about actual play information accumulated up to that point History can affect the nature of the game at any stage all players can know all the accumulated information strategies can be conditioned on this information History can play a role in the equilibrium some interesting outcomes aren t equilibria in a single encounter these may be equilibrium outcomes in the repeated game the game s history is used to support such outcomes April 2018 5
Repeated games: Structure The stage game take an instant in time specify a simultaneous-move game payoffs completely specified by actions within the game Repeat the stage game indefinitely there s an instance of the stage game at time 0,1,2,,t, the possible payoffs are also repeated for each t payoffs at t depends on actions in stage game at t A modified strategic environment all previous actions assumed as common knowledge so agents strategies can be conditioned on this information Modifies equilibrium behaviour and outcome? April 2018 6
Equilibrium Simplified structure has potential advantages whether significant depends on nature of stage game concern nature of equilibrium Possibilities for equilibrium new strategy combinations supportable as equilibria? long-term cooperative outcomes absent from a myopic analysis of a simple game Refinements of subgame perfection simplify the analysis: can rule out empty threats and incredible promises disregard irrelevant might-have-beens April 2018 7
Overview Repeated Games Basic structure Developing the basic concepts Equilibrium issues Applications April 2018 8
Equilibrium: an approach Focus on key question in repeated games: how can rational players use the information from history? need to address this to characterise equilibrium Illustrate a method in an argument by example outline for the Prisoner's Dilemma game same players face same outcomes from their actions that they may choose in periods 1, 2,, t, Prisoner's Dilemma particularly instructive given: its importance in microeconomics pessimistic outcome of an isolated round of the game April 2018 9
* detail on slide can only be seen if you run the slideshow Prisoner s dilemma: Reminder Payoffs in stage game If Alf plays [RIGHT] Bill s best response is [right] Alf [LEFT] [RIGHT] 2,2 3,0 0,3 1,1 If Bill plays [right] Alf s best response is [RIGHT] Nash Equilibrium Outcome that Pareto dominates NE [left] Bill [right] The highlighted NE is inefficient Could the Pareto-efficient outcome be an equilibrium in the repeated game? Look at the structure April 2018 10
* detail on slide can only be seen if you run the slideshow Repeated Prisoner's dilemma 1 [LEFT] Alf [RIGHT] Stage game between (t=1) Stage game (t=2) follows here or here or here Bill or here [left] [right] [left] [right] 2 2 [LEFT] Alf Alf Alf (2,2) (0,3) (3,0) (1,1) 2 2 Alf [LEFT] [RIGHT][LEFT] [RIGHT] [LEFT] [RIGHT] [RIGHT] Bill Bill Bill Bill [left] [right] [left] [left] [right] [left] [right] [left] [left] [right] [left] [right] [left] [right] (2,2) (2,2) (0,3) (2,2) (3,0) (0,3) (2,2) (3,0) (1,1) (0,3) (3,0) (0,3) (1,1) (3,0) (1,1) (1,1) Repeat this structure indefinitely? April 2018 11
Repeated Prisoner's dilemma 1 [LEFT] Alf [RIGHT] The stage game repeated though time Bill [left] [right] [left] [right] (2,2) (0,3) (3,0) (1,1) Alf t [LEFT] [RIGHT] Bill [left] [right] [left] [right] (2,2) (0,3) (3,0) (1,1) Let's look at the detail April 2018 12
Repeated PD: payoffs To represent possibilities in long run: first consider payoffs available in the stage game then those available through mixtures In the one-shot game payoffs simply represented it was enough to denote them as 0,,3 purely ordinal arbitrary monotonic changes of the payoffs have no effect Now we need a generalised notation cardinal values of utility matter we need to sum utilities, compare utility differences Evaluation of a payoff stream: suppose payoff to agent h in period t is υ h (t) value of (υ h (1), υ h (2),, υ h (t) ) is given by [1 δ] δ t 1 υ h (t) t=1 where δ is a discount factor 0 < δ < 1 April 2018 13
PD: stage game A generalised notation for the stage game consider actions and payoffs in each of four fundamental cases Both socially irresponsible: they play [RIGHT], [right] get ( υ a, υ b ) where υ a > 0, υ b > 0 Both socially responsible: they play [LEFT],[left] get (υ *a, υ *b ) where υ *a > υ a, υ *b > υ b Only Alf socially responsible: they play [LEFT], [right] get ( 0, υ b ) where υ b > υ *b Only Bill socially responsible: they play [RIGHT], [left] get ( υ a, 0) where υ a > υ *a A diagrammatic view April 2018 14
Repeated Prisoner s dilemma payoffs _ υ b υ b Space of utility payoffs Payoffs for Prisoner's Dilemma Nash-Equilibrium payoffs Payoffs Pareto-superior to NE Payoffs available through mixing Feasible, superior points "Efficient" outcomes UU * ( υ*a, υ *b ) ( υ a, υ b ) 0 _ υ a υ a April 2018 15
Choosing a strategy: setting Long-run advantage in the Pareto-efficient outcome payoffs (υ *a, υ *b ) in each period clearly better than ( υ a, υ b ) in each period Suppose the agents recognise the advantage what actions would guarantee them this? clearly they need to play [LEFT], [left] every period The problem is lack of trust: they cannot trust each other nor indeed themselves: Alf tempted to be antisocial and get payoff υ a by playing [RIGHT] Bill has a similar temptation April 2018 16
Choosing a strategy: formulation Will a dominated outcome still be inevitable? Suppose each player adopts a strategy that 1. rewards the other party's responsible behaviour by responding with the action [left] 2. punishes antisocial behaviour with the action [right], thus generating the minimax payoffs (υ a, υ b ) Known as a trigger strategy Why the strategy is powerful punishment applies to every period after the one where the antisocial action occurred if punishment invoked offender is minimaxed for ever Look at it in detail April 2018 17
Repeated PD: trigger strategies s T a Bill s action in 0,,t Alf s action at t+1 [left][left],,[left] [LEFT] Anything else [RIGHT] s T b Alf s action in 0,,t Bill s action at t+1 Take situation at t First type of history Response of other player to continue this history Second type of history Punishment response Trigger strategies [s Ta, s Tb ] [LEFT][LEFT],,[LEFT] Anything else [left] [right] Will it work? April 2018 18
Will the trigger strategy work? Utility gain from misbehaving at t: υ a υ *a What is value at t of punishment from t + 1 onwards? Difference in utility per period: υ *a υ a Discounted value of this in period t + 1: V := [υ *a υ a ]/[1 δ ] Value of this in period t: δv = δ[υ *a υ a ]/[1 δ ] So agent chooses not to misbehave if υ a υ *a δ[υ *a υ a ]/[1 δ ] But this is only going to work for specific parameters value of δ relative to υ a, υ a and υ *a What values of discount factor will allow an equilibrium? April 2018 19
Discounting and equilibrium For an equilibrium condition must be satisfied for both a and b Consider the situation of a Rearranging the condition from the previous slide: δ[υ *a υ a ] [1 δ] [ υ a υ *a ] δ[ υ a υ a ] [ υ a υ *a ] Simplifying this the condition must be δ δ a where δ a := [ υ a υ *a ] / [ υ a υ a ] A similar result must also apply to agent b Therefore we must have the condition: δ δ where δ := max {δ a, δ b } April 2018 20
Repeated PD: SPNE Assuming δ δ, take the strategies [s Ta, s Tb ] prescribed by the Table If there were antisocial behaviour at t consider subgame that would start at t + 1 Alf could not increase his payoff by switching from [RIGHT] to [LEFT], given that Bill is playing [left] a similar remark applies to Bill so strategies imply a NE for this subgame likewise for any subgame starting after t + 1 But if [LEFT],[left] has been played in every period up till t: Alf would not wish to switch to [RIGHT] a similar remark applies to Bill again we have a NE So, if δ is large enough, [s Ta, s Tb ] is a Subgame-Perfect Equilibrium yields the payoffs (υ *a, υ *b ) in every period April 2018 21
Folk Theorem The outcome of the repeated PD is instructive illustrates an important result the Folk Theorem Strictly speaking a class of results finite/infinite games different types of equilibrium concepts A standard version of the Theorem: for a two-person infinitely repeated game: suppose discount factor is sufficiently close to 1 any combination of actions observed in any finite number of stages this is the outcome of a subgame-perfect equilibrium April 2018 22
Assessment The Folk Theorem central to repeated games perhaps better described as Folk Theorems a class of results Clearly has considerable attraction Put its significance in context makes relatively modest claims gives a possibility result Only seen one example of the Folk Theorem let s apply it to well known oligopoly examples April 2018 23
Overview Repeated Games Basic structure Some well-known examples Equilibrium issues Applications April 2018 24
Cournot competition: repeated Start by reinterpreting PD as Cournot duopoly two identical firms firms can each choose one of two levels of output [high] or [low] can firms sustain a low-output (i.e. high-profit) equilibrium? Possible actions and outcomes in the stage game: [HIGH], [high]: both firms get Cournot-Nash payoff Π C > 0 [LOW], [low]: both firms get joint-profit maximising payoff Π J > Π C [HIGH], [low]: payoffs are ( Π, 0) where Π > Π J Folk theorem: get SPNE with payoffs (Π J, Π J ) if δ is large enough Critical value for the discount factor δ is Π Π J δ = Π Π C But we should say more Let s review the standard Cournot diagram April 2018 25
Cournot stage game q 2 q 2 χ 1 ( ) Firm 1 s Iso-profit curves Firm 2 s Iso-profit curves Firm 1 s reaction function Firm 2 s reaction function Cournot-Nash equilibrium Outputs with higher profits for both firms Joint profit-maximising solution Output that forces other firm s profit to 0 (q 1 C, q 2 C ) χ 2 ( ) (q1 J, q2 J ) 0 q 1 q 1 April 2018 26
Repeated Cournot game: Punishment Standard Cournot model is richer than simple PD: action space for PD stage game just has the two output levels continuum of output levels introduces further possibilities Minimax profit level for firm 1 in a Cournot duopoly is zero, not the NE outcome Π C arises where firm 2 sets output to q 2 such that 1 makes no profit Imagine a deviation by firm 1 at time t raises q 1 above joint profit-max level Would minimax be used as punishment from t + 1 to? clearly (0, q 2 ) is not on firm 2's reaction function so cannot be best response by firm 2 to an action by firm 1 so it cannot belong to the NE of the subgame everlasting minimax punishment is not credible in this case April 2018 27
Repeated Cournot game: Payoffs Π Π 2 Space of profits for the two firms Cournot-Nash outcome Joint-profit maximisation Minimax outcomes Payoffs available in repeated game (Π J,Π J ) (Π C,Π C ) 0 Π Π 1 Now review Bertrand competition April 2018 28
Bertrand stage game p 2 Marginal cost pricing Monopoly pricing Firm 1 s reaction function Firm 2 s reaction function Nash equilibrium p M c c p M p 1 April 2018 29
Bertrand competition: repeated NE of the stage game: set price equal to marginal cost c results in zero profits NE outcome is the minimax outcome minimax outcome is implementable as a Nash equilibrium in all the subgames following a defection from cooperation In repeated Bertrand competition firms set p M if acting cooperatively split profits between them if one firm deviates from this others then set price to c Repeated Bertrand: result can enforce joint profit maximisation through trigger strategy provided discount factor is large enough April 2018 30
Repeated Bertrand game: Payoffs Π M Π 2 Space of profits for the two firms Bertrand-Nash outcome Firm 1 as a monopoly Firm 2 as a monopoly Payoffs available in repeated game 0 Π M Π 1 April 2018 31
Repeated games: summary New concepts: Stage game History The Folk Theorem Trigger strategy What next? Games under uncertainty April 2018 32