The Effects of Nature on Learning in Games

Transcription

1 The Effecs of Naure on Learnng n Games C.-Y. Cynha Ln Lawell 1 Absrac Ths paper develops an agen-based model o nvesgae he effecs of Naure on learnng n games. In parcular, I exend one commonly used learnng model sochasc fcous play by addng a player o he game Naure whose play s random and non-sraegc, and whose underlyng exogenous sochasc process may be unknown o he oher players. Because he process governng Naure s play s unknown, players mus learn hs process, jus as hey mus learn he dsrbuon governng her opponen s play. Naure represens any random sae varable ha may affec players payoffs, ncludng global clmae change, weaher shocks, flucuaons n envronmenal condons, naural phenomena, changes o governmen polcy, echnologcal advancemen, and macroeconomc condons. Resuls show ha when Naure s Markov and he game beng played s a sac Courno duopoly game, neher play nor payoffs converge, bu hey each evenually ener respecve ergodc ses. JEL Classfcaon: C72, C73, C87, L13 Keywords: sochasc fcous play, learnng n games, common shock, Naure, agen-based model Ths draf: March Unversy of Calforna a Davs; ccln@prmal.ucdavs.edu. I am graeful o Adya Sunderam for excellen research asssance. Ths paper also benefed from dscussons wh Gary Chamberlan, Drew Fudenberg, Arel Pakes, Burkhard Schpper, Karl Schlag, and Sam Thompson, and from commens from parcpans a he workshop n mcroeconomc heory a Harvard Unversy and a he Second World Congress of he Game Theory Socey. I am a member of he Gannn Foundaon of Agrculural Economcs. All errors are my own.

2 1. Inroducon Alhough mos work n noncooperave game heory has radonally focused on equlbrum conceps such as Nash equlbrum and her refnemens such as perfecon, models of learnng n games are mporan for several reasons. The frs reason why learnng models are mporan s ha mere nrospecon s an nsuffcen explanaon for when and why one mgh expec he observed play n a game o correspond o an equlbrum. For example, expermenal sudes show ha human subjecs ofen do no play equlbrum sraeges he frs me hey play a game, nor does her play necessarly converge o he Nash equlbrum even afer repeaedly playng he same game (see e.g., Erev and Roh, 1998). In conras o radonal models of equlbrum, learnng models appear o be more conssen wh expermenal evdence (Fudenberg and Levne, 1999). These models, whch explan equlbrum as he long-run oucome of a process n whch less han fully raonal players grope for opmaly over me, are hus poenally more accurae depcons of acual real-world sraegc behavor. For example, n her analyss of a newly creaed marke for frequency response whn he UK elecrcy sysem, Doraszelsk, Lews and Pakes (2016) show ha models of fcous play and adapve learnng predc behavor beer han Nash equlbrum pror o convergence. By ncorporang exogenous common shocks, hs paper brngs hese learnng heores even closer o realy. In addon o beer explanng acual sraegc behavor, he second reason why learnng models are mporan s ha hey can be useful for smplfyng compuaons n emprcal work. Even f hey are played, equlbra can be dffcul o derve analycally and compuaonally n real-world games. For cases n whch he learnng dynamcs converge o an equlbrum, dervng he equlbrum from he learnng model may be compuaonally less burdensome han 2

3 aempng o solve for he equlbrum drecly. Indeed, he fcous play learnng model was frs nroduced as a mehod of compung Nash equlbra (Hofbauer and Sandholm, 2001). Pakes and McGure (2001) use a model of renforcemen learnng o reduce he compuaonal burden of calculang a sngle-agen value funcon n her algorhm for compung symmerc Markov perfec equlbra. Lee and Pakes (2009) examne how dfferen learnng algorhms selec ou equlbra when mulple equlbra are possble. As wll be explaned below, he work presened n hs paper furher enhances he applcably of hese learnng models o emprcal work. In hs paper, I exend one commonly used learnng model sochasc fcous play by addng a player o he game Naure whose play s random and non-sraegc, and whose underlyng exogenous sochasc process may be unknown o he oher players. Because he process governng Naure s play s unknown, players mus learn hs process, jus as hey mus learn he dsrbuon governng her opponen s play. Naure represens any random sae varable ha may affec players payoffs, ncludng global clmae change, weaher shocks, flucuaons n envronmenal condons, naural phenomena, changes o governmen polcy, echnologcal advancemen, and macroeconomc condons. By addng Naure o a model of sochasc fcous play, my work makes several conrbuons. Frs, ncorporang Naure brngs learnng models one sep closer o realsm. Sochasc fcous play was nroduced by Fudenberg and Kreps (1993), who exended he sandard deermnsc fcous model by allowng each player s payoffs o be perurbed each perod by..d. random shocks (Hofbauer and Sandholm, 2001). Asde from hese dosyncrac shocks, however, he game remans he same each perod. Indeed, o dae, much of he learnng 3

4 leraure has focused on repeons of he same game (Fudenberg and Levne, 1999). 2 My model exends he sochasc fcous play model one sep furher by addng a common shock ha may no be..d. over me, and whose dsrbuon and exsence may no necessarly be common knowledge. Because aggregae payoffs are sochasc, he game self s sochasc; players do no necessarly play he same game each perod. Thus, as n mos real-world suaons, payoffs are a funcon no only of he players sraeges and of ndvdual-specfc dosyncrac shocks, bu also of common exogenous facors as well. If he players were frms, for example, as wll be he case n he parcular model I evaluae, hese exogenous facors may represen global clmae change, weaher shocks, flucuaons n envronmenal condons, naural phenomena, changes o governmen polcy, echnologcal advancemen, macroeconomc flucuaons, or oher condons ha affec he profably of all frms n he marke. The second man conrbuon my work makes s ha provdes a framework for emprcal srucural esmaon. Alhough he use of srucural models o esmae equlbrum dynamcs has been popular n he feld of emprcal ndusral organzaon, lle work has been done eher esmang srucural models of non-equlbrum dynamcs or usng learnng models o reduce he compuaonal burden of esmang equlbrum models. One prmary reason learnng models have no been gnored by emprcal economss hus far s ha none of he models o dae have allowed for he possbly of sochasc sae varables and are hus oo unrealsc for esmaon. 3 My work removes hs obsacle o srucural esmaon. My research objecve s o nvesgae he effecs of addng Naure o he Courno duopoly on he sochasc fcous play dynamcs. In parcular, I analyze he effecs of Naure on: 2 Excepons are L Calz (1995) and Romaldo (1995), who develop models of learnng from smlar games. 3 Arel Pakes, personal communcaon. 4

5 () Trajecores: Wha do he rajecores for sraeges, assessmens, and payoffs look lke? () Convergence: Do he rajecores converge? Do hey converge o he Nash equlbrum? How long does convergence ake? () Welfare: How do payoffs from sochasc fcous play compare wh hose from he Nash equlbrum? When do players do beer? Worse? (v) Prors: How do he answers o ()-() vary when he prors are vared? Resuls show ha when Naure s frs-order Markov and he game beng played s a sac Courno duopoly game, he rajecores of play and payoffs are dsconnuous and can be vewed as an assemblage of he dynamcs ha arse when Naure evolves as several separae..d. processes, one for each of he possble values of Naure s prevous perod play. Neher play nor payoffs converge, bu hey each evenually ener respecve ergodc ses. The resuls of hs paper have mporan mplcaons for envronmenal and naural resource ssues such as global clmae change for whch here s uncerany abou he dsrbuon of possble oucomes, and abou whch agens such as ndvduals, frms, and/or polcy-makers seek o learn. The balance of hs paper proceeds as follows. I descrbe my model n Secon 2. I oulne my mehods and descrbe my agen-based model n Secon 3. In Secon 4, I analyze he Courno duopoly dynamcs n he benchmark case whou Naure. I analyze he resuls when Naure s added n Secon 5. Secon 6 concludes. 5

6 2. Model 2.1 Courno duopoly The game analyzed n hs paper s a sac homogeneous-good Courno duopoly. 4 I choose a Courno model because s one of he wdely used conceps n appled ndusral organzaon (Huck, Normann and Oeschssler, 1999); 5 I focus on wo frms only so ha he phase dagrams for he bes response dynamcs can be dsplayed graphcally. 6 In a one-sho Courno game, each player chooses a quany q o produce n order o maxmze her one-perod prof (or payoff) 7 1 ( q, qj) D ( q qj) q C( q) where D 1 () s he nverse marke demand funcon and C ( q ) s he cos o frm of producng q. Each frm s prof-maxmzaon problem yelds he bes-response funcon: BR( q ) argmax ( q, q ) j j q I assume ha he marke demand D() for he homogeneous good as a funcon of prce p s lnear and s gven by: D( p) a bp where a 0 and b 0. I assume ha he cos C ( ) o each frm of producng q s quadrac and s gven by: C ( q ) cq 2 4 Alhough he parcular game I analyze n hs paper s a Courno duopoly, my agen-based model can be used o analyze any sac normal-form wo-player game. 5 For expermenal sudes of learnng n Courno olgopoly, see Huck, Normann and Oeschssler (1999). 6 Alhough my agen-based model can only generae phase dagrams for wo-player games, can be easly modfed o generae oher graphcs for games wh more han wo players. 7 Throughou hs paper, I wll use he erms prof and payoff nerchangeably. Boh denoe smply he oneperod prof. 6

7 where c 0. Wh hese assumpons, he one-perod payoff o each player s gven by: a b q q j 2 ( q, qj) q cq, he bes-response funcon for each player s gven by: b ( a qj ) BR( q j) 2(1 cb) and he Nash equlbrum quany for each player s gven by: q a(1 2 cjb) 4(1 cb)(1 c b) 1 j. Throughou he smulaons, I se a = 20, b = 1. Wh hese parameers, he maxmum oal producon q, correspondng o p = 0, s q 20. The pure-sraegy space S for each player s hus he se of neger quanes from 0 o q 20. I examne wo cases n erms of cos funcons. In he symmerc case, I se c 1 = c 2 = 1/2; n he asymmerc case, he hgher-cos player 1 has c 1 = 4/3, whle he lower-cos player 2 has c 2 = 5/12. The Nash equlbrum quanes are hus q 1 = 5, q 2 = 5 n he symmerc case and q 1 = 3, q 2 = 6 n he asymmerc case. These correspond o payoffs of 1 = 37.5, 2 = 37.5 n he symmerc case and 1 = 21, 2 = 51 n he asymmerc case. The monopoly prof or, equvalenly, he maxmum jon prof ha could be acheved f he frms cooperaed, s m = 80 n he m symmerc case and =75.67 n he asymmerc case. 8 8 As a robusness check, I also run all he smulaons under an alernave se of cos parameers. The alernave se of parameers n he symmerc cos case are c 1 = c 2 = 0, whch yelds a Nash equlbrum quany of q = q = and a Nash equlbrum payoff of = 1 2 = The alernave se of parameers n he asymmerc cos case are c 1 = 0.5, c 2 = 0, whch yelds Nash equlbrum quanes of q = 4, q = 8 and Nash equlbrum payoffs of 1 2 7

8 2.2 Logsc smooh fcous play The one-sho Courno game descrbed above s played repeaedly and he players aemp o learn abou her opponens over me. The learnng model I mplemen s ha of sochasc fcous play. In fcous play, agens behave as f hey are facng a saonary bu unknown dsrbuon of her opponens sraeges; n sochasc fcous play, players randomze when hey are nearly ndfferen beween several choces (Fudenberg and Levne, 1999). The parcular sochasc play procedure I mplemen s ha of logsc smooh fcous play. Alhough he one-sho Courno game s played repeaedly, I assume, as s sandard n learnng models, ha curren play does no nfluence fuure play, and herefore gnore colluson and oher repeaed game consderaons. As a consequence, he players regard each perod- game as an ndependen one-sho Courno game. There are several possble sores for why mgh be reasonable o absrac from repeaed play consderaons n hs duopoly seng. One of-used jusfcaon s ha each perod here s an anonymous random machng of he frms from a large populaon of frms (Fudenberg and Levne, 1999). Ths machng process mgh represen, for example, random enry and/or ex behavor of frms. I mgh also depc a seres of one-me markes, such as aucons, he parcpans of whch dffer randomly marke by marke. A second possble sory s ha legal and regulaory facors may preclude colluson. For my model of logsc smooh fcous play, I use noaon smlar o ha used n Fudenberg and Levne (1999). As explaned above, he pure-sraegy space S for each player s = 24, 1 2 feaures. = 64. Excep where noed, he resuls across he wo ses of parameers have smlar qualave 8

9 he se of neger quanes from 0 o q 20. A pure sraegy q s hus an elemen of hs se: q S. The per-perod payoff o each player s smply he prof funcon ( q, qj). 9 A each perod, each player has an assessmen ( q ) of he probably ha hs opponen wll play q j. Ths assessmen s gven by where he wegh funcon ( q ) s gven by: j ( q ~ j ) ( q ~ j ) q, ( q ) q j 0 j j ( qj) 1 ( qj) + I{ qj, 1 qj} wh exogenous nal wegh funcon ( ): 0 q S j j. Thus, for all perods,, and 0 are all 1 q vecors. For my varous smulaons, I hold he lengh of he fcous hsory, q q 0 j ( q ), consan a q 1 21 and vary he dsrbuon of he nal weghs. j In logsc smooh fcous play, a each perod, gven her assessmen ( q ) of her opponen s play, each player chooses a mxed sraegy so as o maxmze her perurbed uly funcon: ~ U (, ) E [ ( q, q ), ] v ( ), q, q j j where v ( ) s an admssble perurbaon of he followng form: q v ( ) ( q ) ln ( q ). q 0 j 9 In Fudenberg and Levne (1999), a pure sraegy s denoed by s nsead of q, and payoffs are denoed by j u ( s, s ) nsead of ( q, q ). j 9

10 Wh hese funconal form assumpons, he bes-response dsrbuon exp(1/ ) E [ ( ~, ) ] )[ ~ q q q j BR ( q ]. q exp(1/ ) E [ ( q, q ) ] q 0 The mxed sraegy played by player a me s herefore gven by: BR ( ). q j BR s gven by The pure acon q acually played by player a me s drawn for player s mxed sraegy: q. ~ Because each of he sores of learnng n sac duopoly I oulned above sugges ha each frm only observes he play of s opponen and no he plays of oher frms of he opponen s ype n dencal and smulaneous markes, I assume ha each frm only observes he acual pure- sraegy acon q played by s one opponen and no he mxed sraegy from whch ha play was drawn. I choose he logsc model of sochasc fcous play because of s compuaonal smplcy and because corresponds o he log decson model wdely used n emprcal work (Fudenberg and Levne, 1999). For he smulaons, I se Addng Naure I now exend he sochasc fcous play model by addng a player o he game Naure whose play s random and non-sraegc, and whose underlyng exogenous sochasc process may be unknown o he oher players. Because he process governng Naure s play s unknown, players mus learn hs process, jus as hey mus learn he dsrbuon governng her opponen s play. Naure represens any random sae varable ha may affec players payoffs, 10

11 ncludng global clmae change, weaher shocks, flucuaons n envronmenal condons, naural phenomena, changes o governmen polcy, echnologcal advancemen, and macroeconomc condons. In parcular, I model Naure s play a me as a shock o he demand funcon such ha he acual demand funcon a me s: D( p) a bp. As a resul, each player s one-perod payoff s gven by: a q q q q q cq b b j 2 (, j, ). In he parcular model of Naure I consder, Naure behaves as a Markov process wh suppor N {-4, 0, +4} and wh a Markov ranson marx M gven by: Pr( ). j j 1 For example, f Naure represens global clmae change, he shocks may represen weaher shocks ha may vary year o year followng a frs-order Markov process. For my smulaons, I use he followng specfcaon for M: where Ths parcular Markov ranson marx creaes processes ha have a hgh degree of perssence, wh long sreaks of negave shocks and long sreaks of posve shocks. Shocks 11

12 of value zero are rare and ransory: anyme he shock akes on value zero, s lkely o become eher posve or negave. 10 In he parcular model of players belef abou Naure ha I consder, players are aware ha Naure s Markov and hey know he suppor of Naure s dsrbuon, bu hey do no know he value of Naure s ranson marx. Each player begns wh he pror belef ha each Markov ranson s equally lkely. To learn he dsrbuon of Naure s play, frms updae her assessmens for each ranson probably s a manner analogous o ha n fcous play. Because he uncondonal dsrbuon of Naure has mean zero, I can compare he quanes and payoffs arsng from sochasc fcous play n he presence of Naure wh he quanes and payoffs ha would arse n Nash equlbrum n he absence of Naure. 3. Mehods To analyze he dynamcs of logsc smooh fcous play, I develop an agen-based model ha enables one o analyze he followng. 10 I also have prelmnary resuls for wo oher models of Naure as well (no shown). In he smples model, Naure s shock n each perod s ndependen and dencally dsrbued (IID) wh mean zero and dscree suppor. In parcular, s drawn randomly from he se N {-4, 0, +4}, where each elemen of N has an equal probably of beng drawn. In he oher model of Naure, Naure behaves as an auoregressve process of order 1 (AR(1)). Thus, I defne u 1 where 0 < < 1 and u s normally dsrbued wh mean 0 and sandard devaon 2. 0 s drawn from he uncondonal margnal dsrbuon of : a normal dsrbuon wh mean 0 and sandard devaon 2 /(1-2 ). Ths case s smlar o he Markov model because agan s dependen on -1. However, he auoregressve model dffers from he prevous wo snce here Naure s shocks ake on connuous, raher han dscree, values. I hope o fully analyze hese wo addonal models of Naure n fuure work. Moreover, n fuure work, n addon o allowng Naure o evolve n hree dfferen ways (IID, Markov and AR(1)), one can model players belefs abou Naure n fve dfferen ways. Players can beleve ha Naure behaves as an IID, Markov, or auoregressve process, or hey can be gnoran of he presence of Naure alogeher. I also consder he case n whch players have asymmerc belefs abou Naure. 12

13 () Trajecores For each player, I examne he rajecores over me for he mxed sraeges chosen, he acual pure acons q played and payoffs acheved. I also examne, for each player, he rajecores for he per-perod mean quany of each player s mxed sraegy: [ q ] (1) as well as he rajecores for he per-perod mean quany of hs opponen s assessmen of hs sraegy: [ q j ]. (2) I also examne hree measures of he players payoffs. 11 Frs, I examne he ex ane payoffs, whch I defne o be he payoffs a player expecs o acheve before her pure-sraegy acon has been drawn from her mxed sraegy dsrbuon:, [ ( q, q ), ]. (3) q qj j The second form of payoffs are he nerm payoffs, whch I defne o be he payoffs a player expecs o acheve afer she knows whch parcular pure-sraegy acon q has been drawn from her mxed sraegy dsrbuon, bu before her opponen has played: [ ( q, q ) ] (4) qj j The hrd measure of payoffs I analyze s he acual realzed payoff ( q, q ). j 11 I examne he payoffs (or, equvalenly, profs) nsead of he perurbed uly so ha I can compare he payoff from sochasc fcous play wh he payoffs from equlbrum play. 13

14 () Convergence The merc I use o examne convergence s he Eucldean norm d(). Usng he noon of a Cauchy sequence and he resul ha n fne-dmensonal Eucldean space, every Cauchy sequence converges (Rudn, 1976), I say ha a vecor-valued rajecory {X } has converged a me f for all mn, he Eucldean dsance beween s value a perods m and n, d X m, X ) ( n, falls below some hreshold value d. In pracce, I se d 0.01 and requre ha d( X, X ) d m, n [, T ], where T=1000. I examne he convergence of wo rajecores: he mxed sraeges{ } and ex ane payoffs { q, q[ ( q, qj), ]}. j In addon o analyzng wheher or no eher he mxed sraeges or he ex ane payoffs converge, I also examne wheher or no hey converge o he Nash equlbrum sraegy and payoffs, respecvely. I say ha a vecor-valued rajecory {X } has converged o he Nash equlbrum a me f he Eucldean dsance beween s value a and ha of he Nash equlbrum analog, d( X, X ), falls below some hreshold value d for all perods afer. In pracce, I se d. 01 and requre ha d( X, X ) d [, T], where T=1000. m n () Welfare The resuls above are compared o non-cooperave Nash equlbrum as well as he cooperave oucome ha would arse f he frms aced o maxmze jon profs. The cooperave oucome corresponds o he monopoly oucome. 14

15 (v) Prors Fnally, I examne he effec of varyng boh he mean and spread of players prors 0, he above resuls. These prors reflec he nal belefs each player has abou hs opponen pror o he sar of play. The agen-based model I develop for analyzng he dynamcs of logsc smooh fcous play can be used for several mporan purposes. Frs, hs agen-based model enables one o confrm and vsualze exsng analyc resuls. For example, for classes of games for whch convergence resuls have already been proven, my agen-based model enables one no only o confrm he convergence, bu also o vsualze he ransonal dynamcs. I demonsrae such a use of he agen-based model n my analyss of he benchmark case of sochasc fcous play n he absence of Naure. A second way n whch my agen-based model can be used s o generae deas for fuure analyc proofs. Paerns gleaned from compuer smulaons can sugges resuls ha mgh hen be proven analycally. For example, one canddae for an analyc proof s he resul ha, when coss are asymmerc and prors are unformly weghed, he hgher-cos player does beer under sochasc fcous play han she would under he Nash equlbrum. Anoher canddae s he resul s wha I erm he overconfdence premum: he worse off a player nally expecs her opponen o be, he beer off she herself wll evenually be. A hrd way n whch of my agen-based model can be used s o analyze games for whch analyc soluons are dffcul o derve. For example, an analyss of he effecs of addng Naure s more easly done numercally raher han analycally. 15

16 A fourh poenal use for my agen-based model s pedagogcal. The agen-based model can supplemen sandard exs and papers as a learnng or eachng ool n any course coverng learnng dynamcs and sochasc fcous play. I apply he agen-based model o analyze he sochasc fcous play dynamcs of he Courno duopoly game boh n he absence of Naure and n he presence of Markov Naure. 12 Alhough he enre agen-based model was run for wo ses of parameers, I presen he resuls from only one. Unless oherwse ndcaed, qualave resuls are robus across he wo ses of parameers. 4. Benchmark case: No Naure Before addng Naure, I frs analyze he sochasc fcous play dynamcs of he Courno duopoly game n he absence of Naure. I do so for wo reasons. Frs, resuls from he no Naure case provde a benchmark agans whch I can compare my resuls wh Naure. Second, snce my Courno duopoly game wh lnear demand falls no a class of games for whch heorems abou convergence have already been proven, 13 a presenaon of my resuls enables one no only o confrm he prevous proven analyc resuls, bu also o assess how my numercal resuls may provde addonal nformaon and nuon prevously naccessble o analyc analyss alone. 12 In fuure work I hope o examne several dfferen scenaros, each correspondng o a dfferen specfcaon of Naure (e.g., no Naure, IID Naure, Markov Naure, AR(1) Naure) and o a dfferen specfcaon of players belefs abou Naure (e.g., no Naure, IID Naure, Markov Naure, AR(1) Naure, asymmerc belefs). 13 More specfcally, because my game s a 2X2 game ha has a unque src Nash equlbrum, he unque nersecon of he smoohed bes response funcons s a global aracor (Fudenberg and Levne, 1999). Leon (2008) exends he convergence resul of Kala and Lehrer (1993a, 1993b) o a class of games n whch players have a payoff funcon connuous for he produc opology, and consruc a Nash equlbrum such ha, under ceran condons and afer some fne me, he equlbrum oucome of learnng n repeaed games s arbrarly close o he consruced Nash equlbrum. Arel and Young (2016) provde explc bounds on he speed of convergence for he general case of weakly acyclc games wh global neracon. 16

17 Frs, I presen resuls ha arse when each player nally beleves ha he oher plays each possble pure sraegy wh equal probably. In hs case, each player s pror pus unform wegh on all he possble pure sraeges: 0 =(1,1,, 1). I call hs form of pror a unformly weghed pror. When a player has a unformly weghed pror, he wll expec hs opponen o produce quany 10 on average, whch s hgher han he symmerc Nash equlbrum quany of 1 q = q 2 = 5 n he symmerc cos case and also hgher han boh quanes q 1 = 3, q 2 = 6 ha arse n he Nash equlbrum of he asymmerc cos case. Fgure 1.1 presens he rajecores of each player s mxed sraegy over me when each player has a unformly weghed pror. Each color n he fgure represens a pure sraegy (quany) and he hegh of he band represens he probably of playng ha sraegy. As expeced, n he symmerc case, he players end up playng dencal mxed sraeges. In he asymmerc case, player 1, whose coss are hgher, produces smaller quanes han player 2. In boh cases he players converge o a fxed mxed sraegy, wh mos of he change occurrng n he frs 100 me seps. I seems ha convergence akes longer n he case of asymmerc coss han n he case of symmerc coss. Noe ha he sraeges ha evenually domnae each player s mxed sraegy nally have very low probables. The explanaon for hs s ha wh unformly weghed prors, each player s grossly overesmang how much he oher wll produce. Each player expecs he oher o produce quanes beween 0 and 20 wh equal probables, and hus has a mean pror of quany 10. As a consequence, each frm nally produces much less he Nash equlbrum quany o avod floodng he marke. In subsequen perods, he players wll updae her assessmens wh hese lower quanes and change her sraeges accordngly. 17

18 (a) Fgure 1.1. Dynamcs of players mxed sraeges wh (a) symmerc and asymmerc coss as a funcon of me n he absence of Naure. As a benchmark, he Nash equlbrum quanes are q (5,5) n he symmerc cos case and q (3,6) n he asymmerc cos case. Each player has a unformly weghed pror. Fgure 1.2 presens he rajecores for he acual payoffs acheved by each player a each me perod. Once agan, I assume ha each player has a unformly weghed pror. The large varaon from perod o perod s a resul of players randomly selecng one sraegy o play from her mxed sraegy vecors. In he symmerc case, each player s per-perod payoff hovers close o he symmerc Nash equlbrum payoff of = On average, however, boh players do slghly worse han he Nash equlbrum, boh averagng payoffs of 37.3 (s.d. = 2.96 for player 1 and s.d. = 2.87 for player 2). 14 In he asymmerc case, he vecor of players per-perod payoffs s once agan close o he Nash equlbrum payoff vecor = (21, 51). However, player 1 slghly ouperforms her Nash equlbrum, averagng a payoff of (s.d. = 2.16), whle player 2 underperforms, averagng a payoff of (s.d. = 2.59). Thus, when coss 14 The average and sandard devaon for he payoffs are calculaed as follows: means and sandard devaons are frs aken for all T=1000 me perods for one smulaon, and hen he values of he means and sandard devaons are averaged over 20 smulaons. 18

19 are asymmerc, he hgh-cos frm does beer on average under logsc smooh fcous play han n he Nash equlbrum, whle he low-cos frm does worse on average. 15 (a) Fgure 1.2. Acual payoffs acheved by each player as a funcon of me n he (a) symmerc and asymmerc cases n he absence of Naure. Each player has a unformly weghed pror. Much of he varaon n he acheved payoff arses from he fac a each me, each player randomly selecs one sraegy q o play from hs me- mxed sraegy vecor. By akng he mean over hese vecors a each me, I can elmnae hs varaon and gan a clearer pcure of he dynamcs of each player s sraegy. Fgure 1.3 presens he evoluon of he expeced per-perod quanes, where expecaons are aken a each me eher over players mxed sraeges or over opponens assessmens a me, values correspondng o expressons (1) and (2), respecvely. As before, each player has a unformly weghed pror. Fgures 1.3(a) 1 and 1.3 presen he boh mean of player 1 s mxed sraegy (.e., [ q ]) and he mean of player 2 s assessmen of wha player 1 wll play (.e., [ q j ]) for he symmerc- and 1 15 Ths qualave resul s robus across he wo ses of cos parameers I analyzed. 19

20 asymmerc-cos cases, respecvely. Fgure 1.3(c) gves he mean of player 2 s mxed sraegy and he mean of player 1 s assessmen of player 2 n he asymmerc case. For boh he symmerc and asymmerc cos cases, he mean of player 2 s assessmen s nally very hgh and asympocally approaches he Nash equlbrum. As explaned above, hs s a resul of he unformly weghed pror. Inally, player 2 expecs player 1 o play an average sraegy of 10. Smlarly, player 1 expecs player 2 o play an average sraegy of 10, and consequenly player 1 s mxed sraegy nally has a very low mean, whch rses asympocally o he Nash equlbrum. I s neresng o noe ha n he asymmerc case, he mean over player 1 s chosen mxed sraegy slghly overshoos he Nash equlbrum and hen rends back down owards. Fgure 1.3 also provdes sandard devaons over player 1 s mxed sraegy and player 2 s assessmen. Noe ha he sandard devaon of player 2 s assessmen s much hgher han he sandard devaon of player 1 s mxed sraegy, ndcang player 2 s relave uncerany abou wha player 1 s dong. 16 (a) 16 Alhough he resuls presened n hese fgures are he oucome of one parcular smulaon, n general he varaon n he values for he expeced quanes across smulaons s small. 20

21 (c) Fgure 1.3. Means and varances of quanes, as aken over players me- mxed sraeges and opponen s me- assessmens n he absence of Naure n he (a) symmerc case and he asymmerc case for he hgher-cos player 1 and (c) he lower-cos player 2 as a funcon of me. Each player has a unformly weghed pror. Jus as an examnaon of he expeced per-perod quany nsead of he mxed sraegy vecor can elucdae some of he dynamcs underlyng play, analyzng expeced payoffs can smlarly elmnae some of he varaon presen n he rajecores of players acheved payoffs n Fgure 1.2. Fgure 1.4 presens he evoluons of players ex ane and nerm expeced payoffs, correspondng o expressons (3) and (4), respecvely. Fgures 1.4(a) and 1.4 depc hese quanes for player 1. The nerm payoff has a large varance from perod o perod because s calculaed afer player 1 has randomly seleced a sraegy from hs mxed sraegy. In he symmerc case, depced n Fgure 1.4(a), boh he ex ane and nerm expeced payoffs asympoe o he Nash equlbrum payoff, bu reman slghly below. In he asymmerc cos case, he hgh-cos player 1 evenually does beer han she would have n Nash equlbrum whle he low-cos player 2 evenually acheves approxmaely hs Nash equlbrum payoff. 17 For all cases, on average, he nerm expeced payoff s below he ex ane expeced payoff. Fgure 1.4 also presens sandard devaons for he ex ane and nerm expeced payoffs; n 17 In he alernave se of cos parameers I red, he hgh-cos player 1 evenually acheves approxmaely her Nash equlbrum payoff n he long run whle he low-cos player 2 does worse han hs Nash equlbrum payoff. 21

22 general hey seem roughly equal. 18 Thus, whle players n he symmerc cos case do slghly worse han n Nash equlbrum n he long run, he hgh-cos player 1 n he asymmerc cos case does beer n he long run under sochasc fcous play han she would n Nash equlbrum. (a) (c) Fgure 1.4. Means and varances of ex ane and nerm payoff n he absence of Naure n he (a) symmerc case and he asymmerc case for he hgher-cos player 1 and (c) he lower-cos player 2 as a funcon of me. Each player has a unformly weghed pror. Havng shown ha expeced quany and expeced payoff seem o converge o he Nash equlbrum, I now es wheher hs s ndeed he case. Frs, I examne wheher or no he mxed 18 As before, alhough he resuls presened n hese fgures are he oucome of one parcular smulaon, n general he varaon n he values for he ex ane and nerm payoffs across smulaons s small. 22

23 sraeges do converge and he speed a whch hey converge. Fgure 1.5 gves a measure of he convergence of smooh fcous play when prors are unformly weghed. As explaned above, I defne how close o seady-sae player s a me as he maxmum Eucldean dsance beween player s mxed sraegy vecor a mes m, n. Indeed, he mxed sraeges do converge: he Eucldean dsance asympoes o zero. In he symmerc case, Fgure 1.5(a), boh players converge a approxmaely he same rae. In he asymmerc case, Fgure 1.5, he player wh hgher coss, player 1, appears o converge more quckly. Now ha I have esablshed ha he mxed sraeges do ndeed converge, he nex queson I hope o answer s wheher hey converge o he Nash equlbrum. Fgures 1.5(c) and 1.5(d) depc he Eucldean dsance beween player s mxed sraegy vecor and he Nash equlbrum. In he symmerc case, boh players converge a abou he same rae, bu neher ges very close o he Nash equlbrum. In he asymmerc case, player 1 agan sablzes more quckly. Furhermore, player 1 comes much closer o he Nash equlbrum han player 2 does. Wh unformly-weghed prors, s never he case ha d( X, X ) d, where d 0.01; hus neher player converges o he Nash equlbrum. A me T=1000, he dsance o he Nash equlbrum s 0.21 n he symmerc case. In he asymmerc case, he dsance o Nash equlbrum s 0.39 for he hgher cos player and 0.53 for he lower cos player. 23

24 (a) (c) (d) Fgure 1.5. Maxmum Eucldean dsance beween player s mxed sraegy vecor n perods m, n n he absence of Naure n he (a) symmerc and asymmerc cases as a funcon of me. Dsance beween player s mxed sraegy vecor and he Nash equlbrum n he (c) symmerc and asymmerc cases. Each player has a unformly weghed pror. Because he players pror belefs are responsble for much of he behavor observed n he early rounds of play, I now examne how he mean and he spread of he prors affec he convergence properes. Frs, I examne how my resuls may change f nsead of a unformly weghed pror, each player s had a pror ha concenraed all he wegh on a sngle sraegy: 0 = (0, 0,, 21, 0, 0,, 0). I call such a pror a concenraed pror. Fgure 1.6 repeas he analyses n Fgure 1.5, bu wh concenraed prors ha place all he wegh on quany 9. The fgures show ha n boh he symmerc and asymmerc cases, he 24

25 form of he pror affecs he speed of convergence bu no s asympoc behavor. Even wh concenraed prors, each player s play sll converges o a seady sae mxed sraegy vecor. Wh concenraed prors, jus as wh unformly weghed prors, he dsance o he Nash equlbrum converges o 0.21 n he symmerc case and 0.39 and 0.53 n he asymmerc case. (a) (c) (d) Fgure 1.6. Maxmum Eucldean dsance beween player s mxed sraegy vecor n perods m, n n he absence of Naure n he (a) symmerc and asymmerc cases as a funcon of me. Dsance beween player s mxed sraegy vecor and he Nash equlbrum n he (c) symmerc and asymmerc cases. Each player has a concenraed pror ha places all he wegh on he sraegy q j = 9. 25

26 I now examne he effec of varyng he means of he concenraed prors on he mean quany [ q ] of each player s mxed sraegy. For each player, I allow he sraegy wh he enre wegh of 21 o be eher 4, 8, 12, or 16. Thus, I have 16 dfferen combnaons of nal prors. The phase porras n Fgure 1.7 are produced as follows. For each of hese combnaons of prors, I calculae each player s expeced quany over her mxed sraeges and plo hs as an ordered par for each me. Each rajecory hus corresponds o a 1 [ q1 ] dfferen specfcaon of he prors, and dsplays he evoluon of he mxed sraegy over T=1000 perods. The fgure shows ha n boh cases, no maer wha he pror, he players converge o a pon close o he Nash equlbrum. In fac, he endpons, correspondng o T=1000, appear o fall on a lne. I s also neresng o noe ha many of he rajecores are no sragh lnes, ndcang ha players are no akng he mos drec roue o her endpons. Noce ha n he asymmerc case player 2 s quany never ges very far above her Nash equlbrum quany. (a) Fgure 1.7. Phase porras of expeced quany show he effec of varyng (concenraed) prors n he absence of Naure n he (a) symmerc and asymmerc cases. 26

27 Whle Fgure 1.7 shows phase porras of expeced quany, Fgure 1.8 shows phase porras of ex ane expeced payoff. For comparson, he payoffs from he Nash and cooperave equlbra are ploed as benchmarks. Once agan, no maer he nal pror, he payoffs converge close o he Nash equlbrum payoffs n boh cases. Agan, he endpons, correspondng o T=1000, appear o fall on a lne. In hs case, however, he Nash equlbrum appears o be slghly above he lne. Thus, n he seady-sae oucome of logsc smooh fcous play, he players are worse off han hey would be n a Nash equlbrum. (a) Fgure 1.8. Phase porras of ex ane expeced payoff n he absence of Naure n he (a) symmerc and asymmerc cases. As noed above, he fnal pons of he rajecores of expeced quany shown n Fgure 1.7 seem o form a lne, as do he fnal pons of he rajecores of ex aned expeced payoffs n Fgure 1.8. Fgure 1.9 shows only hese fnal pons and her bes-f lne for boh he expeced quany and for he ex ane expeced payoff. As seen n Fgure 1.6, each of he fnal pons represens he long-run seady sae reached by he players. 27

28 Several feaures of he resuls n Fgure 1.9 should be noed. The frs feaure s he lnear paern of he fnal pons. In he symmerc case, he slope of he bes-f lne, whch les below he Nash equlbrum, s approxmaely (s.e. = 3e-6). Thus, varyng he pror appears only o affec he dsrbuon of producon beween he wo frms, bu no he oal expeced quany produced, and hs oal expeced quany s weakly less han ha whch arses n he Nash equlbrum. In he asymmerc case, he slope of he lne, whch agan les below he Nash equlbrum, s (s.e. = 3e-6). Thus, a weghed sum of he expeced quanes, where he hgher cos player 1 s gven a greaer wegh, s relavely consan across dfferen prors. Smlar saemens can be made abou he payoffs as well: ha s, he sum of he payoffs s robus o he prors bu lower han he sum of he Nash equlbrum payoffs n he symmerc cos case, and a weghed sum of he payoffs s robus o he prors bu lower han he weghed sum of he Nash equlbrum payoffs n he asymmerc cos case. A second feaure of Fgure 1.9 o noe regards how each player performs relave o hs Nash equlbrum across dfferen prors. In he symmerc case, he fnal pons are dsrbued farly evenly abou he Nash equlbrum along he bes-f lne. Ths mples ha he number of prors for whch player 1 does beer han he Nash equlbrum s approxmaely equal o he number of prors for whch player 2 does beer han he Nash equlbrum. In he asymmerc case, on he oher hand, mos of he endpons le below he Nash equlbrum, mplyng ha he number of prors for whch player 1 does beer han he Nash equlbrum s larger han he number of prors for whch player 2 does beer han he Nash equlbrum. Ths seems o confrm he earler observaon ha he hgher cos player usually ouperforms her Nash equlbrum n he asymmerc case. 28

29 A hrd mporan feaure of Fgure 1.9 regards convergence. Noe ha Fgures 1.9(a) and show ha here are several combnaons of prors ( (4,4), (8,8), (12, 12), and (16, 16) n he symmerc case, and (8,4), (12, 8) and (16,12) n he asymmerc case) ha lead o seady sae expeced quanes very close o he Nash equlbrum (whn a Eucldean dsance of 0.01). However, convergence as earler defned requres ha he players mxed sraegy vecors, no he expecaons over hese vecors, come whn a Eucldean dsance of 0.01 of he Nash equlbrum. Ths does no happen for any se of prors; under no combnaon of prors do he players mxed sraegy vecors converge o he Nash equlbrum. A fourh mporan feaure of Fgure 1.9 regards he effecs of a player s pror on hs long-run quany and payoff. Accordng o Fgure 1.9, when he opponen s pror s held fxed, he lower he pror a player has over her opponen (.e., he less she expecs he oher o produce), he more she wll produce and he hgher her per-perod prof n he long run. There hus appears o be wha I erm he overconfdence premum: he worse off a player nally expecs her opponen o be, he beer off she herself wll evenually be. (a) 29

30 (c) (d) Fgure 1.9. Bes f lnes of he endpons of he rajecores of expeced quany shown n Fgure 1.7 n he absence of Naure n he (a) symmerc and asymmerc cases. Plos (c) and (d) gve smlar bes f lnes for he rajecores of ex ane expeced payoff shown n Fgure 1.8. Havng seen he effec of varyng he mean of each player s pror on he learnng dynamcs, I now fx he mean and vary he spread. Fgure 1.10 shows he effec of spread n he pror. I fx player 2 s pror, wh all wegh on one sraegy ( q 1 = 10). 19 Thus, player 2 s pror 2 looks lke 0 = (0,, 0, 21, 0,, 0). I vary player 1 s pror, keepng s mean he same (also producng a quany of 10), bu spreadng s wegh over 1, 3, or 7 sraeges. Thus, player 1 s 1 pror looks lke one of 0 = (0,, 0, 21, 0,, 0), (0,, 0, 7, 7, 7, 0, 0), or (0,, 0, 3, 3, 3, 3, 3, 3, 3, 0,, 0). As I see n Fgure 1.10, spreadng he pror n hs manner does affec he rajecory of expeced quanes bu does no aler he nal or fnal pons n eher he symmerc or asymmerc case. Each rajecory has he exac same sarng pon, whle her fnal pons vary jus slghly. Ths varaon decreases as he number of me seps ncreases. The same resul arses f I plo phase porras of he ex ane payoffs. 19 I chose o concenrae he pror on he mean pure sraegy n he sraegy se boh because dd no correspond o any Nash equlbrum, hus ensurng ha he resuls would be non-rval, and also so ha varyng he spread would be sraghforward. 30

31 (a) (c) (d) Fgure The effecs of varyng he spread of player 1 s pror around he same mean ( q 2 = 10) n he absence of Naure on he rajecores of expeced quany n he (a) symmerc and asymmerc cases, and on he rajecores of ex ane payoff n he (c) symmerc and (d) asymmerc cases. I nex examne he effec of varyng each player s pror on he rae of convergence. Fgure 1.11 shows he effec of dfferen prors on he speed of convergence of he mxed sraegy for player 1. I hold player 2 s pror fxed, wh all wegh on player 1 s Nash equlbrum sraegy (.e., q 1 = q 1 = 5 for he symmerc cos case and q 1 = q 1 = 3 for he asymmerc cos case). I hen vary player 1 s pror, keepng all wegh on one sraegy, bu 31

32 varyng ha sraegy beween 2, 4, 6, 8, 10, 12, 14, 16, and 18. Player 2 s Nash equlbrum sraegy s ndcaed by he vercal dashed lne. 20 In boh cases, he me o convergence s mnmzed when player 1 s pror pus all he wegh on q 2 = 6. Ths s no surprsng n he asymmerc case, Fgure 1.9, because q = (3, 6) s he Nash equlbrum for ha case. In he symmerc case, when player 1 s pror pus all he wegh on q 2 = 6, hs s very close o player 2 s Nash equlbrum quany of q 2 = 5. (a) Fgure The number of me seps unl convergence n he absence of Naure n he (a) symmerc and asymmerc cases. Player 1 has a concenraed pror. Player 2 s Nash equlbrum sraegy s ndcaed by he dashed lne. I now examne he effec of varyng each player s pror on convergence o he Nash equlbrum. Fgure 1.12 shows he effec of dfferen prors on he fnal dsance o he Nash equlbrum. Agan, I hold player 2 s pror fxed wh all wegh on player 1 s Nash equlbrum sraegy. I hen vary player 1 s pror as before. Player 2 s Nash equlbrum sraegy s agan 20 For he asymmerc cos case, one can also generae an analogous plo as a funcon of player 2 s pror, holdng player 1 s pror consan a player 2 s Nash equlbrum sraegy. 32

33 shown by a doed vercal lne. 21 The dsance beween player 1 s mxed sraegy vecor and hs Nash equlbrum quany a me T=1000 s smalles when player 1 s pror s concenraed a a value close o player 2 s Nash equlbrum quany. The dsance grows as player 1 s pror ges furher away from he Nash equlbrum quany. (a) Fgure Dsance beween player 1 s mxed sraegy vecor and he Nash equlbrum a me T=1000 as a funcon of player 1 s (concenraed) pror n he absence of Naure n he (a) symmerc and asymmerc cases. Player 2 s Nash equlbrum sraegy s ndcaed by he dashed lne. Fnally, I examne he effec of varyng each player s pror on fnal-perod ex ane payoff, as compared o he Nash equlbrum. Fgure 1.13 shows he effec of dfferen prors on he fnal ex ane payoff mnus he Nash equlbrum payoff. Agan, I hold player 2 s pror fxed wh all wegh on player 1 s Nash equlbrum sraegy. I hen vary player 1 s pror as before. Player 2 s Nash equlbrum sraegy s ndcaed by a doed vercal lne. 22 The dfference beween player 1 s ex ane payoff and he Nash equlbrum payoff a me T=1000 s larges when player 1 s pror s smalles. The dfference declnes (and becomes negave) as player 1 s pror grows. 21 For he asymmerc cos case, one can also generae an analogous plo as a funcon of player 2 s pror, holdng player 1 s pror consan a player 2 s Nash equlbrum sraegy. 22 For he asymmerc cos case, one can also generae an analogous plo as a funcon of player 2 s pror, holdng player 1 s pror consan a player 2 s Nash equlbrum sraegy. 33

34 When player 1 s pror s smalles, he beleves ha player 2 wll produce a small quany. Thus, he wll produce a large quany, and reap he benefs of a larger payoff. Ths resul confrms he overconfdence premum resuls from Fgure 1.9: he worse off a player nally expecs hs opponen o be, he beer off he hmself wll evenually be. (a) Fgure Dfference beween player 1 s fnal-perod ex ane payoff and he Nash equlbrum payoff a me T=1000 n he absence of Naure as a funcon of player 1 s concenraed pror n he (a) symmerc and asymmerc cases. Player 2 s Nash equlbrum sraegy s ndcaed by he dashed lne. In summary, he man resuls arsng from my examnaon of he no Naure case are: 23 1) In he symmerc case wh unformly weghed prors, players on average acheve a slghly smaller payoff han he Nash equlbrum payoff, boh on average and n he long run. 23 These qualave resuls are robus across he wo ses of cos parameers analyzed. 34

35 2) In he asymmerc case wh unformly weghed prors, he hgher cos player ouperforms her Nash equlbrum payoff boh on average and n he long run, whle he lower cos player underperforms hs on average. 3) Wh eher unformly weghed prors or concenraed prors, boh players mxed sraegy vecors converge o a seady sae, bu neher player s mxed sraegy converges o he Nash equlbrum. 4) In he asymmerc case wh unformly weghed prors, he hgher cos player s mxed sraegy vecor converges o seady sae faser han ha of he lower cos player. Furhermore, he hgher cos player ges closer o he Nash equlbrum. 5) In he symmerc cos case, varyng he prors affecs he dsrbuon of producon and of payoffs beween he wo frms, bu no eher he oal expeced quany produced nor he oal payoff acheved, and boh he oal quany and he oal payoff are lower han hey would be n equlbrum. 6) In he asymmerc cos case, varyng he prors affecs he dsrbuon of producon and of payoffs beween he wo frms, bu no eher he weghed sum of expeced quany produced nor he weghed sum of payoff acheved, and boh he weghed sum quany and he weghed sum payoff are lower han hey would be n equlbrum. 7) Varyng he spread of each player s pror whle holdng he mean fxed does no affec he long-run dynamcs of play. 8) The dsance beween player 1 s mxed sraegy vecor and hs Nash equlbrum quany a me T=1000 nversely relaed o he dfference beween he quany a whch player 1 s pror s concenraed and player 2 s Nash equlbrum quany. 35

36 9) There s an overconfdence premum: he worse off a player nally expecs her opponen o be, he beer off she herself wll evenually be. 5. Addng Naure as a Markov process Havng fully analyzed he no Naure benchmark case, I now examne he dynamcs of sochasc fcous play n he presence of Naure. There are several key feaures of he dynamcs ha arse when Naure s a Markov process. 24 The frs key feaure s ha he rajecores of play and payoffs are dsconnuous. The dsconnues arse because players assessmens of Naure a each me are condonal on value of he shock produced by Naure a me -1. Condonal on any gven value of he prevous perod s shock -1, he players mxed sraeges n Fgure 2.1 evolve connuously; dsconnues arse, however, whenever he value of -1 changes. The dynamcs ha arse when Naure s Markov hus peces ogeher he dynamcs ha arse when Naure evolves as each of hree separae..d. processes, one for each of he condonal dsrbuons of. Smlarly, rajecores for he acual payoffs acheved (Fgure 2.2), for he means and varances of quanes as aken over players me- mxed sraeges and opponen s me- assessmens (Fgure 2.3), and for he means and varances of he ex ane and nerm payoffs (Fgure 2.4) are dsconnuous, and can be vewed as an assemblage of peces of hree separae connuous rajecores. 24 Noe ha because he Markov ranson marx generaes a hgh degree of perssence n non-zero Naure shocks, he dynamcs may vary from one smulaon o he nex. Whle he fgures presened n hs secon plo he oucome arsng from he realzaon of one parcular sequence of shocks, my analyss focuses on he qualave feaures of he resuls ha were robus across he wo parameer ses I red. 36

37 (a) Fgure 2.1. Dynamcs of players mxed sraeges wh (a) symmerc and asymmerc coss as a funcon of me when Naure s Markov. As a benchmark, he Nash equlbrum quanes are q (5,5) n he symmerc cos case and q (3,6) n he asymmerc cos case. Each player has a unformly weghed pror over he oher. (a) Fgure 2.2. Acual payoffs acheved by each player as a funcon of me n he (a) symmerc and asymmerc cases when Naure s Markov. Each player has a unformly weghed pror over he oher. 37

38 (a) (c) Fgure 2.3. Means and varances of quanes, as aken over players me- mxed sraeges and opponen s me- assessmens when Naure s Markov n he (a) symmerc case and he asymmerc case for he hgher-cos player 1 and (c) he lower-cos player 2 as a funcon of me. Each player has a unformly weghed pror over he oher. 38

39 (a) (c) Fgure 2.4. Means and varances of ex ane and nerm payoff when Naure s Markov n he (a) symmerc case and he asymmerc case for he hgher-cos player 1 and (c) he lower-cos player 2 as a funcon of me. Each player has a unformly weghed pror over he oher. In addon o he dsconnuous naure of he rajecores of play and payoffs, a second key feaure of he dynamcs ha arse when Naure s Markov s he lack of convergence. Alhough play may converge condonal on any of he hree values of -1, he overall dynamcs, aken over all possble realzaons of -1, does no. In oher words, whle each of he hree separae..d. Naure scenaros may lead o convergence, he assemblage of hese hree dsjon peces does no. Thus, as seen n Fgure 2.5, mxed sraeges do no converge when players 39

40 have unformly weghed prors on each oher, nor do hey approach he Nash equlbrum. 25 Lkewse, as seen n Fgure 2.6, payoffs do no converge eher. Smlarly, play does no converge when prors are concenraed (Fgure 2.11). 26 (a) (c) (d) Fgure 2.5. Maxmum Eucldean dsance beween player s mxed sraegy vecor n perods m, n when Naure s Markov n he (a) symmerc and asymmerc cases as a funcon of me. Dsance beween player s mxed sraegy vecor and he Nash equlbrum n he (c) symmerc and asymmerc cases. Each player has a unformly weghed pror on he oher. 25 In Fgure 2.5, maxmum Eucldean dsance drops o 0 a = 1000 no because convergence occurs, bu because I runcae my smulaons a T=1000. Because I calculae dsance as d() max[ EucldeanDs( X, X )] mn, [, T], m n d(1000)=0. An alernave way o calculae convergence would be o run he smulaons for more han T me seps, say 1500 me seps, bu hen only calculae dsance up o me T. 26 To bes enable comparsons beween he no Naure and he Markov Naure cases, I chose o number he Markov fgures accordng o he order of he analogous fgures n he no Naure case raher n he order of her appearance n he ex. 40

41 (a) (c) (d) Fgure 2.6. Maxmum Eucldean dsance beween player s mxed sraegy vecor n perods m, n when Naure s Markov n he (a) symmerc and asymmerc cases as a funcon of me. Dsance beween player s mxed sraegy vecor and he Nash equlbrum n he (c) symmerc and asymmerc cases. Each player has a concenraed pror ha places all he wegh on he sraegy q j = 9. (a) Fgure The number of me seps unl convergence when Naure s Markov n he (a) symmerc and asymmerc cases. Player 2 s Nash equlbrum sraegy s ndcaed by he dashed lne. 41

42 A hrd key feaure of he dynamcs ha arse when Naure s Markov s ha whle neher play nor payoffs converge, hey appear o evenually ener an ergodc se, as seen n he phase porras n Fgures 2.7 and 2.8. Each of he rajecores n hese phase porras was generaed from he same sequence of shocks. For he parcular sequence of shocks used, T-1= T =4. (a) Fgure 2.7. Phase porras of expeced quany show he effec of varyng prors over each oher when Naure s Markov n he (a) symmerc and asymmerc cases. (a) Fgure 2.8. Phase porras of ex ane expeced payoff when Naure s Markov n he (a) symmerc and asymmerc cases. 42