High frequency repeated games with costly monitoring

Transcription

1 Theoretcal Economcs 13 (2018), / Hgh frequency repeated games wth costly montorng Ehud Lehrer School of Mathematcal Scences, Tel Avv Unversty and INSEAD Elon Solan School of Mathematcal Scences, Tel Avv Unversty We study two-player dscounted repeated games n whch one player cannot montor the other unless he pays a fxed amount. It s well known that n such a model the folk theorem holds when the montorng cost s on the order of magntude of the stage payoff. We analyze hgh frequency games n whch the montorng cost s small but stll sgnfcantly hgher than the stage payoff. We characterze the lmt set of publc perfect equlbrum payoffs as the montorng cost tends to 0. It turns out that ths set s typcally a strct subset of the set of feasble and ndvdually ratonal payoffs. In partcular, there mght be effcent and ndvdually ratonal payoffs that cannot be sustaned n equlbrum. We also make an nterestng connecton between games wth costly montorng and games played between long-lved and short-lved players. Fnally, we show that the lmt set of publc perfect equlbrum payoffs concdes wth the lmt set of Nash equlbrum payoffs. Ths mples that our characterzaton apples also to sequental equlbra. Keywords. Hgh frequency repeated games, costly montorng, Nash equlbrum, publc perfect equlbrum, no folk theorem, characterzaton. JEL classfcaton. C72, C Introducton A key result n the theory of dscounted repeated games wth full montorng s the folk theorem, whch states that as the dscount factor goes to 1, the set of subgameperfect equlbrum payoffs converges to the set of feasble and ndvdually ratonal payoffs. 1 The folk theorem mples n partcular that a long-term nteracton enables effcency: the effcent and ndvdually ratonal feasble payoffs can be sustaned n equlbrum. Ths observaton s vald when players fully montor each other s moves Ehud Lehrer: lehrer@post.tau.ac.l Elon Solan: elons@post.tau.ac.l Ths research was supported n part by the Google Inter-Unversty Center for Electronc Markets and Auctons. Lehrer acknowledges the support of the Israel Scence Foundaton, Grant 963/15. Solan acknowledges the support of the Israel Scence Foundaton, Grants 212/09 and 323/13. We thank Trstan Tomala, Jérôme Renault, and Phl Reny for useful comments on prevous versons of the paper. We are partcularly grateful to Johannes Hörner for drawng our attenton to the paper by Fudenberg et al. (1990). 1 To be precse, an addtonal full-dmensonalty condton s requred. Copyrght 2018 The Authors. Theoretcal Economcs. The Econometrc Socety. Lcensed under the Creatve Commons Attrbuton-NonCommercal Lcense 4.0. Avalable at

2 88 Lehrer and Solan Theoretcal Economcs 13 (2018) and, consequently, can enforce any pattern of behavor that results n an ndvdually ratonal payoff. In practce, more often than not, players do not perfectly montor each other s actons, but do obtan sgnals that depend on ther actons. Ths paper dscusses stuatons where players cannot freely observe the actons taken by ther opponents. Rather, players observe other players actons only f they pay a montorng fee. For nstance, volatons of varous treates, such as the Treaty on the Non-Prolferaton of Nuclear Weapons or the Conventon for the Protecton of Human Rghts and Fundamental Freedoms, are often dffcult to dentfy. As a result, the nternatonal communty conducts perodc nspectons to ensure that these treates are kept. Smlarly, esponage among countres and ndustral esponage wth the goal of revealng actons and ntent of opponents s a common practce. Repeated games wth costly montorng have been prevously studed n the lterature. In such models where the montorng cost s fxed, namely, ndependent of the dscount factor, the folk theorem has been obtaned (see an elaboraton below). Our goal n the current paper s to evaluate the robustness of ths result to changng montorng costs. We are partcularly nterested n the role played by the order of magntude of montorng costs relatve to stage payoffs. To ths end we study a hgh frequency dscrete-tme repeated game, a model that can be thought of as a good approxmaton to a contnuous-tme game. We consder the mpact of small perod length n a two-player repeated game, where the nspecton cost s small, but of hgher order than per perod payoffs. Ths model s relevant when an nspecton requres a fxed amount of effort that does not depend on the nteracton frequency. It may occur, for nstance, when preparng to launch an nspecton team or when the nspecton tself s expected to take a fxed amount of tme and effort that are not affected by the length of the perod nspected. An nspecton by the tax authorty, for example, s hghly tme consumng, even when t ams to nspect just one sngle taxpayer n one sngle year. Another example s a country that uses a collaborator to obtan mportant nformaton from an enemy country. The employment of a collaborator puts hm or her n danger of beng captured, thereby nflctng, regardless of the sgnfcance of the nformaton obtaned, a huge cost on the spyng country. In the presence of nspecton costs, t s too costly to montor the other player at every stage. There are smple equlbra that do not requre any nspecton, such as playng constantly an equlbrum of the one-shot game. Furthermore, by playng dfferent constant one-stage equlbra n dfferent stages, one can obtan, as a lmt of publc perfect equlbrum (PPE) payoffs of the repeated game, any pont n the convex hull of the oneshot game equlbrum payoffs. A natural queston arses as to whether addtonal (and maybe more effcent) payoffs can be supported by equlbra. The man objectve of the paper s to characterze the lmt set of PPE payoffs as the players become more patent. We show that the equlbrum payoffs of a player cannot exceed a certan upper bound determned by the structure of the one-shot game. Ths mples that costly montorng typcally mpedes cooperaton: not all the effcent and ndvdually ratonal payoffs can be sustaned by an equlbrum, and so there s an antfolk theorem.

3 Theoretcal Economcs 13 (2018) Hgh frequency repeated games 89 The goal of ant-folk theorems, such as the one presented here, s to dentfy the assumptons needed to get a folk theorem. An mportant nsght from our result s that whether the folk theorem apples depends on the magntude of the montorng cost relatve to the stage-game payoffs. When the montorng cost s of the same magntude as the stage-game payoffs, a folk theorem s obtaned, whle when the montorng cost s much hgher, effcency s lost and an ant-folk theorem s obtaned. To explan why effcency s lost, suppose that at some stage of an equlbrum, player does not play a best response to player j s mxed acton. In ths case, so as to deter player from ganng by a devaton that would go unnotced, player j must montor player wth a suffcently hgh probablty. Snce montorng s costly, player j should be later compensated for montorng player. Moreover, snce the montorng cost s hgher than the contrbuton of a sngle stage payoff on the total payoff, when player j montors player, her contnuaton payoff that follows the montorng must be hgher than her expected payoff pror to montorng. Now consder player 1 s maxmal equlbrum payoff n a repeated game wth costly montorng and an equlbrum that supports t. If player 1 montors player 2 wth postve probablty at the frst stage, hs contnuaton payoff followng the montorng should be hgher than hs expected payoff pror to the montorng. In other words, the contnuaton payoff should be hgher than the maxmal equlbrum payoff, whch s mpossble. Consequently, at the frst stage, player 1 does not montor hs opponent. Ths mples that n the frst stage, player 2 should not have an ncentve to devate: she already plays a one-shot best response and there s no need for player 1 to nspect her. Ths reasonng not only shows the connecton between player 1 s maxmal equlbrum payoff and acton pars n whch player 2 plays a one-shot best response payoff, t also mposes an upper bound on player 1 s equlbrum payoffs and thereby restrcts effcency. Player 2 s equlbrum payoffs are also subject to a smlar upper bound. It turns out that the upper bound over equlbrum payoffs thus obtaned s smlar to the upper bound over equlbrum payoffs n case of long-lved players playng aganst a sequence of short-lved players (see Proposton 3 n Fudenberg et al. (1990), whch s analogous to our Theorem 1). In such an nteracton, a short-lved player has only short-term objectves, threats of punshment are not effectve aganst such a player, and he therefore always plays n equlbrum a one-shot game best response. Consequently, the maxmal equlbrum payoff of a long-lved player s characterzed by acton profles where the short-lved players play a best response. Ths observaton comprses an nterestng smlarty wth our model: the maxmal equlbrum payoff of each player n our model s precsely the bound on the long-lved player defned n Fudenberg et al. (1990). Despte the smlartes, there are two essental dfferences between the results n the two models. Frst, the restrcted neffcency n a game wth short-lved players s a consequence of the fact that these players consder only the mmedate stage nteracton. In our model, n contrast, both players have long-run objectves. It s only when the expected payoff of one player from ths stage and on s equal to hs maxmal equlbrum payoff that the other player behaves lke a short-lved player. Moreover, even n ths case, ths behavor s temporary and apples only to the current stage of the game. Once the expected payoff of the player from ths stage and on falls below hs maxmal equlbrum

4 90 Lehrer and Solan Theoretcal Economcs 13 (2018) payoff, the opponent s behavor s no longer the behavor of a short-lved player. The second dfference s that n Fudenberg et al. (1990), the upper bound on payoffs apples only to the long-lved player, whle n our model t further restrcts effcency snce t apples to both players. In constructng equlbra, montorng s crucal both to sustan and to enforce equlbrum payoffs. Specfcally, montorng serves three dfferent purposes. 1. Montorng the other player wth suffcently hgh probablty, coupled wth a threat of punshment, ensures that n equlbrum the other player wll not devate to an acton that he s not supposed to play. 2. When a player plays a mxed acton, dfferent actons played wth postve probablty may yeld dfferent payoffs. To make the player ndfferent as to whch acton he takes, dfferent contnuaton payoffs must be attached to dfferent actons. Montorng s used to enable the players to coordnate the contnuaton payoffs. When a player s supposed to play a mxed acton, he s montored wth a postve probablty, and n case he s montored, the contnuaton payoff s set n such a way as to make the player ndfferent between hs actons. 3. Snce montorng s costly, n equlbrum a player can montor the other player so as to burn money. He wll do so because otherwse he wll be punshed, and the resultng payoff would be worse. Ths possblty of forced montorng enables one to desgn relatvely low contnuaton payoff. For nstance, suppose that a player prescrbed to play a certan mxed acton s montored, and t turns out that the realzed pure acton yelds hm a hgh payoff. Ths player can be nstructed later on to montor and pay the montorng cost, and thereby reduce hs own payoff. In our model, montorng s common knowledge. In partcular, both players know ts outcome. Ths mples that the problem of characterzng the set of equlbrum payoffs s recursve. Indeed, our proof method s recursve n nature: we have a conjecture about the lmt set of PPE payoffs, and for each pont n ths set, we provde a proper one-shot game and contnuaton payoffs n the set that render t an equlbrum. The lterature on games wth mperfect montorng When the magntude of montorng costs equals that of stage payoffs, a repeated game wth costly montorng can be recast as a game wth mperfect montorng, whch s surveyed n, e.g., Pearce (1992), Malath and Samuelson (2006), and Mertens et al. (2016). Indeed, the choce weghed by each player at every stage s composed of two components: (a) whch acton to play and (b) whether to montor the other player. The payoff functon can be adapted accordngly: n case no montorng s performed, the stage payoff concdes wth the orgnal payoff and s equal to the orgnal payoff mnus the montorng cost otherwse. In our setup, the montorng cost depends on the dscount factor and s sgnfcantly larger than the stage payoff, and therefore the game cannot be modelled as a repeated nteracton wth mperfect montorng: montorng cannot be consdered as a regular acton of an extended base game.

5 Theoretcal Economcs 13 (2018) Hgh frequency repeated games 91 Undscounted repeated games wth mperfect montorng have been studed by Lehrer (1989, 1990, 1991, 1992). Abreu et al. (1990) analyzed dscounted games and used dynamc programmng technques to characterze the set of publc equlbrum payoffs. Fudenberg et al. (1994) provded condtons that guarantee that any feasble and ndvdually ratonal payoff s a perfect equlbrum payoff when players are suffcently patent. Fudenberg and Levne (1994) characterzed the lmt set of publc perfect equlbrum payoffs n the presence of both publc and prvate sgnals as the dscount factor goes to 1. Compte (1998) and Kandor and Matsushma (1998) proved a folk theorem for repeated games wth communcaton and ndependent prvate sgnals. Hörner et al. (2011) extended the characterzaton to stochastc games. Several authors studed specfcally repeated games wth costly observatons. Ben- Porath and Kahneman (2003) studed a model n whch, at the end of every stage, each player can pay a fxed amount and observe the actons just played by a subset of other players. They proved that f the players can communcate, the lmt set of sequental equlbrum payoffs when players become patent s the set of feasble and ndvdually ratonal payoffs. Myagawa et al. (2008) assumed that montorng decsons are not observed by others, that players have a publc randomzaton devce, and that they observe a stochastc sgnal that depends on other players actons even f they do not purchase nformaton. They proved that under full dmensonalty condton, the folk theorem s stll obtaned. In the model studed by Flesch and Perea (2009), players can purchase nformaton on actons played n past stages and n the current stage. They proved that n case at least three players (resp. four players) are nvolved and each player has at least four actons (resp. three actons), a folk theorem for sequental equlbra holds. The results attaned by the last three papers mentoned s dfferent from ours. They obtan the standard folk theorem, whle we do not. The reason for ths dfference s that n ther models, the montorng cost s bounded. As mentoned above, ths knd of model s a specal case of repeated games wth mperfect montorng. Another related paper n a dfferent strand of lterature s Lpman and Wang (2009), who studed repeated games wth swtchng costs. In ths model a player has to pay a fxed cost whenever playng dfferent actons n two consecutve stages. Smlarly to our cost structure, the swtchng cost n Lpman and Wang (2009) s much hgher than the stage payoff. Nevertheless they obtan a folk theorem. Thestructureofthepaper The model s presented n Secton 2. Secton 3 provdes the upper bounds on the payers payoffs and a no-folk-theorem result. Secton 4 characterzes the set of publc perfect equlbrum payoffs, whle Secton 5 provdes the man deas of the equlbrum constructon. Fnal comments are gven n Secton 6. The proofs appear n the Appendx, avalable n a supplementary fle on the journal webste, /supplement.pdf.

6 92 Lehrer and Solan Theoretcal Economcs 13 (2018) 2. The model 2.1 The base game Let G = ({1 2} A 1 A 2 u 1 u 2 ) be a two-player one-shot base game n strategc form. The set of players s {1 2}, A s the fnte set of player s actons, and u : A R s hs payoff functon, where A := A 1 A 2. As usual, the mult-lnear extenson of u s stll denoted by u. For notatonal convenence, we let j denote the player who s not. The mnmax value (n mxed strateges) of player nthebasegamesgvenby 2 v := mn α j (A j ) max u (α α j ) α (A ) We assume wthout loss of generalty that the maxmal payoff n absolute values, max =1 2 max a A u (a), does not exceed 1. Denote the mnmax pont by v := (v 1 v 2 ). A payoff vector x R 2 s ndvdually ratonal (resp. strctly ndvdually ratonal) for player f x v (resp. x >v ). Denote by F the set of all vectors n R 2 domnated by a feasble vector n the base game: 3 F := { x R 2 : y conv { u(a) a A } such that y x } Snce montorng s costly, players can use the montorng opton to burn money. Therefore, the set of feasble payoff vectors n the repeated game s the set of vectors domnated by feasble payoffs n the base game. 2.2 The repeated game We study a repeated game n dscrete tme, denoted G(r c ), whch depends on three parameters, r (0 1), c>0, and >0, and on the base game G. Ths game s descrbed as follows. 1. The base game G s played over and over agan. 2. The duraton between two consecutve stages s. 3. The dscount factor s r. 4. At every stage of the game each player chooses an acton n the base game and whether to montor the acton chosen by the other player. Montorng the other player s acton costs c and becomes common knowledge. We denote by O (resp. NO ) the choce of player to montor (or observe; resp. not to observe) player j s acton. A prvate hstory of player at stage n (n N) conssts of (a) the sequence of actons he played n stages 1 2 n 1, (b) the stages n whch player j montored hm, (c) the stages n whch he montored player j, and (d) the actons that player j played n those 2 For every fnte set X,wedenoteby (X) the set of probablty dstrbutons over X. 3 Let x y R 2.Wedenotey x f y x for each = 1 2. In ths case we say that x s domnated by y. The vector x s strctly domnated by y when y >x for each = 1 2.

7 Theoretcal Economcs 13 (2018) Hgh frequency repeated games 93 stages. Denote by H (n 1) the set of all such prvate hstores. The set H (n 1) conssts of all player s nformaton sets before makng a decson at stage n. Note that H (n 1) s a fnte set. A publc hstory at stage n conssts of (a) the stages n whch each player montored the other player pror to stage n and (b) the actons that the montored player took n these stages. The publc hstory s commonly known to both players. Denote by H P (n 1) the set of publc hstores at stage n. LetF n 1 be the σ-algebra defned on the space of nfnte plays H and spanned by the set of all publc hstores of length n 1. A pure (resp., publc pure) strategy of player s a functon that assgns two components to every prvate (resp., publc) hstory n H (n 1) (resp., H P (n 1)): an acton n A to play at stage n and a bnary varable, ether O or NO, that ndcates whether player montors player j at stage n. A behavor (resp. publc behavor) strategy of player s a functon that assgns a probablty dstrbuton over A {O NO } for every stage n and every prvate (resp. publc) hstory n H (n 1) (resp. H P (n 1)). In our constructon we only use publc behavor strateges n whch these dstrbutons are product dstrbutons. That s, the acton played at stage n s condtonally ndependent of the decson whether to montor at that stage. Snce the players have perfect recall, by Kuhn s theorem every publc behavor strategy s strategcally equvalent to a mxed publc strategy and vce versa. Every par of strateges (σ 1 σ 2 ) nduces a probablty dstrbuton P σ1 σ 2 over the set of nfnte plays H, supplemented wth the σ-algebra generated by all fnte cylnders. We denote by E σ1 σ 2 the correspondng expectaton operator. Denote by α n player s mxed acton at stage n, andα n = (α n 1 αn 2 ). The total (expected) payoff to player when the players use the strategy par (σ 1 σ 2 ) s ( ) ( ) U (σ 1 σ 2 ) := E σ1 σ 2 [(1 ] r ) r (n 1) u (α n ) c r (τk 1) (1) n=1 where (τ k) k N are the stages n whch player montors player j. It s worth notng that the contrbuton of the stage payoff to the total dscounted payoff depends on the duraton between stages, and s equal to (1 r )u (α n ). The dscounted value of the nth stage payoff s therefore equal to (1 r )r (n 1) u (α n ).Conversely, the montorng cost s much hgher than the stage payoff. It s constant and does not depend on the duraton between stages. Ths s why the cost of the kth observaton, whch s performed at stage τ k,smultpledbyr (τk 1) and not by (1 r ).The dfference between the nature of the stage payoff and that of the montorng cost s the pont where our model departs from the lterature. k 2.3 Equlbrum A par of strateges s a (Nash) equlbrum f no player can ncrease hs total payoff by devatng to another strategy. A publc equlbrum s an equlbrum n publc strateges. In such an equlbrum, no player can proft by devatng to any strategy, publc or not publc. A publc perfect equlbrum s a par of publc strateges that nduces an equlbrum n the contnuaton game that starts after any publc hstory. Let NE(r c )be the

8 94 Lehrer and Solan Theoretcal Economcs 13 (2018) set of Nash equlbrum payoffs n the game G(r c ) and let PPE(r c ) be the set of publc perfect equlbrum payoffs of ths game. Defne NE (r) = lm sup c 0 PPE (r) = lm sup c 0 lm sup NE(r c ) 0 lm sup PPE(r c ) 0 These are the lmt sets of Nash equlbrum payoffs and publc perfect equlbrum payoffs as both the duraton between stages and the observaton cost go to 0, and the former goes to 0 faster than the latter. By defnton, PPE(r c ) NE(r c )for every dscount factor r, every observaton cost c, and every duraton, and therefore PPE (r) NE (r). Our man result characterzes these sets n terms of the base game. It turns out that under a weak techncal condton these two sets concde. Playng a Nash equlbrum of the base game at every stage and after every hstory, wthout montorng each other, s a statonary equlbrum of the game G(r c ). We therefore conclude that the set PPE(r c )contans the set NE of Nash equlbrum payoffs of the base game. By parttonng the set of stages nto dsjont subsets, and playng the same Nash equlbrum n all stages of the subset, wthout montorng the other player, we construct an equlbrum payoff n the convex hull of NE. Whenr > 1 2,ths constructon can yeld any vector n the convex hull of NE. We thus obtan the followng lemma. Lemma 1. For every r>0,thesetppe (r) contans the convex hull of the set NE. A maxmn mxed acton of player nthebasegamesanymxedactonα (A ) that satsfes u (α a j ) v for every a j A j. By repeatng hs maxmn mxed acton n the base game and not montorng the other player, player guarantees a payoff v n the repeated game G(r c ). We conclude wth the followng result. Lemma 2. For every r c > 0 and x NE(r c ),onehasx v. 3. No folk theorem In ths secton, we show that the folk theorem does not hold n games wth costly montorng. In Secton 3.1, we present two quanttes M 1 and M 2, and n Secton 3.2,weshow that M s an upper bound to player s payoff n NE(r c ). In some games these bounds are restrctve, n the sense that they are lower than the hghest payoff of player n the set of feasble and ndvdually ratonal payoffs. In partcular, t may happen that the set NE(r c )may be dsjont from the Pareto fronter of F V. In subsequent sectons, we characterze the sets NE (r) and PPE (r) usng the quanttes M 1 and M 2.

9 Theoretcal Economcs 13 (2018) Hgh frequency repeated games 95 Fgure 1. The prsoner s dlemma. 3.1 Best response and the ndex M We say that player plays a best response at the mxed-acton par α = (α 1 α 2 ) f u (α 1 α 2 ) = max a A u (a α j ) Player s ndfferent at α = (α 1 α 2 ) f for every acton a such that α (a )>0, u (α α j ) = u (a α j ) We now defne two ndces M 1 and M 2 that play a major role n our characterzaton. Let { M := max mn u (a α j ): (α 1 α 2 ) (A 1 ) (A 2 ) a : α (a )>0 (2) and α j s a best response to α } To explan the defnton n (2), consder M 2.Letα 2 be a mxed acton of player 2 and let α 1 be a best response of player 1 to α 2. By playng α 2,player2 does not necessarly optmze aganst α 1, mplyng that any pure acton n the support of α 2 mght nduce a dfferent payoff for player 2. We focus on the mnmum among these payoffs, whch s a functon of the par (α 1 α 2 ). The ndex M 2 s the maxmum of all these mnmal numbers, over all pars (α 1 α 2 ),whereα 1 s a best response of player 1 to α 2. The next example llustrates the quantty M 2 n the prsoner s dlemma. Example 1 (The prsoner s dlemma). The prsoner s dlemma s gven by the base game that appears n Fgure 1. We calculate M 2 for ths game. Fx a mxed acton α 2 of player 2. The best response of player 1 to α 2 s D and mn a2 supp(α 2 ) u 2 (D a 2 ) s ether 1 or 0, wheresupp(α ) := {a A : α (a )>0}. The maxmum over these mnma s 1, mplyng that M 2 = 1. By the defnton of M,fα j s a best response to α,then M u (a α j ) for at least one acton a supp(α ) Consequently, M s at least as hgh as the payoff of player n any equlbrum of the base game. Formally, for every Nash equlbrum α of the base game, M u (α) {1 2}

10 96 Lehrer and Solan Theoretcal Economcs 13 (2018) Fgure 2. The game n Example 2. The followng example shows that M mght be strctly hgher than player s payoffs n all Nash equlbra. Example 2. Consder the 3 3 base game that appears n Fgure 2. By an teratve elmnaton of pure strateges, one deduces that the acton par (T L) s the unque equlbrum of the base game, and t yelds the payoff (1 1). Snce C s a best response to I,wededucethatM 1 2, and snce B s a best response to R,wededuce that M 2 2. The sgnfcance of M becomes apparent n Theorem 1 below. It states that M s the upper bound of player s payoffs n the repeated game G(r c ). The ntuton s as follows. We frst explan why the defnton of M concerns mxed actons at whch player j plays a best response. When player montors player j, the former ncurs a sgnfcant montorng cost (recall that c s sgnfcantly larger than ). Consequently, n equlbrum, player s contnuaton payoff after montorng must be hgher than hs expected payoff before dong so. Ths mples that n equlbrum that supports player s maxmal payoff n the set NE(r c ), he does not montor hs opponent at the frst stage, because otherwse hs contnuaton payoff followng the montorng would exceed the maxmal payoff. Therefore player j must have, n equlbrum, no ncentve to devate n the frst stage, meanng that he must play a one-shot best response. We now explan why M s ndeed an upper bound of the set of equlbrum payoffs. Assume by contradcton that player s maxmal payoff n the set NE(r c ), denoted x, s strctly hgher than M. Denote by α = (α 1 α 2 ) the mxed acton par that the players play at the frst stage of an equlbrum that supports x. As we saw above, player j plays a best response at α. Consder now the event that player plays at the frst stage a pure acton a that mnmzes the stage payoff u (a α j ) among the actons a such that α (a )>0. By the defnton of M we have u (a α j) M <x. Snce a s played wth postve probablty at the frst stage, x s a weghted average of u (a α j) and the contnuaton payoff that follows t. Ths contnuaton payoff s also an equlbrum payoff and therefore cannot exceed x 1. We obtan that x 1 s a weghted average of two smaller numbers, of whch one s strctly smaller. Ths s a contradcton. 3.2 Boundng the set of Nash equlbra In ths subsecton, we consder fxed r c > 0. Snce the set of strateges s compact and the payoff functon s contnuous over the set of strategy pars, one obtans the followng result.

11 Theoretcal Economcs 13 (2018) Hgh frequency repeated games 97 Lemma 3. The set NE(r c ) of Nash equlbrum payoffs n the repeated game s compact. The followng theorem states that when s suffcently small, player s equlbrum payoff cannot exceed M. In partcular, t means that not all feasble and ndvdually ratonal payoffs are equlbrum payoffs (.e., not all of them are n NE(r c )): costly montorng typcally mpars effcency. Theorem 1. Fx >0, {1 2},andx NE(r c ).If < ln(1 1 x ) ln(r) ), then x M. Example 1 (Revsted). As mentoned before, n the prsoner s dlemma M 1 = M 2 = v 1 = v 2 = 1. Snce, by Theorem 1, any Nash equlbrum payoff cannot exceed M,weobtan NE(r c )={(1 1)} provded that s suffcently small. In other words, n the repeated prsoner s dlemma, mutual defecton s the only equlbrum payoff n the presence of a hgh montorng fee. The ntuton behnd ths result s that to mplement a payoff that s not (1 1), at least one player, say player 2, must play the domnated acton C. Ths mples that to deter a devaton to D, player1 has to montor player 2 wth a postve probablty. Whenever player 1 montors player 2, heshouldbecompensatedby a hgher contnuaton payoff for havng to bear the montorng cost. The only crcumstance whereby the contnuaton payoff may compensate player 1 s n case player 2 plays the domnated acton C wth a hgher probablty. In that case, player 1 must contnue montorng player 2 wth postve probablty. It mght therefore happen, although wth a small probablty, that player 1 wll have a long stretch of stages n whch he montors player 2, the contnuaton payoff of player 1 wll keep ncreasng and eventually exceed 4, whch s mpossble. Proof of Theorem 1. Weprovethetheoremfor = 1. ByLemma 3, thesetne(r c ) s compact. Let x be a payoff vector n NE(r c ) that maxmzes player 1 s payoff. That s, x argmax{x 1 : x NE(r c )}. Assume to the contrary that M 1 <x 1.When s suffcently small, we obtan a contradcton. Consder an equlbrum σ that supports x, and denote by α = (α 1 α 2 ) (A 1 ) (A 2 ) the mxed-acton par played under σ at the frst stage. For every acton a 1 A 1, denote by I 1 (a 1 ) the event that player 1 plays the acton a 1 and montors player 2 at the frst stage. Denote by I 1 := a 1 A 1 I 1 (a 1 ) the event that player 1 montors player 2 at the frst stage. Let z 1 be player 1 s contnuaton payoff from stage 2 onward, condtonal on hs nformaton followng stage 1. The proof s dvded nto two cases. Case 1: P σ (I 1 )>0. Snce σ s an equlbrum, the expected payoff of player 1 condtonal on the event that he montors player 2 at the frst stage must be equal to x 1. Furthermore, the event I 1 s common knowledge. If both players montor each other at the frst stage, then the actons of both players are known to both and the contnuaton play s a Nash equlbrum (of the repeated game). If only player 1 montors at the frst stage, an event that s known to both players, then the expected play followng c

12 98 Lehrer and Solan Theoretcal Economcs 13 (2018) the frst stage and condtonal on the acton of player 2 at the frst stage s an equlbrum. Consequently, the expectaton of z 1 condtonal on I 1 and a 1 2 s at most x 1,thats, E σ [z 1 I 1 a 1 2 ] x 1. We therefore deduce that x 1 = E σ [ (1 r )u 1 (α) c + r z 1 I 1 ] 1 r c + r x 1 Ths nequalty s volated when < ln(1 c/(1 x 1 )) ln(r). Case 2: P σ (I 1 ) = 0. Snce player 1 does not montor player 2 at the frst stage, α 2 s a best response at α. Otherwse,player2 would have a proftable devaton at the frst stage that would go unnotced. The defnton of M 1 mples that M 1 mn a1 supp(α 1 ) u 1 (a 1 α 2 ). Denote by a 1 supp(α 1) an acton that attans the mnmum. Snce by assumpton M 1 < x 1,onehasu 1(a 1 α 2)<x 1. We clam that E σ [z 1 a 1 ] x 1.Ifplayer2dd not montor player 1, then each player s play after the frst stage s ndependent of hs opponent s acton at the frst stage, and the expected contnuaton play s a Nash equlbrum. If player 2 montors player 1, thenas n Case 1 the expected play after the frst stage condtoned on the acton of player 1 at the frst stage s an equlbrum. We thus conclude that E σ [z 1 a 1 ] x 1, and therefore x 1 = (1 r )u 1 (a 1 α 2) + r E σ [z 1 a 1 ] <(1 r )x 1 + r x 1 x 1 a contradcton. 4. The man result: Characterzng the set of publc perfect equlbrum payoffs The set of ndvdually ratonal payoff vectors that are (a) domnated by a feasble pont and(b)yeldtoeachplayer at most M, s denoted by F M := {x F : v 1 x 1 M 1 and v 2 x 2 M 2 } Theorem 1 and Lemma 2 mply that NE(r c ) F M, provded that s suffcently small, and consequently the set PPE(r c ) s a subset of F M for every suffcently small. Our man result states that the sets NE(r c ) and PPE(r c ) are close to F M, provded that c and are suffcently small. Ths result mples n partcular that the bound M, establshed n Theorem 1,stght. We now defne the closeness concept between the sets that we use. A set K of payoff vectors s an asymptotc set of Nash equlbrum payoffs (resp. of PPE payoffs) f any pont n the set s close to a pont n NE(r c ) (resp. PPE(r c )) for every c and small enough and every dscount rate r. Defnton 1. A set K R 2 s an asymptotc set of Nash equlbrum payoffs f, for every r>0 and every ɛ>0, theresc ɛ > 0 such that for every c (0 c ɛ ] there s c ɛ r > 0 such that for every (0 c ɛ r ) we have max y K mn x NE(r c ) d(x y) ɛ

13 Theoretcal Economcs 13 (2018) Hgh frequency repeated games 99 Fgure 3. A game where M 1 = v 1. The set K s an asymptotc set of PPE payoffs f an analogous condton holds wth respect to the set PPE(r c ). Note that Defnton 1 concerns only one drecton of the Hausdorff dstance: t requres that any pont n K s close to a Nash equlbrum payoff (or a PPE payoff), but not vce versa. Theorem 2. If M 1 >v 1 and M 2 >v 2, then for every dscount factor r (0 1) the set F M s an asymptotc set of Nash equlbrum payoffs and an asymptotc set of PPE payoffs. In partcular, NE (r) = PPE (r) = F M. The proof of Theorem 2 appears n Secton 5 and n the Appendx. As the followng example shows, the condton n Theorem 2 requrng that M 1 >v 1 and M 2 >v 2 cannot be dsposed of. Example 3. Consder the 2 2 base game llustrated n Fgure 3. The mnmax value of both players s 0. Sncethemaxmumpayoffofplayer1 s 0, we deduce that M 1 = 0, mplyng that M 1 = v 1. Snce B s a best response to R and not to L, t follows that M 2 = 2. In partcular, the set F M s the nterval between (0 0) and (0 2). We argue that the unque equlbrum payoff n the repeated game G(r c ) s (0 0), mplyng that the concluson of Theorem 2 does not hold. Indeed, snce v 1 = 0 and the maxmal payoff of player 1 s 0, hs payoff n every Nash equlbrum of G(r c ) as well as hs contnuaton payoff after any publc hstory s 0. The acton L strctly domnatestheacton R, and thereforewheneverplayer 2 plays R wth postve probablty, he must be montored by player 1. However, player 1 cannot be compensated for montorng; hence n equlbrum player 2 always plays L. Snce u 1 (B L) = 1, player1 always plays T : n equlbrum the players repeatedly play (T L) and, consequently, the unque equlbrum payoff s (0 0),asclamed. Remark 1. We assumed that the montorng fee c s the same for both players. The results reman the same f the montorng fees of the two players are dfferent, provded that the duraton s sgnfcantly smaller than both. That s, for every c 1 c 2 > 0 suffcently small, the set of equlbrum payoffs of the two-player repeated game G(r c 1 c 2 ), n whch the montorng costs of the two players are c 1 and c 2,sclose to the set F M, provded that s suffcently close to 0. In fact, n our proof t s more convenent to assume that the montorng fees of the players dffer. The sets of Nash and publc perfect equlbrum payoffs n G(r c 1 c 2 )are denoted by NE(r c 1 c 2 )and PPE(r c 1 c 2 ), respectvely.

14 100 Lehrer and Solan Theoretcal Economcs 13 (2018) 5. The structure of the equlbrum Theorem 1 mples that the set NE (r) s ncluded n F M. To complete the proof of Theorem 2, t remans to prove that PPE (r) contans F M.WefrstprovethatNE (r) F M by constructng equlbra n whch detectable devatons trgger ndefnte punshment. We then see how the ndefnte punshment can be replaced by a credble threat, mplyng that PPE (r) F M. At a techncal level our proof uses a classcal technque. We dentfy sets X of payoff vectors that have the followng property. Every x X s an equlbrum of a one-shot gamewhosepayoffsarecomposedasapayofffromthebasegameplusacontnuaton payoff from X tself. We start wth a small set X and expand t untl we obtan a set close to F M. The novelty of the proof s n the burnng-money process that we proceed now to ntroduce. Ths process allows one to decompose the contnuaton payoff nto two parts: a target contnuaton payoff and the gap between the actual contnuaton payoff and the target payoff. Ths gap s precsely the amount the players must burn. Whle the recursve calculaton of contnuaton payoffs, based on the actual play n the prevous stage, s rather complcated, each of the two parts can be easly calculated n a recursve way. The decomposton of the contnuaton payoff nto two parts sgnfcantly smplfes the constructon of equlbra n the current model, and may be useful n other models as well. 5.1 Lablty and burnng-money processes To smplfy the computatons n our constructon, we apply a postve affne transformaton on the payoffs. Applyng such an affne transformaton to the player s payoffs n the base game does not change the strategc consderatons of the players. However, t does change ther montorng fees and no longer allows us to assume that the montorng costs are dentcal for both players. We thus assume from now on that the montorng costs dffer and we denote the montorng cost of player by c (see Remark 1). In our constructon, players montor each other for two purposes. Frst, montorng s amed to deter players from devatng. Ths type of montorng takes place at random stages. Second, montorng s used to establsh contnuaton payoffs that ensure that players are ndfferent between ther prescrbed actons. Ths type of montorng takes place at known stages. To mplement the second purpose, we ntroduce burnng-money processes. The value of the burnng-money process at stage n, whch s called the player s debt, represents the amount that the player has to burn from stage n onward. Ths amount s measurable wth respect to the publc hstory at that stage, and thus each player knows the other player s debt. Moreover, each player can verfy whether the other player burned money as requred. The nature of the burnng-money process s that as long as the debt s smaller than c, the debt s deferred to the next perod, and due to dscountng, t ncreases. Ths happens untl the debt exceeds c. At ths pont n tme, player has to montor player j and as a result, hs debt s reduced by c. Falng to do so trggers a punshment. The debt mght also ncrease due to other reasons. Ths mght happen when,

15 Theoretcal Economcs 13 (2018) Hgh frequency repeated games 101 under equlbrum, a player plays wth postve probablty two actons that yeld dfferent stage payoffs. To ensure that the player s ndfferent between the two actons, hs debt ncreases when he s montored and plays the hgher-payoff acton. The defnton of the debt process reles on lablty processes. Defnton 2. A lablty process s a nonnegatve stochastc process ξ = (ξ n ) n N such that ξ n s measurable wth respect to F n+1 for every n N. The lablty s meant to stand for the addtonal debt that a player ncurs at stages n whch he s montored. The role of the lablty process s to make all actons played by the montored player payoff-wse equvalent to hm. Ths s the reason why the lablty at stage n depends on the play at that stage and, therefore, ξ n s F n+1 -measurable. Defnton 3. Let ξ = (ξ n) n N be a lablty process of player. A burnng-money process based on ξ s a stochastc process D = (D n ) n N that satsfes the followng propertes: We have D 1 0: the ntal debt s a nonnegatve real number. If D n c,thend n+1 = Dn c +ξ n. The nterpretaton s that at a stage n whch the r debt exceeds c,player has to montor the other player and ncurs a cost of c, thereby reducng hs debt by ths amount. The debt D n+1 s obtaned by addng the lablty ξ n to the revsed debt, and the total s dvded by the dscount rate r. If D n <c,thend n+1 = Dn +ξn. When the debt s below c r, no mandatory nspecton takes place, the lablty ξ n s added to the current debt, and the total s dvded by the dscount rate. Note that the debt D n at stage n depends only on the hstory up to (and ncludng) stage n: D n s measurable wth respect to F n. Ths mples that at the begnnng of stage n the debts of both players are common knowledge. Moreover, the debts are always nonnegatve. In our constructon, the lablty of player at stage n s at most 2(1 r ).Here2 s the maxmal dfference between two stage payoffs, and (1 r ) s the weght of a sngle stage payoff. Consequently, player s debt s bounded by c +2(1 r ) r. 5.2 Montorng to detect devatons Let α = (α 1 α 2 ) be a par of mxed actons played at some stage. When player 1 s ndfferent at α and α 1 s not a best response at α, the only way player 1 can gan s by devatng to an acton outsde the support of α 1. Suppose that player 2 montors player 1 wth probablty p. A threat to punsh player 1 down to hs mnmax level n case a devaton s detected s effectve f the expected loss due to the punshment s greater than the potental gan: 2(1 r )<p r (x 1 v 1 ),wherex 1 s player 1 s expected contnuaton payoff

16 102 Lehrer and Solan Theoretcal Economcs 13 (2018) when he s montored and no devaton occurs. It follows that to deter devatons to actons outsde the support of α 1, we need to set the per-stage probablty of montorng p to satsfy p> 2(1 r ) r (3) (x 1 v 1 ) An analogous nequalty holds when player 1 tres to deter devatons of player 2. Note 1 r that lm 0 = ln(r), and thus the probablty that a player s montored should be larger than 2 ( ln(r)) x 1 v 1, whch s of magntude. 5.3 The general structure of the equlbrum In ths secton, we descrbe the outlne of our constructon of Nash equlbra, whch are allpublc equlbra. The publcstrategy of player s based on a burnng-money process D = (D n ) n N, and for every publc strategy of length n 1, t assgns two parameters: (a) the one-shot mxed acton α n to play at stage n and (b) the probablty pn to montor player j at that stage. The montorng probablty p n takes one of three possble values: The value p n = 1. Hereplayer s requred to burn money, whch takes place when hs debt exceeds c. When player s debt s below c, The value p n = 0 when player j plays a best response and need not be montored. The value p n = p,wherep s some fxed postve but low constant that satsfes (3). Ths happens when player j s not playng a best response, hence has to be deterred from devatng. In prncple, the decson whether to montor may be correlated wth the player s acton. In our constructon, however, the random varables α n and p n are ndependent, condtonal on the current hstory of length n 1. To facltate the descrpton of the strategy, we ntroduce a real-valued process x = (x n ) n N. The quantty x n s the dscounted value of the future stream of payoffs startng at stage n, ncludng montorng fees at stages m n, wherep m < 1. Player s debt at stage n, D n, ndcates the debt of player at stage n, to be pad by montorng fees n stages m n n whch p m = 1. The actual contnuaton payoff n the repeated game followng the publc hstory h n 1 of length n 1 under the strategy par σ = (σ 1 σ 2 ) s therefore U(σ h n 1 ) = x n D n (4) The process D = (D n ) n N ndcates the amount of money player should burn. We thus requre the followng condton. Condton C1. We have p n = 1 whenever Dn c, for each player and every stage n. Condton C1 means that whenever D n exceeds c,player should burn money by montorng the other player.

17 Theoretcal Economcs 13 (2018) Hgh frequency repeated games 103 The process (α n pn ) 1=1 2;n=1 2 nduces a publc equlbrum f for every stage n and player, the followng Condtons (C2) (C6) are satsfed: Condton C2. We have x n Dn v. Condton C2 ensures that the payoff of each player along the play s ndvdually ratonal: t s at least hs mnmax value. Condton C3. When player j does not play a best response at α n, for every acton a j / supp(α n j ), p n > 2(1 r ) r (E α n a j [x n+1 D n+1 ] v ) where E α n a j [ ] denotes the expected value when, at stage n,player plays the mxed acton α n and player j plays the pure acton a j. As dscussed n Secton 5.2 (see (3)), Condton C3 ensures that, provded an observed devaton trggers an ndefnte punshment at the mnmax level, player j cannot proft by devatng to an acton that s not n the support of α n j. Because G(r c ) s a dscounted game, a par of publc strateges s a Nash equlbrum f the behavor of the players followng every publc hstory that occurs wth postve probablty s an equlbrum n the statc game n whch the payoffs consst of the actual stage payoff plus the contnuaton payoff (the one nduced by σ). The next condtons take care of the ncentve compatble constrants assocated wth ths statc game. Denote by I n the event n whch player montors player j at stage n. Denote by E a NO α n [ ] the expectaton operator when at stage n player plays a, he does not j pn j montor (NO ), whle player j plays α n j and montors wth probablty pn j. The notaton E a O α n [ ] receves an analog nterpretaton wth the dfference that here player does j pn j montor at stage n. Condton C4. If 0 <p n < 1, then for every acton a supp(α n ), x n Dn = (1 r )u (a α n j ) + E [ a NO α n j pn j r (x n+1 D n+1 ] ) c 1 I n = (1 r )u (a α n j ) + E [ a O α n j pn j r (x n+1 D n+1 ] ) c 1 I n Condton C5. If p n = 0, then for every acton a supp(α n ), x n Dn = (1 r )u (a α n j ) + E [ a NO α n j pn j r (x n+1 D n+1 ] ) c 1 I n (1 r )u (a α n j ) + E [ a O α n j pn j r (x n+1 D n+1 ] ) c 1 I n Condton C6. If p n = 1, then for every acton a supp(α n ), x n Dn = (1 r )u (a α n j ) + E [ a O α n j pn j r (x n+1 D n+1 ] ) c 1 I n (1 r )u (a α n j ) + E [ a NO α n j pn j r (x n+1 D n+1 ] ) c 1 I n

18 104 Lehrer and Solan Theoretcal Economcs 13 (2018) Condtons C4 C6 guarantee that no player can proft by an undetectable devaton. Condton C4 states that n case an nspecton has a nontrval probablty, all actons n the support of α n guarantee the same payoff, both when an nspecton takes place and when t does not. Condton C5 states that n case an nspecton occurs wth probablty 0, all actons n the support of α n guarantee the same payoff f an nspecton does not take place and guarantee a lower payoff f an nspecton takes place. Thus, there s no ncentve to montor when the probablty of montorng s 0. Smlarly, Condton C6 states that n case an nspecton occurs wth probablty 1, all actons n the support of α n guarantee the same payoffs f an nspecton takes place, and a lower one f nspecton does not take place. 4 Condtons C4 C6 mply that x n Dn = E p n α n [ (1 r )u (a n ) + r (x n+1 D n+1 ) c 1 I n ] (5) whch guarantees that (4) holds. Indeed, usng (5) recursvely one obtans (compare wth (1)) [ ] x N D N = E σ (1 r )r (n 1) u (a n ) c r (n 1) 1 I n F n 1 (6) n=n where 1 I n s the ndcator of the event I n that player montors at stage n. The rght-hand sde of (6) splayer s payoff n the repeated game, startng at stage N. n=n 5.4 Montorng to deter devatons To better explan the dea of our constructon, we start wth a smple case n whch montorng s performed for one purpose: to deter devatons. In partcular, a burnng-money process s unnecessary n ths case. In Secton 5.5, we handle the general case, whch requres the use of burnng-money processes. Suppose that there are two mxed-acton pars β γ (A 1 ) (A 2 ) that satsfy (see Fgure 4) (a)u 1 (β) < u 1 (γ), (b)u 2 (β) > u 2 (γ), (c)atβ, player1 plays a best response whle player 2 s ndfferent, and (d) at γ, player2 plays a best response whle player 1 s ndfferent. Roughly speakng, we show that any pont n the lne segment between u(β) and u(γ) s an equlbrum pont. For every η>0, letj η be the lne segment that connects the ponts u(β) + (η 2η) and u(γ) + ( 2η η) (see Fgure 4). Assume that the parameters η, r,, c 1, and c 2 satsfy the followng smallness condtons: Condton A1. η> 1 r r ; Condton A2. c 1 (u 1(γ) u 1 (β) η)(r 1 2 ) ; Condton A3. c 2 (u 2(β) u 2 (γ) η)(r 1 2 ) ; Condton A4. r η 2 > 2c 1+r 1+r > 1 r for = 1 2. Condtons A1 A4 hold whenever c 1 c 2 u 1 (γ) u 1 (β) η u 2 (β) u 2 (γ) η. 4 We could state Condtons C4 C6 more concsely. Condton C5 could be requred to hold whenever p n < 1 (nstead of whenever pn = 0) and Condton C6 whenever pn > 0 (nstead of whenever pn = 1). In ths case, Condton C4 would be redundant. We prefer to keep the three condtons as above for expostonal purposes.

19 Theoretcal Economcs 13 (2018) Hgh frequency repeated games 105 Fgure 4. The constructon n the proof of Lemma 4. Lemma 4. Let β γ (A 1 ) (A 2 ) be two mxed-acton profles that satsfy the followng condtons: () We have u 1 (β) < u 1 (γ) and u 2 (β) > u 2 (γ). () At β,player1 plays a best response and player 2 s ndfferent. () At γ,player2 plays a best response and player 1 s ndfferent. Then the set NE(r c 1 c 2 )contans the lne segment J η, provded that the parameters η, r, c 1, c 2,and satsfy Condtons A1 A4. Snce player 1 plays a best response at β, wehavev 1 u 1 (β). Smlarly, v 2 u 2 (γ). In partcular, all the ponts on the lne segment J η are feasble and strctly ndvdually ratonal. The proof of Lemma 4 s relegated to Secton A1 n the Appendx. The key dea of the constructon s the followng. Any pont between u(β) and u(γ) can be obtaned by playng only the mxed actons β and γ throughout the game. When playng β, player 2 may have a proftable devaton, and when playng γ, player1 mayhaveaproftable devaton. To deter devatons, when playng β, player 1 montors player 2 wth small probablty, and when playng γ, player 2 montors player 1 wth small probablty. We set the probablty of montorng n such a way that the expected cost of montorng per stage s η, the dstance between the lne segment [u(β) u(γ)] and J η. The gst of the constructon appears n Fgure 4. Let x n J η and suppose furthermore that x n les on the upper half of the lne segment J η. To support x n as the expected payoff from stage n onward, at stage n the players play the mxed acton β, and player 1 montors player 2 wth probablty p 1. The contnuaton payoff x n+1 depends on whether player 1 montored player 2: fplayer1 montored player 2, hepadthemontorng cost, and hs contnuaton payoff should be hgher than the case when he dd not montor player 2. We thus have to choose two contnuaton payoffs on J η that leave the players ndfferent. The way ths s done and the precse calculatons appear n Secton A1 n the Appendx. In case x n les on the lower half of the lne segment J η,theplayers play n an analogous fashon: they play the mxed acton γ and player 2 montors player 1 wth probablty p 2.