WHEN IS THE LOWEST EQUILIBRIUM PAYOFF IN A REPEATED GAME EQUAL TO THE MINMAX PAYOFF? OLIVIER GOSSNER and JOHANNES HÖRNER

Transcription

1 WHEN IS THE LOWEST EQUILIBRIUM PAYOFF IN A REPEATED GAME EQUAL TO THE MINMAX PAYOFF? BY OLIVIER GOSSNER and JOHANNES HÖRNER COWLES FOUNDATION PAPER NO COWLES FOUNDATION FOR RESEARCH IN ECONOMICS YALE UNIVERSITY Box New Haven, Connectcut

2 Journal of Economc Theory 145 (2010) When s the lowest equlbrum payoff n a repeated game equal to the mn max payoff? Olver Gossner a,b,, Johannes Hörner c a Pars School of Economcs, 48 Boulevard Jourdan, Pars, France b Department of Mathematcs, London School of Economcs, Houghton Street, WC2A 2AE London, Unted Kngdom c Department of Economcs, Yale Unversty, 30 Hllhouse Avenue, New Haven, CT , Unted States Receved 30 March 2007; fnal verson receved 8 July 2009; accepted 8 July 2009 Avalable onlne 30 July 2009 Abstract We study the relatonshp between a player s lowest equlbrum payoff n a repeated game wth mperfect montorng and ths player s mn max payoff n the correspondng one-shot game. We characterze the sgnal structures under whch these two payoffs concde for any payoff matrx. Under an dentfablty assumpton, we further show that, f the montorng structure of an nfntely repeated game nearly satsfes ths condton, then these two payoffs are approxmately equal, ndependently of the dscount factor. Ths provdes condtons under whch exstng folk theorems exactly characterze the lmtng payoff set Elsever Inc. All rghts reserved. JEL classfcaton: C72; C73; D82 Keywords: Folk theorem; Repeated game; Indvdually ratonal payoff; Mn max payoff; Sgnals; Entropy; Condtonal ndependence The authors thank Trstan Tomala and Jonathan Wensten for very useful dscussons, as well as audences at Brown Unversty, the Unversty of Pennsylvana, Yale Unversty, the Unversty of Montréal and McGll Unversty. Suggestons from the edtor, Chrstan Hellwg, sgnfcantly helped mprove the exposton of the paper. They are gratefully acknowledged. * Correspondng author. E-mal address: ogossner@gmal.com (O. Gossner) /$ see front matter 2009 Elsever Inc. All rghts reserved. do: /j.jet

3 64 O. Gossner, J. Hörner / Journal of Economc Theory 145 (2010) Introducton Folk theorems am at characterzng the entre set of payoff vectors that can be attaned at equlbrum n repeated games. That s, the purpose of a folk theorem s to determne whch payoff vectors are, and whch are not, equlbrum payoffs when players are suffcently patent. Whle the hallmark of ths lterature les n the wealth of payoffs that can be supported, t mght not be as well known that, n notable cases, these results only provde lower bounds on the lmt set of equlbrum payoffs, rather than actual characterzatons. For nstance, the folk theorem under mperfect publc montorng [7] asserts that, under some statstcal condtons, every feasble and ndvdually ratonal payoff vector s an equlbrum payoff vector under low dscountng. Indvdual ratonalty refers to the (stage-game, mxed) mn max payoff, defned as mn α max g (a,α ), j A j a A where a A s player s pure acton, α j A j s player j s mxed acton and g s player s payoff functon. That s, the mn max payoff s the lowest payoff player s opponents can hold hm to n the stage game by any choce α of ndependent actons, provded that player correctly foresees α and plays a best-response to t. Yet as Fudenberg, Levne and Maskn [7] acknowledge, n some games, equlbra can be constructed n whch a player s equlbrum payoff s strctly lower than hs mn max payoff. (See Exercse 5.10 n [9] for an llumnatng example.) Ths s because actons are unobserved, so that, f the stage game s such that player s correlated mn max payoff s strctly below hs mn max payoff, players mght be able to use ther prvate hstores to correlate ther actons. 1 Ths paper provdes condtons under whch the mn max payoff provdes a tght bound to the equlbrum set, n repeated games wth mperfect publc, or prvate montorng. Dong so does not merely provde a converse for some of these folk theorems, but also helps understand n whch stuatons there s scope for punshments that are harsher than those usually assumed. Ths s mportant because, to compute the greatest equlbrum payoff for a fxed dscount factor, one must typcally also compute the lowest such payoff. To understand the statstcal requrement under whch the mn max payoff provdes the lower bound on the equlbrum payoff set, we start our analyss wth statc Bayesan games: each player receves a payoff-rrelevant sgnal before choosng hs acton. In ths framework, we characterze the correlaton devces that do not lead to equlbrum payoffs that are strctly worse than the uncorrelated mnmax payoffs. 2 It s rather mmedate to see that a player can always assure hmself of no less than hs uncorrelated mnmax payoff f the sgnal structure s such that ether the other player s sgnals are ndependently dstrbuted condtonal on player s sgnal, or there exsts a garblng of player s sgnal, condtonal on whch the other player s sgnals are ndependently dstrbuted. By condtonng hs actons on such an ndependence-nducng garblng, player ensures hmself aganst the possblty that the other players use ther sgnals to correlate ther actons. We prove 1 The defnton of correlated mn max payoff s obtaned by replacng j A j by j A j as the doman of the mnmzaton n the defnton of the mn max payoff. 2 A complementary problem s to determne the payoff matrces for whch no sgnal structure allows some player to be held down below hs mn max payoff. The answer s rather mmedate, as t amounts to comparng the mn max and the correlated mn max payoff of the payoff matrx. Our queston s motvated by the folk theorems, n whch condtons are dentfed on the sgnal structure that are suffcent for all games.

4 O. Gossner, J. Hörner / Journal of Economc Theory 145 (2010) that the exstence of an ndependence-nducng garblng s not only a suffcent condton, but s also necessary: the exstence of an ndependence-nducng garblng for player characterzes the correlaton structures for whch player cannot be held below hs mn max payoff. In repeated games, studed next, sgnals are nfluenced by actons. Thus, sgnal and acton sets can no longer be treated ndependently. The condton must be modfed: a player s sgnal ncludes now both hs own acton and the actual sgnal he observed. The same condtonal ndependence (for some garblng) requrement guarantees that the (stage-game) mn max payoff and the repeated game mn max payoff the lowest payoff player s opponents can hold hm to by any choce of strateges n the repeated game concde. Ths s not only a feasblty statement, but also an equlbrum statement. Indeed, by a result of von Stengel and Koller [27], the repeated game mn max payoff n a gven game s an equlbrum payoff n the game obtaned by settng the payoff of all players but equal to the opposte of player s payoff n the orgnal game. In ths sense, the result s tght: f the condton s volated, there exsts a payoff matrx for whch the lowest equlbrum payoff s below the stage-game mn max payoff; f t s satsfed, then the lowest equlbrum payoff s greater than, and for some payoff matrces equal to, the mn max payoff. Because a growng lterature examnes the robustness of folk theorems wth respect to small perturbatons n the montorng structures, startng from ether perfect montorng (see [25,6, 2,23,14]), or mperfect publc montorng [18,19], we actually prove a stronger result: as the dstance of the montorng structure to any montorng structure satsfyng the aforementoned condton converges to zero, so does the dstance between the stage-game and repeated-game mn max payoffs. Ths convergence s unform n the dscount factor, provded that the montorng structure satsfes some standard dentfablty assumptons. The condton that s dentfed s by no means mld: as mentoned, there are known examples n games wth publc montorng, where the stage-game and repeated-game mn max payoffs fal to concde. In fact, we provde smple examples to show that ths s possble even when: montorng s almost-perfect; the punshed player perfectly montors hs opponents. But our result mples, for nstance, that the two mn max payoffs are arbtrarly close f montorng s almost-perfect montorng and attenton s restrcted to the canoncal sgnal structure, n whch players sgnals are (not necessarly correct) acton profles of ther opponents. Ths provdes a converse to Theorem 1 of [14]. Our condton also generalzes the varous specal cases for whch t s well known that these two payoffs concde, namely: f there are only two players, as correlated mn max and mn max payoffs then concde; f montorng s perfect, as all players then hold the same nformaton at any pont n tme, so that the probablty dstrbuton over player s opponents actons gven hs nformaton corresponds to ndependent randomzatons by hs opponents; 3 f montorng s publc, but nformaton s sem-standard (as n [16]); f montorng s publc, but attenton s restrcted to publc strateges, as n ths case as well the nformaton relevant to forecastng future play s commonly known. Secton 2 presents examples that motvate the characterzaton. Secton 3 consders statc Bayesan games. Secton 4 extends the analyss to the case of nfntely repeated games. Secton 5 concludes. 3 See, among others, [1,24,8].

5 66 O. Gossner, J. Hörner / Journal of Economc Theory 145 (2010) Fg. 1. The duenna game. 2. The duenna game Many of the deas can be conveyed through a smple example, whch we call the duenna game. 4 Two lovers (players 1 and 2) attempt to coordnate on a place of secret rendezvous. They can ether meet on the landscape garden brdge (B) or at the woodcutter s cottage (C). Unfortunately, the ncorruptble duenna (player 3) prevents any communcaton between them, and wshes to dsrupt ther meetng. Therefore, the rendezvous only succeeds f both lovers choose the same place and the duenna pcks the other place. In all other cases, the duenna exults. The common payoff to the lovers s the probablty of a successful meetng, and the duenna s payoff s the opposte of ths probablty. Fg. 1 dsplays ths probablty, as a functon of the players actons (lovers choose row and column; the duenna chooses the matrx). In the absence of any correlaton devce, players 1 and 2 (the team ) can secure a common payoff of 1/4 by randomzng evenly and ndependently. Ths payoff of 1/4 s also the best equlbrum payoff for the team, as t s an equlbrum that all three players randomze evenly. Yet f the team could secretly coordnate, they could guarantee a probablty of 1/2, by randomzng evenly between (B, B) and (C, C). Now, suppose that ths game s repeated nfntely often, and that montorng s mperfect. Let Ω denote player s (fnte) set of sgnal, wth generc element ω. The dstrbuton of ω := (ω 1,ω 2,ω 3 ) Ω := Ω under acton profle a A s denoted q a, wth margnal dstrbuton on player s sgnal gven by q a. A montorng structure s denoted (Ω, q), where q := {q a : a A}. We examne, for dfferent examples of montorng structures, whether sgnals allow the team to generate some amount of secret correlaton or not. Example 1 (Almost-perfect montorng). Recall from [18] that the montorng structure (Ω, q) s ε-perfect f there exst sgnalng functons f : Ω A for all such that, for all a A, = 1, 2, 3: q a({ }) ω: f (ω ) = a 1 ε. That s, a montorng structure s ε-perfect f the probablty that the acton profle suggested by the sgnal s ncorrect does not exceed ε>0, for all possble acton profles. Let Ω 1 ={ω1 a,ω a 1 : a A}, Ω 2 ={ω2 a,ω a 2 : a A}, Ω 3 ={ω3 a : a A}. Consder q a(( ω1 a )),ωa 2,ωa 3 = q a (( ω 1 a )) 1 ε,ω a 2,ωa 3 =, all a A, 2 4 Ths game, whch appears n varous place n the lterature, s sometmes referred to as the three player matchng pennes game (see [20]). We fnd ths name slghtly confusng, gven that the three person matchng pennes game ntroduced earler by Jordan [15] s a dfferent, perhaps more natural generalzaton of matchng pennes.

6 O. Gossner, J. Hörner / Journal of Economc Theory 145 (2010) where ε>0 s small enough, and set f 3 (ω3 a) = a, f (ω a) = f (ω a ) = a, all = 1, 2 and a A. The specfcaton of the remanng probabltes s arbtrary. Observe that montorng s ε-perfect, snce the probablty that a player receves ether ω a or ω a s at least 1 ε under acton profle a. Yet players 1 and 2 can secure (1 ε)/2 ε 0 1/2, as the dscount factor tends to one. Indeed, they can play B f ω a s observed at the prevous stage, and C f ω a s observed at the prevous stage, ndependently of a. Therefore, even under almost-perfect montorng, the payoff of player 3 n ths equlbrum s bounded away from hs mn max payoff. Example 1 llustrates that the set of equlbrum payoffs under almost-perfect montorng may be bounded away from the one under perfect montorng. In ths example, the set of sgnals s rcher under mperfect prvate montorng than under perfect montorng. Therefore, one may argue that the comparson of the mn max levels across montorng structures s not approprate. In ths example, the natural lmtng montorng structure, as ε 0, should allow for a prvate correlaton devce for players 1 and 2. Indeed, t s the case that the repeated-game mn max payoff s a contnuous functon of the sgnal dstrbuton for fxed sets of sgnals and a fxed dscount factor. But restrctng the set of sgnals does not rule out correlaton, because t also arses under the canoncal sgnal structure n whch Ω = A, as shown by our next example. Nevertheless, our man result mples that, f the montorng structure s almost-perfect and canoncal, then both mn max payoffs concde. Example 2 shows that t s not enough to requre that player 3 have perfect nformaton about hs opponents actons, and/or that the sgnal structure be canoncal. Example 2 (Perfect montorng by player 3, canoncal sgnal structure). Each player s set of sgnals s equal to hs opponents set of actons: Ω = A, for all. Player 3 s nformaton s perfect: q a 3 (a 3) = 1, a A. Player 1 perfectly observes player 2 s acton, and smlarly player 2 perfectly observes player 1 s acton. Ther sgnal about player 3 s acton s ndependent of the acton profle, but perfectly correlated. In partcular: q1 a ( (a2,c) ) = q1 a ( (a2,b) ) = 1/2, q2 a ( (a1,c) ) = q2 a ( (a1,b) ) = 1/2. Consder the followng strateges for players = 1, 2: randomze unformly n the frst perod; afterwards, play C f the last sgnal about player 3 s C, and B otherwse. Ths guarantees 1/2. In ths example, player 1 and 2 s sgnals are unnformatve about player 3 s acton, but t s easy to construct varatons n whch ther sgnals are arbtrarly nformatve, and yet such that the mn max payoff s bounded away from the repeated-game mn max payoff. One may argue that the problem here s that player 3 s sgnal set s not nearly rch enough, as t does not nclude hs opponents sgnal about hs own acton. However, enlargng the sgnal sets takes us back to our ntal example. The ssue s not solved ether by requrng that the player s sgnals be almost publc, a stronger requrement ntroduced and studed n [19]. Indeed, even under publc montorng, the repeatedgame mn max payoff may be lower than the mn max payoff (see [9, Exercse 5.10]).

7 68 O. Gossner, J. Hörner / Journal of Economc Theory 145 (2010) Fg. 2. Condtonal dstrbutons and garbled condtonal dstrbutons. In both examples, players 1 and 2 are able to secretly coordnate n the context of montorng structures close to standard structures for the player that s punshed. The reader may have guessed by now a condton rulng out any such example: f, condtonal on any sgnal of player 3 that has postve probablty for some acton profle, player 1 and 2 s sgnals are ndependent, then players 1 and 2 cannot secretly correlate. Ths ensures that the probablty dstrbuton over player 3 s opponents actons, gven hs nformaton, corresponds to ndependent randomzatons. The next example shows that ths condton s, however, stronger than necessary. Example 3 (A montorng structure wthout condtonal ndependence for whch the repeatedgame mn max payoff equals the stage-game mn max payoff). For each player, Ω ={ω,ω }. Probabltes of sgnal profles are ndependent of the acton profle. Sgnals ω 3 and ω 3 have probablty 1/2. The dstrbuton of player 1 and 2 s sgnals, condtonal on player 3 s sgnal, s gven n Fg. 2 s left panel. Player 1 and 2 s sgnals are not ndependent condtonal on any value of player 3 s sgnal. Yet we clam that, n any game that may be played along wth ths sgnal structure, player 3 guarantees her mn max payoff. Why? Observe that player 3 can always decde to garble hs nformaton, and base hs decson on the garbled nformaton, as summarzed by two fcttous sgnals, ω 3 and ω 3. Upon recevng sgnal ω 3, he can use a random devce selectng ω 3 wth probablty 1/5, and selectng ω 3 otherwse; smlarly, upon recevng sgnal ω 3, he can use a devce selectng ω 3 wth probablty 1/5 and ω 3 otherwse. The rght panel of Fg. 2 shows the dstrbuton of player 1 and 2 s sgnals, condtonal on the value of the garbled sgnal of player 3. Note that player 1 and 2 s sgnal are ndependent, condtonally on any value of player 3 s sgnal. We now show how, respondng to players 1 and 2 s strateges, player 3 can prevent players 1 and 2 from obtanng more than 1/4 n the repeated game. In the frst stage of the repeated game, player 3 plays a best-response to the mxed strateges of players 1 and 2. In the second stage, player 3 can garble the sgnal of the frst stage, and play accordng to ths garbled sgnal only: condtonally on the garbled sgnal, the sgnals of players 1 and 2 n the frst stage are ndependent, and so are ther actons n the second stage. Therefore, by playng a best-response to the dstrbuton of actons of players 1 and 2 n the second stage gven hs garbled sgnal, player 3 ensures that players 1 and 2 do not receve more than 1/4 n the second stage. The constructon extends to any repetton of the game, as Corollary 3 establshes more generally. Ths example shows the connecton between the repeated-game mn max payoff and the exstence of a garblng of player 3 s sgnal satsfyng condtonal ndependence. To dsentangle the role of actons and sgnals, we frst abstract from repeated games and pose our problem as a statc Bayesan game wth exogenous sgnals.

8 O. Gossner, J. Hörner / Journal of Economc Theory 145 (2010) Statc games As the prevous examples show, mperfect montorng may allow a group of players to secretly coordnate ther actons at the expense of another player. In ths secton, we examne when such coordnaton s possble n a statc framework n whch sgnals are exogenous and a game s played only once. Ths allows us to leave asde for now dffcultes that arse due to the dynamc dmenson of repeated games. For nstance, n the repeated game, n each perod, player s opponents must trade off the generaton of such correlaton for future use, and the mmedate use of the exstng correlaton at the expense of player. The man result of ths secton characterzes the dstrbutons of sgnals for whch t s possble, for some payoff functon of a gven player, to drve hs payoff below the mn max payoff. We start out by ntroducng the notons of an nformaton structure q, and of garblngs between these nformaton structures n Secton 3.1. A central noton s that of an ndependence nducng garblng, whch generalzes the dea underlyng Example 3. Garblngs and ndependence nducng garblngs are defned n nformatonal terms only, wthout any reference to payoffs. Turnng to payoffs and strategc notons n Secton 3.2, we defne games extended by an nformaton structure, and compare the mn max payoff for some player between the game extended by the nformaton structure, and the game wthout any nformaton structure. We say that an nformaton structure s mn max preservng f, n every game, the two values concde. In Secton 3.3 we present the man result of ths secton, characterzng the mn max preservng nformaton structures n terms of ndependence nducng garblng of sgnals Informaton structures and ndependence nducng garblngs Gven any measurable space B, B denotes the set of probablty dstrbutons over B, and when B s a subset of a vector space, co B denotes the convex hull of B. Gven a collecton of sets {B }, B denotes the Cartesan product of these sets. When each B has a measurable structure, a product dstrbuton Q over B s one that s obtaned by the product of ts margnals: Q( B) = Q(B ) for any collecton (B ) of measurable sets n B.Thesetof product dstrbutons over B s dentfed wth B. An nformaton structure s gven by a fnte set of sgnals Ω for each player = 1,...,n (n 1) along wth a probablty dstrbuton q over Ω := Ω. We denote such an nformaton structure by q. A garblng (for player ) safamlyp = (p ω ) ω Ω of probabltes over a measurable space X. The nterpretaton s that, upon recevng sgnal ω n the nformaton structure q, player randomly draws a sgnal n X, accordng to the probablty dstrbuton p ω. The probablty q over Ω together wth the garblng p nduces a probablty over Ω X, denoted by p q, and gven by (p q)(ω, S ) = q(ω) p ω (S ), for every ω Ω and measurable set S of X. Defnton 1. An ndependence nducng garblng of q for player s a garblng of q such that, almost surely wth respect to the garbled sgnal x, the dstrbuton of sgnals of players other than gven x s a product dstrbuton: (p q)( x) j Ω j a.s.

9 70 O. Gossner, J. Hörner / Journal of Economc Theory 145 (2010) That s, p s ndependence nducng whenever, condtonal on player s garbled sgnal, other players sgnals are ndependently dstrbuted. In Example 3, the devce that randomly selects ω 3 or ω 3 from ω 3 or ω 3 s an ndependence nducng garblng. Note that the exstence of an ndependence nducng garblng for player s a property of the nformaton structure q only. In partcular, t does not nvolve payoffs Mn max preservng nformaton structures A normal form game G specfes a set of players j = 1,...,n; for each player j, a fnte acton set A j, and a payoff functon g j : A := j A j R. We sngle out some player, and all payoff functons for players j are rrelevant for our purposes. We let S j := A j be the set of mxed strateges of player j, and extend g to A (and n partcular to S := j S j )usngthe expectaton n the usual way. The mn max payoff of player n the game G (wthout sgnals) s: v (G) = mn α max g (a,α ). (1) j A j a A Gven a game G and an nformaton structure q, we denote by Γ(q,G) the game obtaned from G by adjonng the sgnal structure q (where the set of sgnals s gven by the doman of q). A (behavoral) strategy σ j Σ j for player j n Γ(q,G)s a mappng from Ω j (j s set of sgnals n q) to A j. A profle of strateges σ = (σ j ) j nduces, for each ω Ω, a profle of mxed strateges σ(ω)= (σ j (ω j )) j j A j.toσ corresponds the payoff for player n Γ(q,G), ( ) γ (σ ) = E q g σ(ω), where E q denotes the expectaton under the probablty dstrbuton q. The mn max for player n Γ(q,G)s thus V (q, G) = mn σ max γ (σ ). j Σ j σ Σ For every game G and nformaton structure q, V (q, G) v (G). To see ths, consder a profle (α ) j that acheves the mnmum n the defnton of v and the strateges σ j for j gven by σ j (ω j ) = αj ndependently of ω j. A best-response of player aganst these strateges yelds exactly v (G) to player. We are nterested n characterzng the nformaton structures q that gve scope to strategc correlaton between players j, n the sense that they allow for a punshment of player that s strctly lower than v (G). Of course, whether V (q, G) < v (G) or V (q, G) = v (G) s not merely a property of q, as t also depends on G. For nstance, V (q, G) = v (G) for every q whenever g s constant. Hence, the approprate noton of an nformaton structure that gves rse to strategc correlaton s the one requrng that V (q, G) < v (G) for some game G. Ifthe nformaton structure q does not gve rse to such correlaton, the mn max n G and n Γ(q,G) concde for all games G. Ths dea s captured by the followng defnton. Defnton 2. An nformaton structure q s mn max preservng (for player ) whenever, for every game G, V (q, G) = v (G). Thus, f q s not mn max preservng, there exsts a game G n whch the mn max payoff to player s strctly lower when players have access to sgnals dstrbuted accordng to q.

10 O. Gossner, J. Hörner / Journal of Economc Theory 145 (2010) The equvalence result We characterze the strategc noton of a mn max preservng nformaton structure usng the nformatonal noton of an ndependence nducng garblng. Theorem 1. An nformaton structure q s mn max preservng for player f and only f there exsts an ndependence nducng garblng of q for player. Proof. If part. Let p = (p ω ) ω be an ndependence nducng garblng of q for player, and let G be a normal form game. Consder strateges σ = (σ j ) j for players j n Γ(q,G). For every x X such that (p q)( x) belongs to j Ω j,letτ(x) S be such that g (τ(x), (p q)( x)) v. We defne a response σ for player to σ by σ (ω ) = E pω τ(x). The strategy σ conssts n frst applyng the garblng p ω to player s sgnal, then choosng the acton accordng to τ, gven the resultng garbled sgnal. We now verfy that σ allows player to defend v (G) aganst σ : γ (σ,σ ) = E q g ( σ(ω) ) ( = E ω E x g σ (ω ), τ (x) ) ( = E x E (p q)( x) g σ (ω ), τ (x) ) v (G). Only f part. Gven q Ω, letω be a random varable wth law q. Player s belef on ω gven sgnal ω s q(ω ω ), whch we vew as a random varable wth values n Ω.We let β q denote the dstrbuton of ths random varable (note that β q depends only on q, not on the partcular choce of random varable ω). Ths dstrbuton β q s the dstrbuton of belefs of player about the other players sgnals nduced by q. It characterzes the nformaton about ω contaned n ω. Assumng that q s mn max preservng, we prove the exstence of an ndependence nducng garblng p of q havng the further property that the garbled sgnal s dentfed wth the condtonal dstrbuton of sgnals of players gven player s garbled sgnal. Let M := j Ω j be the set of product dstrbutons on Ω = j Ω j. We defne an M-garblng of q as a garblng p wth M as set of garbled sgnals and such that (p q)(ω m) = m a.s. n m. Obvously, any such M-garblng s ndependence nducng. Gven some M-garblng p of q, let μ p denote the dstrbuton of the garbled sgnal,.e. the margnal of p q on M. Ths represents the dstrbuton of belefs on ω of some hypothetcal agent nformed of m, but not of ω. To prove the exstence of an M-garblng, we rely on the followng result, whch s a characterzaton of more nformatve experments àlablackwell [3]. Lemma 1. Let q be an nformaton structure and μ M. There exsts an M-garblng p of q such that μ p = μ f and only f, for every bounded convex functon ψ on Ω, E βq ψ E μ ψ.

11 72 O. Gossner, J. Hörner / Journal of Economc Theory 145 (2010) Ths lemma s an extenson of the theorem of [3] to the nfnte-dmensonal case. See [4,26]. We are now n poston to complete the proof of Theorem 1. Endow A := M, subset of the set of dstrbutons over D = Ω, wth the weak topology. Let B be the set of contnuous convex functons on D bounded by 0 and 1, endowed wth the unform topology. Both A and B are compact convex sets. Assume that q admts no M-garblng for player. By Lemma 1, μ A, ψ B such that E βq ψ<e μ ψ. (2) The map g: A B R gven by g(μ,ψ) = E βq ψ E μ ψ s b-lnear and contnuous, so, by the mn max theorem, the two-player zero-sum game n whch player I s acton set s A, player II s acton set s B and the payoff to I s gven by g has a value v, and (2) mples v<0. There exsts an optmal strategy for player II, whch s ψ B such that μ A, E βq ψ E μ ψ + v. In partcular, m M, E βq ψ ψ(m)+ v. (3) Let ψ = ψ E βq ψ + v/2. We have E βq ψ = v/2 < 0, and (3) mples that, for every m M, ψ (m) v/2 > 0. For every m M, the convexty of ψ mples the exstence of a lnear map φ m on D such that φ m (m) > 0 and φ m ψ.leto m be an open neghborhood of m such that φ m > 0onO m. Snce M s a compact subspace of D and (O m ) m M s a coverng of M, there exsts a fnte subcoverng (O m ) m M0 of M. We use the famly (φ m ) m M0 to construct a game G showng that q s not mn max preservng. In G, each player j has strategy set Ω j, player has strategy set M 0, and s payoff functon g s defned by g (m 0,ω ) = φ m0 (ω ). Let m M be a profle of mxed strateges for players j.form 0 M 0 such that m O m0, E m g (m 0,ω ) = φ m0 (m) > 0. Hence, v (G) = mn max g (m 0,m)>0. m M m 0 M 0 Now consder G extended by q, and the strateges for players j that specfy as actons n G ther sgnal n q. Gven a sgnal ω, a best-response of player yelds an expected payoff of ( max φ m0 q(ω ω ) ). m 0 M 0 Thus, a best-response strategy for player yelds an expected payoff n Γ(q,G)of ( E q max φ m0 q(ω ω ) ) E q ψ ( q(ω ω ) ) m 0 M 0 = E βq ψ < 0, whch shows that V (q, G) < 0. We have thus shown that f q admts no M-garblng, q s not mn max preservng.

12 O. Gossner, J. Hörner / Journal of Economc Theory 145 (2010) Repeated games wth mperfect montorng Theorem 1 suggests that the exstence of an ndependence nducng garblng s the approprate condton for our purposes. However, n the repeated game, the problem s slghtly more complcated. After a gven hstory, the prvate nformaton of a player (what we called hs sgnal n the prevous secton) has two components: the actons that he has taken so far, and the actual sgnal that he has observed n each perod. Therefore, for our purposes, a sgnal n a gven perod s such a par. It s clear, however, that, unlke n the prevous secton, the dstrbuton of ths par s no longer exogenous, both because t contans a player s own past acton, whch he chose, and a sgnal whose dstrbuton hs choces of acton affected. Snce there are as many dstrbutons as acton profles, the condton must be strengthened to the exstence of a garblng provdng condtonal ndependence, for each possble acton profle, whether pure or mxed. 5 The suffcency of ths condton s establshed n Theorem 2. Second, n many applcatons, condtonal ndependence need not hold exactly (consder, for nstance, a montorng structure that s almost-perfect, but not perfect). Therefore, we wsh to allow for small departures from condtonal ndependence, whch complcates the analyss, especally snce we am for a bound that s unform n the dscount factor. Snce even small departures from condtonal ndependence may allow patent players to accumulate secret correlaton over tme (as we show n Example 4 below), such unformty can only be acheved f player s opponents necessarly dsspate ths correlaton whenever they take advantage of t. Theorem 4, whch s the man result of ths secton, formalzes ths logc. Recall that a stage game G specfes a (fnte) acton set A j for each player = 1,...,n and, for each player, a payoff functon g : A := j A j R. We restrct attenton to games G for whch g (a) 1 for every player and acton profle a (the specfc choce of the upper bound s obvously rrelevant). Players can use mxed actons α A. Mxed actons are unobservable. No publc randomzaton devce s assumed, and there s no communcaton. We consder the nfntely repeated game, denoted G. Perods are ndexed by t = 0, 1,... In each perod, player observes a prvate sgnal ω from some fnte set Ω, whose dstrbuton depends on the acton profle beng played. Therefore, player s nformaton acqured n perod t conssts of both hs acton a and hs prvate sgnal ω.lets = (a,ω ) denote ths nformaton, or sgnal for short, and defne S := A Ω. The montorng structure determnes a dstrbuton over prvate sgnals for each acton profle. For our purpose, t s more convenent to defne t drectly as a dstrbuton over S := S 1 S n. Gven acton profle a A, q a (s) denotes the dstrbuton over sgnal profles s = (s 1,...,s n ). We extend the doman of ths dstrbuton to mxed acton profles α A, and wrte q α (s). Letq α denote the margnal dstrbuton of q α over player s sgnals s, and gven s S and α A such that q α(s )>0, let q α ( s ) denote the margnal dstrbuton over hs opponents sgnals, condtonal on player s sgnal s. From now on, a montorng structure refers to such a famly of dstrbutons q. Players share a common dscount factor δ (0, 1), but as wll be clear, ts specfc value s rrelevant (statements do not requre that t be suffcently large). Repeated game payoffs are dscounted, and ther doman s extended to mxed strateges n the usual fashon; unless explctly mentoned otherwse (as wll occur), all payoffs are normalzed by a factor 1 δ. Recall that player s mn max payoff v (G) n G s defned by Eq. (1). 5 Condtonal ndependence of sgnals for each pure acton profle does not mply condtonal ndependence for all mxed acton profles.

13 74 O. Gossner, J. Hörner / Journal of Economc Theory 145 (2010) A prvate hstory of length t for player s an element of H t := St (let H 0 := { }). A (behavoral) prvate strategy for player s a famly σ = (σ t) t, where σ t: H t 1 A. We denote by Σ the set of these strateges. Bearng n mnd the earler dscussons regardng the omsson of any equlbrum consderaton here, we defne player s ndvdually ratonal payoff v δ n the repeated game as the lowest payoff he can be held down to by any collecton σ = (σ j ) j of ndependent choces of strateges n the repeated game, provded that player correctly foresees σ and plays a best-reply to t. Formally, the ndvdually ratonal payoff (or mn max payoff n the repeated game) s defned as v δ := mn σ j Σ j max σ Σ E σ (1 δ)δ t ( g a t,a ) t. t=0 It s straghtforward to see that, for any game G and montorng structure q, the ndvdually ratonal payoff does not exceed the mn max payoff: v (G) v δ. Ths s a consequence of the fact that, n the repeated game, players can repeatedly play a profle of mxed strateges that acheves the mnmum n the defnton of v (G). We wsh to dentfy condtons under whch the ndvdually ratonal payoff and the (stagegame) mn max payoff concde, or are close to one another. The frst condton we defne s that of condtonal ndependence. Defnton 3. A montorng structure q satsfes condtonal ndependence for player f, for every profle of mxed strateges α j A j, player s sgnals are ndependent condtonal on player s sgnal: s S such that q α (s )>0, q α ( s ) j S j. Theorem 2. If the montorng structure satsfes condtonal ndependence for player, then, for all δ [0, 1), player s ndvdually ratonal payoff s equal to hs mn max payoff. Ths result s proved n the next secton, as an mmedate consequence of the frst step of the proof of Theorem 4. To state the next mportant but straghtforward extenson of Theorem 2, one must, n the sprt of Secton 3, ntroduce the noton of an ndependence nducng garblng of a montorng structure. Defnton 4. An ndependence nducng garblng of a montorng structure q for player s a garblng p such that p s an ndependence nducng garblng of the nformaton structure q α,for every profle of mxed strateges α. Theorem 2 mples the followng: Corollary 3. If the montorng structure admts an ndependence nducng garblng for player, then, for all δ [0, 1), player s ndvdually ratonal payoff s equal to hs mn max payoff. Observe that ths corollary generalzes Theorem 2, as f q satsfes condtonal ndependence for player, t automatcally admts an ndependence nducng garblng for player.

14 O. Gossner, J. Hörner / Journal of Economc Theory 145 (2010) Another stuaton n whch an ndependence nducng garblng exsts s when f q α s a product dstrbuton, that s, f sgnals of players j are ndependently dstrbuted, because player can always gnore hs sgnal, and play a best-reply to hs pror belef. Ths may occur for dstrbutons n whch q does not satsfy condtonal ndependence for player. In fact, Corollary 3 encompasses a set of stuatons that s much rcher than those specal cases. However, whle t s straghtforward to check whether a montorng structure satsfes condtonal ndependence for any gven player, we do not know of a smple algorthm allowng to ascertan whether a montorng structure admts an ndependence nducng garblng for ths player. For a gven dscount factor, snce the payoff functon n the repeated game s contnuous n the montorng structure, the ndvdually ratonal payoff s also contnuous n the montorng structure. In partcular, f q almost satsfes condtonal ndependence for player, or almost admts an ndependence garblng for player, then the ndvdually ratonal payoff for player s approxmately equal to s mn max payoff. Such a result s unsatsfactory because t does not rule out that, for such montorng structures, there may exst a dscount factor, suffcently close to one, for whch the ndvdually ratonal payoff s bounded away from the mn max payoff. Intutvely, montorng structures that almost admt an ndependence nducng garblng may stll provde small amounts of correlaton to player s opponents. Over tme, these small amounts may accumulate, allowng them to successfully coordnate ther play eventually. The next example llustrates ths possblty. Example 4 (A montorng structure that almost satsfes ndependence). The payoff matrx s gven by the duenna game. Player 1 and 2 s sgnal set each has two elements, Ω ={ω,ω }. Player 3 receves no sgnal. The dstrbuton of player 1 and 2 s sgnals s ndependent of the acton profle, and perfectly correlated. Wth probablty ɛ>0, the sgnal profle s (ω 1,ω 2 ), and t s equal to (ω 1,ω 2 ) wth probablty 1 ɛ>0. Gven ɛ>0, let H T denote the set of prvate hstores of sgnals of length T for players = 1, 2, and let H,T denote the subset of H T consstng of those hstores n whch the observed number of sgnals ω exceeds the expectaton of ths number, Tɛ. Observe that, by the central lmt theorem, the probablty that a hstory of length T s n H T tends to 1/2 ast. Defne σ T as the strategy consstng n playng C for the frst T perods and n playng C forever after, f the prvate hstory s n H,T, and B f t s not. The payoff to players 1 and 2 from usng (σ1 T,σT 2 ) when player 3 plays a best-response tends to 1/2 asδ 1, T provded δ T 1. Ths shows that, for any value of ɛ>0, when δ 1, equlbrum payoffs exst that approach 1/2 for players 1 and 2. On the other hand, for any fxed δ, players 1 and 2 cannot secure more than 1/4 as ɛ 0. As Example 4 shows, the order of lmts s mportant n general. Whle the set of payoffs s contnuous n the montorng structure for a fxed dscount factor, the lmt of ths set as the dscount factor tends to one may be dscontnuous n the montorng structure. Our next result, Theorem 4, shows that such cases are ruled out when player s sgnals allow to statstcally dscrmnate among acton profles of the other players.

15 76 O. Gossner, J. Hörner / Journal of Economc Theory 145 (2010) Defnton 5 (Identfablty). The montorng structure q satsfes dentfablty for player f, for all a n A and α n A, q a,α / co { q a,α : a a, a A }. That s, q satsfes dentfablty f, for any possbly mxed acton of player, the dstrbuton over hs sgnals that s generated by any pure acton profle of hs opponents cannot be replcated by some convex combnaton of other acton profles of thers. Let d denote the total varaton dstance between probablty measures. Gven ρ>0, the montorng structure q satsfes ρ-dentfablty for player f, for all a n A and α n A, and any dstrbuton q n co{q a,α : a a, a A }, d ( q a,α,q ) >ρ. Thus, the concept of ρ-dentfablty captures the dstance between the montorng structure q and the nearest one that fals to satsfy dentfablty. Fnally, we need to ntroduce a measure of the dstance between a montorng structure and the nearest one that satsfes condtonal ndependence for player. Forε>0, the montorng structure q s ε-dependent for player f, for all acton profles α n j A j, there exsts a famly of product dstrbutons (q (s )) s n j S j such that E [ d ( q α ( s ), q (s ) )] <ε. That s, q s ε-dependent for player f those sgnals for whch the condtonal dstrbuton of player s not close to a product dstrbuton are suffcently unlkely, gven any acton profle that corresponds to ndependent randomzatons. In the sequel, when there s no ambguty as to whch player s consdered, we drop the reference to player when usng the expressons ρ-dentfablty or ε-dependence. Theorem 4. For any ν>0,fq satsfes ρ-dentfablty, for some ρ>0, there exsts ε>0 such that, f q s ε-dependent, then, for all δ [0, 1), player s ndvdually ratonal payoff s wthn ν of hs mn max payoff. Theorem 4 strengthens Theorem 2 and provdes a contnuty result that s unform n the dscount factor. Ths theorem s mportant for the lterature on the robustness of equlbra for prvate montorng structures that are n a neghborhood of perfect, or mperfect publc montorng. Indeed, whle almost-perfect montorng structures need not be ε-dependent for small ε (as expected gven Example 1), t s mmedate to see that they must be f attenton s restrcted to canoncal sgnal structures. Therefore, Theorem 4 provdes a converse to Theorem 1 n [14]. Theorem 4 extends to dstrbutons for whch there exsts a garblng that nduces an approxmate verson of ndependence, provded that the garbled sgnal satsfes the dentfablty condton (that s, the belef of player, condtonal on hs garbled sgnal, should satsfy ρ- dentfablty). The generalzaton s straghtforward and omtted. Note also that the dentfablty condton used n Theorem 4 can be weakened. Indeed, we do not need that each acton of player allows for statstcal dscrmnaton of hs opponents actons. Rather, t s enough that, for each α α, there exsts one acton of player that dscrmnates between them. To prove ths result, t s enough to consder strateges of player that play each acton wth probablty at least ɛ>0ateach stage, then let ɛ 0.

16 O. Gossner, J. Hörner / Journal of Economc Theory 145 (2010) Fnally, a large lterature has consdered a restrcted class of strateges, namely publc strateges, n the context of games wth publc montorng. In such games, the mn max payoff n publc strateges n the repeated game cannot be lower than the statc mn max payoff, a result whch s not generally true wthout the restrcton on strateges (see agan [9, Exercse 5.10]). It s then natural to ask to what extent the ε-dependence assumpton can be weakened for such a class of strateges. To be specfc, assume that strateges must be a functon of the hstory of prvate sgnals ω alone, rather than of the hstory of all sgnals s. Observe that ths reduces to publc strateges n the case of publc montorng, but s well-defned even under prvate montorng. Then Theorem 4 remans vald, f we requre that only the restrcton of the montorng structure to prvate sgnals be ε-dependent. Ths s a sgnfcantly weaker restrcton, whch s ndeed trvally satsfed f montorng s publc. The proof s a trval modfcaton of the proof of Theorem 4. All statements are ether trval or follow from the proof of Theorem 4. The proof of ths theorem s rather delcate can be found n Appendx A. The frst part of the proof, presented n Secton A.1, reduces the study to a repeated game wth an alternate montorng structure n whch the sgnals to players s publc among these players (and only among them), and these players are restrcted to publc strateges (dependng only on past realzatons of these publc sgnals). When studyng the repeated game wth the alternate montorng structure, tools from nformaton theory are brought to bear. Ths s done n Secton A.2. There we show that, under ε-epslon dependence and ρ-dentfablty, t takes tme to accumulate suffcent publc nformaton for player s opponents to successfully correlate ther acton profle, relatve to the tme t takes player to detect whch of the plays hs opponents have coordnated upon. 5. Concludng comments In ths paper, we provde the necessary and suffcent condton on the nformaton structure for whch the lowest equlbrum payoff n any Bayesan game assocated wth ths nformaton structure s no lower than the mn max level determned by the payoff matrx only. Ths provdes a suffcent condton under whch the stage-game mn max payoff s the approprate lower bound on possble equlbrum payoffs n a repeated game, whether the montorng s mperfect or not. We also show under whch condtons ths remans approxmately true when the montorng structure s arbtrarly close to one that satsfes ths condton. Ths provdes a condton under whch a converse to the folk theorems whch can be found n [7,14] hold. An mportant queston left open s how to actually determne the repeated-game mn max payoff when t s below the stage-game mn max payoff. Characterzatons are only known for some classes of montorng structures n [16,11,12]. When condtonally on each player s sgnal, other player s sgnals are ndependent, the equlbrum payoff set possesses a natural recursve structure, and methods from dynamc programmng can be brought to bear. Wth three players or more, the paper [10] characterzes the nformaton structures for whch condtonal on each player s sgnal, other player s sgnals are ndependent. Those are the nformaton structures such that all player s sgnals are ndependent condtonal on an underlyng common-knowledge varable. The more general queston of the characterzaton of montorng structures whch admt condtonal ndependence garblngs for every player, case n whch each player s ndvdually ratonal payoff s gven by hs mn max payoff n the stage game, s left for future research.

17 78 O. Gossner, J. Hörner / Journal of Economc Theory 145 (2010) Appendx A. Proof of Theorem 4 The proof of Theorem 4 s dvded n two parts. In the frst part, we replace the prvate montorng structure by another one. Player s nformaton s unchanged. Hs opponents nformaton s now publc among them, but t s not smply the nformaton resultng from poolng ther ndvdual sgnals from the orgnal montorng structure: dong so would enable them to correlate ther play n many stuatons n whch they would not be able to do so f ther strategy were only based on ther own sgnals. The common nformaton must be poorer than that, but we stll need to make sure that any probablty dstrbuton over plays that could be generated n the orgnal montorng structure by some strategy profle of player s opponents (for some strategy of player ) can stll be generated n ths alternate montorng structure. We shall refer to players as the team, and t wll be understood that ther objectve s to mnmze player s payoff. A.1. Reducton to publc strateges A result that wll prove useful n the sequel s the followng. Lemma 2. If q s a dstrbuton over some product fnte set S := k K S k, then there exsts a product dstrbuton p j S j and a resdual dstrbuton r such that q = λp + (1 λ)r, for some λ = λ(q) n [0, 1]. Further, for every ν>0, there exsts ε>0 such that, f d(q,q )<ε for some q j S j, then we can choose λ>1 ν. Proof. Indeed, f we defne λ as the supremum over all such values n the unt nterval for whch we can wrte q as a convex combnaton of dstrbutons p and r, wth p j S j, t follows from the maxmum theorem that () ths maxmum s acheved, () t s contnuous n q. In fact, snce q belongs to a compact metrc space, ths contnuty s unform, by the Hene Cantor theorem. The result follows, snce λ = 1fq j S j. Gven ths result, we can vew sgnals n the repeated game as beng drawn n three stages. Gven the acton profle (α,a ): frst, the sgnal s s drawn accordng to the margnal dstrbuton q α,a.gvens, apply (2) and wrte q α,a ( s ) = λp α,a ( s ) + (1 λ)r α,a ( s ), where λ depends on q α,a ( s ); second, a Bernoull random varable l wth P{l = 1}=1 P{l = 0}=λ s drawn; thrd,fl = 0, the sgnal profle s s drawn accordng to r α,a s drawn accordng to p α,a ( s ). ( s ); f nstead l = 1, s We now use ths representaton to show that player s ndvdually ratonal payoff s no larger under the orgnal montorng structure than under an alternate montorng structure n whch player s opponents can condton ther strategy on the hstory of values of s, l, and, whenever l = 0, of s. Ths s non-trval because player s opponents are not allowed to condton ther

18 O. Gossner, J. Hörner / Journal of Economc Theory 145 (2010) strategy on ther own sgnals any longer, unless λ = 0. Yet the concluson s rather ntutve, for when λ = 1, the sgnals of player s opponents are ndependently dstrbuted anyway (condtonal on s ). Ths result wll allow us n the next subsecton to vew the hstores used by player s opponents as common. Before statng the result, further notaton must be ntroduced. Hstores. Hstores Recall that a hstory of length t n the orgnal game s an element of S t.the set of plays s H = S N endowed wth the product σ -algebra. We defne an extended hstory of length t as an element of (S {0, 1} S ) t, that s, as a hstory n the orgnal game augmented by the hstory of realzatons of the Bernoull varable l. The set of extended plays s H = (S {0, 1}) N endowed wth the product σ -algebra. A prvate hstory of length t for player j (n the orgnal game) s an element of Hj t = St j. A publc hstory of length t s an element of Hp t := St p, where S p := S {0} S S {1}; that s, S p s the set of publc sgnals s p, where s p = (s, 0,s ) f l = 0 and s p = (s, 1) f l = 1. Strateges. A (behavoral) prvate strategy for player j (n the orgnal game) s a famly σ j = (σj t) t, where σj t: H j t 1 A j.letσ j denote the set of these strateges. A (behavoral) publc strategy for player j s a famly τ j = (τj t ) t, where τj t : H p t 1 A j.letσ p,j denote the set of these strateges. Fnally, a (behavoral) general strategy for player j s a famly σ j = ( σ j t) t, where σ j t: (S p S j ) t 1 A j.let Σ j denote the set of these strateges. Note that both Σ p,j and Σ j can naturally be dentfed as subsets of Σ j,butσ p,j and Σ j cannot be ordered by set ncluson. A (pure) strategy for player s a famly σ = (σ t) t, where σ t: St 1 A. Any profle of general strateges σ for player s opponents, together wth a strategy σ for player, nduces a probablty dstrbuton P σ,σ on H. Proposton 1. For any prvate strategy profle σ, there exsts a publc strategy profle τ such that, for every pure strategy σ, h t St, st+1 S, ( ) ( ) P τ,σ h t = Pσ,σ h t, ( P τ,σ s t+1 h t ( ) = Pσ,σ s t+1 h t ) f P σ,σ ( h t ) > 0. (A.1) (A.2) Proof. We frst defne a publc strategy up to stage t for player j as a famly τ t,j = (τt,j t ) t where { τ t t,j : St 1 p A j, f t t; τt,j t : St 1 p S t t+1 j A j, otherwse. The proof of the proposton reles on the followng lemma. Ths lemma exhbts a sequence of strategy profles for player s opponents, up to stage t, based on σ, that do only depend on the frst t publc sgnals, and not on the realzatons of the frst t prvate sgnals (condtonal on these publc sgnals). Ths sequence of strateges s constructed by terated applcatons of Kuhn s theorem, as shown n the next lemma. Lemma 3. For any prvate strateges σ, there exst strateges (τ t, ) t = (τ t,j ) j,t where τ t,j s a publc strategy up to stage t for player j and