Individual Learning and Cooperation in Noisy Repeated Games

Transcription

1 Indvdual Learnng and Cooperaton n Nosy Repeated Games Yuch Yamamoto July 6, 2013 Abstract We nvestgate whether two players n a long-run relatonshp can mantan cooperaton when the detals of the underlyng game are unknown. Specfcally, we consder a new class of repeated games wth prvate montorng, where an unobservable state of the world nfluences the payoff functons and/or the montorng structure. Each player prvately learns the state over tme but cannot observe what the opponent learned. We show that there are robust equlbra n whch players eventually obtan payoffs as f the true state were common knowledge and players played a belef-free equlbrum. We also provde explct equlbrum constructons n varous economc examples. Journal of Economc Lterature Classfcaton Numbers: C72, C73. Keywords: repeated game, prvate montorng, ncomplete nformaton, belef-free equlbrum, ex-post equlbrum, ndvdual learnng. Ths work s based on the frst chapter of my Ph.D. dssertaton at Harvard Unversty. I am grateful to my advsors, Attla Ambrus, Susan Athey, and especally Drew Fudenberg, for encouragement and extensve dscussons. I also thank Rcardo Alonso, Dasuke Hrata, George Malath, Tomasz Strzaleck, a co-edtor, anonymous referees, and semnar partcpants at varous places for nsghtful comments. Department of Economcs, Unversty of Pennsylvana. Emal: yyam@sas.upenn.edu 1

2 1 Introducton Consder an olgopolstc market where frms sell to ndustral buyers and nteract repeatedly. The prce and the volume of transacton n such a market are typcally determned by blateral negotaton between a seller and a buyer so that both prce and sales are prvate nformaton. In such a stuaton, a frm s sales level can be regarded as a nosy prvate sgnal about the opponents prce, as t s lkely to drop f the opponents (secretly) undercut ther prce. 1 Ths s the so-called secret prcecuttng game of Stgler (1964) and s a leadng example of repeated games wth mperfect prvate montorng where players receve nosy prvate sgnals about the opponents actons n each perod. 2 Recent work has shown that a long-term relatonshp helps provde players wth ncentves to cooperate even under prvate montorng. 3 However, all the exstng results rely heavly on the assumpton that players know the exact dstrbuton of prvate sgnals as a functon of the actons played, whch s not approprate n some cases. For example, when frms enter a new market, ther nformaton about the market structure s often lmted, and hence they may not know the dstrbuton of sales levels as a functon of ther prce. How does the uncertanty about the market structure nfluence decson makng by the frms? Do they have ncentves to sustan colluson even n the presence of such uncertanty? Motvated by these questons, we develop a general model of repeated games wth prvate montorng and unknown montorng structure. Formally, we consder two-player repeated games n whch the state of the world, chosen by Nature at the begnnng of play, nfluences the dstrbuton of prvate sgnals of the stage game. Snce players do now observe the true state, they do not know the dstr- 1 Harrngton and Skrzypacz (2011) report that these propertes are common to the recent lysne and vtamn markets. 2 Other examples nclude relatonal contracts wth subjectve evaluatons (Levn (2003) and Fuchs (2007)) and nternatonal trade agreements n the presence of concealed trade barrers (Park (2011)). 3 For example, effcency can be approxmately acheved n the prsoner s dlemma when observatons are nearly perfect (Sekguch (1997), Bhaskar and Obara (2002), Hörner and Olszewsk (2006), Chen (2010), and Malath and Olszewsk (2011)), nearly publc (Malath and Morrs (2002), Malath and Morrs (2006), and Hörner and Olszewsk (2009)), statstcally ndependent (Matsushma (2004)), or even fully nosy and correlated (Fong, Gossner, Hörner, and Sannkov (2011) and Sugaya (2010b)). Kandor (2002) and Malath and Samuelson (2006) are excellent surveys. See also Lehrer (1990) for the case of no dscountng and Fudenberg and Levne (1991) for the study of approxmate equlbra wth dscountng. 2

3 buton of sgnals n ths setup. The state can affect the payoff functons ndrectly through the effect on the dstrbuton of sgnals. For example, n a secret prcecuttng game, frms obtan hgher expected payoffs at a gven prce at states where hgh sales are lkely. Thus, even f the payoff to each sales level s known, uncertanty about the dstrbuton of sales yelds uncertanty about the expected payoffs of the stage game. In our model, players update ther belefs about the true state each perod through observed sgnals. Snce these sgnals are prvate nformaton, players posteror belefs about the true state need not concde even f they have a common pror at the begnnng of the game. In partcular, whle each player may learn the true state from observed sgnals over tme, ths learnng process may not lead to common learnng n the sense of Crpps, Ely, Malath, and Samuelson (2008); that s, the true state may not become approxmate common knowledge even n the long run. For example, n the context of secret prce-cuttng, each frm prvately learns the true dstrbuton of sales from ts own experence, but ths dstrbuton may not become approxmate common knowledge, e.g., a frm may beleve that the rval frm has a dfferent belef about the dstrbuton of sales. Such a possblty may hurt players wllngness to coordnate. Another ssue n our model s that players may strategcally conceal what they learned n the past play; snce players learn the state from prvate sgnals, they can conceal ther nformaton by pretendng as f they observed somethng dfferent from the actual sgnals. The man fndng of ths paper s that despte these potental complcatons, players can stll mantan some level of cooperaton through approprate use of ntertemporal ncentves. Snce there are nfntely many perods, keepng track of the evoluton of players belefs s ntractable, and thus characterzng the entre equlbrum set s not an easy task n our model. To avod ths dffculty, we look at a tractable subset of Nash equlbra, called belef-free ex-post equlbra, or BFXE. Ths allows us to obtan a clean characterzaton of the equlbrum payoff set; and n addton we show that a large set of payoffs (ncludng Pareto-effcent outcomes) can be acheved by BFXE n many economc examples. A strategy profle s a BFXE f ts contnuaton strategy consttutes a Nash equlbrum gven any state and gven any hstory. In a BFXE, a player s belef about the true state s rrelevant to her best reply, and hence we do not need to 3

4 track the evoluton of these belefs over tme. Ths dea s an extenson of expost equlbra of statc games to dynamc settng. Another mportant property of BFXE s that a player s best reply does not depend on her belef about the opponent s prvate hstory, so that we do not need to compute these belefs as well. Ths second property s closely related to the concept of belef-free equlbra of Ely, Hörner, and Olszewsk (2005, hereafter EHO), whch are effectve n the study of repeated games wth prvate montorng and wth no uncertanty. Note that BFXE reduce to belef-free equlbra f the state space s a sngleton so that players know the structure of the game. As shown by past work, most of belef-free equlbra are mxed strateges, and players randomzaton probabltes are carefully chosen to make the opponent ndfferent. These ndfference condtons are volated once the sgnal dstrbuton s perturbed; as a result, the exstng constructons of belef-free equlbra are not robust to a perturbaton of the montorng structure. A challenge n constructng belef-free equlbra n our setup s that we need to fnd randomzaton probabltes that satsfy all the ndfference condtons even when players do not know the sgnal dstrbuton and ther belefs about the sgnal dstrbuton can be perturbed. If the same randomzaton probablty satsfes the ndfference condtons for all states, then t s a good canddate for an equlbrum. We fnd that such a strong requrement can be satsfed and the resultng equlbra can support a large set of non-trval payoffs f the sgnal dstrbuton satsfes the statewse full-rank condton. Roughly speakng, the statewse full-rank condton says that each player can statstcally dstngush the true state usng prvate sgnals regardless of the play by the opponent. Ths condton requres that there be more possble sgnals than n the case of the canoncal sgnal space studed n the past work, whch ensures that there be enough room to choose approprate randomzaton probabltes. To llustrate the dea of BFXE, we begn wth smple examples. In Secton 3.1, we consder prvate provson of publc goods where the margnal proft from contrbuton s unknown and players learn t through prvate sgnals. In ths stuaton, players cannot observe what the opponent has learned about the margnal proft; thus, t s unclear how players coordnate ther play n equlbrum, and as a result, varous folk theorems derved n past work do not apply. We explctly construct a BFXE and show that t attans the Pareto-effcent outcome n ths example. Then n Secton 3.2, we consder another example n whch frms decde 4

5 whether to advertse ther products and there s uncertanty about the effect of advertsement. Agan, we construct a BFXE and show that t acheves non-trval payoffs. Wth these complete descrptons of equlbrum strateges, t s easy to see how players learn the state from prvate sgnals and use that nformaton n BFXE. In partcular, t s worth notng that the equlbrum strateges dscussed n Secton 3.1 exhbt a smple form of punsh-and-forgve behavor, whle those dscussed n Secton 3.2 take a dfferent smple form of learnng-and-adjustment behavor. In BFXE, players belefs about the true state are rrelevant to ther best reples, and hence one may wonder what s the value of state learnng n ths class of equlbra. The key s that even though players play the same strategy profle regardless of the true state n an ex-post equlbrum, the dstrbuton of future actons may depend on the true state; ths s because players future play depends on sgnals today, and the dstrbuton of these sgnals s nfluenced by the state. In partcular, there may be an ex-post equlbrum where for each state of the world, the dstrbuton of actons condtonal on that state assgns a hgh probablty to the effcent acton for that state. In ths sense, state learnng s valuable even f we look at ex-post equlbra. In Secton 5, we extend ths dea to a general setup and obtan our man result, the state-learnng theorem. It characterzes the set of BFXE payoffs wth patent players under the statewse full-rank condton and shows that there are BFXE n whch players eventually obtan payoffs as f the true state were common knowledge and players played a belef-free equlbrum for that state. Ths mples that BFXE can do as well as belef-free equlbra can do n the known-state game and that the man results of EHO extend to the case n whch players do not know the montorng structure. Whle the statewse full-rank condton guarantees that players prvately learn the true state n the long run, t does not necessarly mply that the state becomes (approxmate) common knowledge, and thus the result here s not an mmedate consequence of the assumpton. Applyng ths state-learnng theorem, we show that frms can mantan colluson even f they do not have precse nformaton about the market. As argued, the set of BFXE s only a subset of Nash equlbra and s empty for some cases (although we show that BFXE exst when players are patent and some addtonal condtons are satsfed; see Remark 5). Nevertheless, the study 5

6 of BFXE can be motvated by the followng consderatons. Frst, BFXE can often approxmate the effcent outcome, as we show n several examples. Second, BFXE are robust to any specfcaton of the ntal belefs, just as for ex-post equlbra. That s, BFXE reman equlbra when players are endowed wth arbtrary belefs that need not arse from a common pror. Thrd, BFXE are robust to any specfcaton of how players update ther belefs. For example BFXE are stll equlbra even f players employ non-bayesan updatng of belefs, or even f each player observes unmodeled sgnals that are correlated wth the opponent s past prvate hstory and/or the true state. Fnally, BFXE have a recursve property, n the sense that any contnuaton strategy profle of a BFXE s also a BFXE. Ths property greatly smplfes our analyss and may make our approach a promsng drecton for future research. 1.1 Lterature Revew The noton of BFXE s a generalzaton of belef-free equlbra, whch plays a central role n the study of repeated games wth prvate montorng. The dea of belef-free equlbra s proposed by Pccone (2002) and extended by Ely and Välmäk (2002), EHO, and Yamamoto (2007). Its lmt equlbrum payoff set s fully characterzed by EHO and Yamamoto (2009). Olszewsk (2007) provdes an ntroductory survey. Kandor and Obara (2006) show that belef-free equlbra can acheve better payoffs than perfect publc equlbra for games wth publc montorng. Kandor (2011) proposes a generalzaton of belef-free equlbra, called weakly belef-free equlbra. Takahash (2010) constructs a verson of belef-free equlbra n repeated random matchng games. Bhaskar, Malath, and Morrs (2008) nvestgate the Harsany-purfablty of belef-free equlbra. Sugaya and Takahash (2010) show that belef-free publc equlbra of games wth publc montorng are robust to prvate-montorng perturbatons. BFXE are also related to ex-post equlbra. Some recent papers use the expost equlbrum approach n dfferent settngs of repeated games, such as perfect montorng and fxed states (Hörner and Lovo (2009) and Hörner, Lovo, and Tomala (2011)), publc montorng and fxed states (Fudenberg and Yamamoto (2010) and Fudenberg and Yamamoto (2011a)), and changng states wth an..d. dstrbuton (Mller (2012)). Note also that there are many papers that dscuss 6

7 ex-post equlbra n undscounted repeated games; see Koren (1992) and Shalev (1994), for example. Among these, the most closely related work s Fudenberg and Yamamoto (2010); see Secton 5.4 for a detaled comparson. We also contrbute to the lterature on repeated games wth ncomplete nformaton. Many papers study the case n whch there s uncertanty about the payoff functons and actons are observable; for example, see Forges (1984), Sorn (1984), Hart (1985), Sorn (1985), Aumann and Maschler (1995), Crpps and Thomas (2003), Gossner and Velle (2003), Wseman (2005), and Wseman (2012). Crpps, Ely, Malath, and Samuelson (2008) consder the stuaton n whch players try to learn the unknown state of the world by observng a sequence of prvate sgnals over tme and provde a condton under whch players commonly learn the state. In ther model, players do observe prvate sgnals but do not choose actons. On the other hand, we consder strategc players who mght want to devate to slow down the speed of learnng. Therefore, ther result does not drectly apply to our settng. 2 Repeated Games wth Prvate Learnng Gven a fnte set X, let X be the set of probablty dstrbutons over X, and let P(X) be the set of non-empty subsets of X,.e., P(X) = 2 X \ {/0}. Gven a subset W of n, let cow denote the convex hull of W. We consder two-player nfntely repeated games, where the set of players s denoted by I = {1,2}. At the begnnng of the game, Nature chooses the state of the world ω from a fnte set Ω. Assume that players cannot observe the true state ω, and let µ Ω denote ther common pror over ω. 4 Throughout the paper, we assume that the game begns wth symmetrc nformaton: Each player s belefs about ω correspond to the pror. But t s straghtforward to extend our analyss to the case wth asymmetrc nformaton as n Fudenberg and Yamamoto (2011a). 5 4 Because our arguments deal only wth ex-post ncentves, they extend to games wthout a common pror. However, as Dekel, Fudenberg, and Levne (2004) argue, the combnaton of equlbrum analyss and a non-common pror s hard to justfy. 5 Specfcally, all the results n ths paper extend to the case n whch each player has ntal prvate nformaton θ about the true state ω, where the set Θ of player s possble prvate nformaton s a partton of Ω. Gven the true state ω Ω, player observes θ ω 7 Θ, where θ ω denotes

8 Each perod, players move smultaneously, and player I chooses an acton a from a fnte set A and observes a prvate sgnal σ from a fnte set Σ. 6 Let A I A and Σ = I Σ. The dstrbuton of a sgnal profle σ Σ depends on the state of the world ω and on an acton profle a A, and s denoted by π ω ( a) Σ. Let π ω ( a) denote the margnal dstrbuton of σ Σ at state ω condtonal on a A, that s, π ω (σ a) = σ Σ π ω (σ a). Player s realzed payoff s u ω(a,σ ), so that her expected payoff at state ω gven an acton profle a s g ω(a) = σ Σ π ω (σ a)u ω(a,σ ). We wrte π ω (α) and g ω (α) for the sgnal dstrbuton and expected payoff when players play a mxed acton profle α I A. Smlarly, we wrte π ω (a,α ) and g ω(a,α ) for the sgnal dstrbuton and expected payoff when player plays a mxed acton α A. Let g ω (a) denote the vector of expected payoffs at state ω gven an acton profle a. 7 As emphaszed n the ntroducton, uncertanty about the payoff functons and/or the montorng structure s common n applcatons. Examples that ft our model nclude secret prce-cuttng wth unknown demand functon and moral hazard wth subjectve evaluaton and unknown evaluaton dstrbuton. Also, a repeated game wth observed actons and ndvdual learnng s a specal case of the above model. To see ths, let Σ = A Z for some fnte set Z and assume that π ω (σ a) = 0 for each ω, a, and σ = (σ 1,σ 2 ) = ((a,z 1 ),(a,z 2 )) such that a a or a a. Under ths assumpton, actons are perfectly observable by players (as σ must be consstent wth the acton profle a), and players learn the true state ω from prvate sgnals z. More concrete examples wll be gven n the next secton. In the nfntely repeated game, players have a common dscount factor δ (0,1). Let (a τ,στ ) be player s pure acton and sgnal n perod τ, and we denote player s prvate hstory from perod one to perod perod t 1 by h t = θ Θ such that ω θ. In ths setup, prvate nformaton θ ω allows player to narrow down the set of possble states; for example, player knows the state f Θ = {(ω 1 ),,(ω o )}. For games wth asymmetrc nformaton, we can allow dfferent types of the same player to have dfferent best reples as n PTXE of Fudenberg and Yamamoto (2011a); to analyze such equlbra, regme R should specfy recommended actons for each player and each type θ,.e., R = (R θ ) (,θ ). 6 Here we consder a fnte Σ just for smplcty; our results extend to the case wth a contnuum of prvate sgnals, as n Ish (2009). 7 If there are ω Ω and ω ω such that u ω (a,σ ) u ω (a,σ ) for some a A and σ Σ, then t mght be natural to assume that player does not observe the realzed value of u as the game s played; otherwse players mght learn the true state from observng ther realzed payoffs. Snce we consder ex-post equlbra, we do not need to mpose such a restrcton. 8

9 (a τ,στ )t τ=1. Let h0 = /0, and for each t 0, let H t be the set of all prvate hstores h t. Also, we denote a par of t-perod hstores by ht = (h t 1,ht 2 ), and let H t be the set of all hstory profles h t. A strategy for player s defned to be a mappng s : t=0 H t A. Let S be the set of all strateges for player, and let S = I S. We defne the feasble payoff set for a gven state ω to be V (ω) co{g ω (a) a A}, that s, V (ω) s the set of the convex hull of possble stage-game payoff vectors gven ω. Then we defne the feasble payoff set for the overall game to be V ω Ω V (ω). Thus, a vector v V specfes payoffs for each player and for each state,.e., v = ((v1 ω,vω 2 )) ω Ω. Note that a gven v V may be generated usng dfferent acton dstrbutons n each state ω. If players observe ω at the start of the game and are very patent, then any payoff n V can be obtaned by a state-contngent strategy of the nfntely repeated game. Lookng ahead, there wll be equlbra that approxmate payoffs n V f the state s dentfed by the sgnals so that players learn t over tme. 3 Motvatng Examples In ths secton, we consder a seres of examples to llustrate the dea of our equlbrum strateges when players learn the true state from prvate sgnals. We assume that actons are observable n these examples, but we would lke to stress that ths assumpton s made for expostonal ease. Indeed, as wll be explaned, a smlar equlbrum constructon s vald even f players observe nosy nformaton about actons. 3.1 Prvate Provson of Publc Goods There are two players and two possble states, so Ω = {ω 1,ω 2 }. In each perod t, each player decdes whether to contrbute to a publc good or not. Let A = {C,D} be the set of player s possble actons, where C means contrbutng to 9

10 the publc good and D means no contrbuton. After makng a decson, each player receves a stochastc output z from a fnte set Z. An output z s prvate nformaton of player and ts dstrbuton depends on the true state ω and on the total nvestment a A. Note that many economc examples ft ths assumpton, as frms profts are often prvate nformaton, and frms are often uncertan about the dstrbuton of profts. We also assume that a choce of contrbuton levels s perfectly observable to all the players. Ths example can be seen as a specal case of the model ntroduced n Secton 2, by lettng Σ = A Z and by assumng π ω (σ a) = 0 for each ω, a, and σ = (σ 1,σ 2 ) = ((a,z 1 ),(a,z 2 )) such that a a or a a. Wth an abuse of notaton, let π ω (z a) denote the jont dstrbuton of z = (z 1,z 2 ) gven (a,ω); that s, π ω (z a) = π ω ((a,z 1 ),(a,z 2 ) a). We do not mpose any assumpton on the jont dstrbuton of (z 1,z 2 ) so that outputs z 1 and z 2 can be ndependent or correlated. When z 1 and z 2 are perfectly correlated, our setup reduces to the case n whch outputs are publc nformaton. Player s actual payoff does not depend on the state ω and s gven by u (a,σ ) = ũ (z ) c (a ), where ũ (z ) s player s proft from an output z and c (a ) s the cost of contrbutons. We assume c (C) > c (D) = 0, that s, contrbuton s costly. As n Secton 2, the expected payoff of frm at state ω s denoted by g ω(a) = σ Σ π ω (σ a)u (a,σ ). Note that a player s expected payoff depends on the true state ω, as t nfluences the dstrbuton of outputs z. We assume that the expected payoffs are as n the followng tables: C D C D C 3, 3 1, 4 C 3, 3 1, 4 D 4, 1 0, 0 D 4, 1 0, 0 The left table denotes the expected payoffs for state ω 1, and the rght table for state ω 2. Note that the stage game s a prsoner s dlemma at state ω 1, and s a chcken game at state ω 2. Ths captures the stuaton n whch contrbutons are socally effcent but players have a free-rdng ncentve; ndeed, n each state, (C,C) s effcent, but a player s wllng to choose D when the opponent chooses C. Another key feature of ths payoff functon s that players do not know the margnal beneft from contrbutng to a publc good and do not know whether they should contrbute, gven that the opponent does not contrbute. Specfcally, the margnal beneft s low n ω 1 so that a player prefers D to C when the opponent chooses D, whle the margnal proft s hgh n ω 2 so that a player prefers C. 10

11 Snce actons are observable, one may expect that the effcent payoff vector ((3, 3),(3, 3)) can be approxmated by standard trgger strateges. But ths approach does not work because there s no statc ex-post equlbrum n ths game and how to punsh a devator n a trgger strategy s not obvous. Note also that the folk theorems of Fudenberg and Yamamoto (2010) and Wseman (2012) do not apply here, as they assume that players obtan publc (or almost publc) nformaton about the true state n each perod. In ths example, players learn the true state ω only through prvate nformaton z. In what follows, we wll construct a smple equlbrum strategy that acheves the payoff vector ((3,3),(3,3)) when players are patent. We assume that for each, there are outputs z ω 1 and z ω 2 such that ω 2 π ω 1 (z ω 1 a) π ω 2 (z ω 1 a) 2 and π (z ω 2 a) π ω 1 (z ω 2 a) 2 (1) for each a, where π ω ( a) s the margnal dstrbuton of z gven (ω,a). Intutvely, (1) means that the margnal dstrbutons of z are suffcently dfferent at dfferent states so that there s an output level z ω that has a suffcently hgh lkelhood rato to test for the true state beng ω. Ths assumpton s not necessary for the exstence of asymptotcally effcent equlbra (see Secton 5.3 for detals), but t consderably smplfes our equlbrum constructon, as shown below. In our equlbrum, each player uses a strategy that s mplemented by a twostate automaton. Specfcally, player 1 uses the followng strategy: Automaton States: Gven any perod t and gven any hstory h t 1, player 1 s n one of the two automaton states, ether x(1) or x(2). In state x(1), player 1 chooses C to reward player 2. In state x(2), player 1 chooses D to punsh player 2. Transton After State x(1): Suppose that player 1 s currently n the reward state x(1) so that she chooses C today. In the next perod, player 1 wll swtch to the punshment state x(2) wth some probablty, dependng on today s outcome. Specfcally, gven player 2 s acton a 2 A 2 and player 1 s output z 1 Z 1, player 1 wll go to the punshment state x(2) wth probablty β(a 2,z 1 ) and stay at the reward state x(1) wth probablty 1 β(a 2,z 1 ). We set β(c,z 1 ) = 0 for all z 1 ; that s, player 1 wll reward player 2 for sure f player 2 chooses C today. β(d,z 1 ) wll be specfed later, but we wll have β(d,z 1 ) > 0 for all z 1, that s, player 1 11

12 wll punsh player 2 wth postve probablty f player 2 chooses D today. Transton After State x(2): Suppose that player 1 s n the punshment state x(2) so that she chooses D today. In the next perod, player 1 wll swtch to the reward state x(1) wth some probablty, dependng on today s outcome. Specfcally, gven (a 2,z 1 ), player 1 wll go to x(1) wth probablty γ(a 2,z 1 ) and stay at x(2) wth probablty 1 γ(a 2,z 1 ). γ(a 2,z 1 ) wll be specfed later, but we wll have γ(a 2,z 1 ) > 0 for all a 2 and z 1, that s, player 1 wll swtch to the reward state wth postve probablty no matter what player 2 does. 1 β(a 2,z 1 ) 1 γ(a 2,z 1 ) State x(1) β(a 2,z 1 ) State x(2) Play C Play D γ(a 2,z 1 ) Fgure 1: Automaton The equlbrum strategy here s smple and ntutve. If player 1 s n the reward state x(1) today, she wll stay at the same state and contnue to choose C unless the opponent chooses D. If she s n the punshment state x(2), she chooses D today to punsh the opponent and then returns to the reward state x(1) wth a certan probablty to forgve the opponent. In what follows, we wll show that ths punsh-and-forgve behavor actually consttutes an equlbrum f we choose the transton probabltes carefully. The key dea s to choose player 1 s transton probabltes β and γ n such a way that player 2 s ndfferent between C and D regardless of the true state ω and of player 1 s current automaton state. Ths property ensures that player 2 s ndfferent between C and D after every hstory so that any strategy s a best response for player 2. Also, we construct player 2 s strategy n the same way so that player 1 s always ndfferent between C and D. Then the par of such strateges consttutes an equlbrum, as they are best reples to each other. An advantage of ths equlbrum constructon s that each player s best reply s ndependent of her belef 12

13 about the true state ω and of her belef about the opponent s hstory so that we do not need to compute these belefs to check the ncentve compatblty. We call such a strategy profle belef-free ex-post equlbrum (BFXE). More specfcally, we wll choose the transton probabltes n such a way that the followng propertes are satsfed: If player 1 s currently n the reward state x(1), then player 2 s contnuaton payoff from today s 3 regardless of the true state ω and of player 2 s contnuaton play. If player 1 s currently n the punshment state x(2), then player 2 s contnuaton payoff s 2 at ω 1 and 7 3 at ω 2, no matter what player 2 plays. For each ω and k = 1,2, let v2 ω (k) denote the target payoff of player 2 specfed above when the true state ω and player 1 s automaton state x(k) are gven. That s, let v ω 1 2 (1) = vω 2 2 (1) = 3, vω 1 2 (2) = 2, and vω 2 2 (2) = 7 3. Let v 2(k) denote the target payoff vector of player 2 gven x(k),.e., v 2 (k) = (v ω 1 2 (k),vω 2 2 (k)) for each k. Fgure 2 descrbes these target payoffs and stage-game payoffs. The horzontal axs denotes player 2 s payoff at ω 1, and the vertcal axs denotes player 2 s payoff at ω 2. The pont (4,4) s the payoff vector of the stage game when (C,D) s played. Lkewse, the ponts (3,3), ( 1,1), and (0,0) are generated by (C,C), (D,C), and (D,D), respectvely. The bold lne s the convex hull of the set of target payoff vectors, v 2 (1) and v 2 (2). When the dscount factor δ s close to one, there ndeed exst transton probabltes β and γ such that these target payoffs are exactly acheved. To see ths, consder the case n whch player 1 s currently n the reward state x(1) so that the target payoff s v 2 (1) = (3,3). If player 2 chooses C today, then player 2 s stagegame payoff s 3 regardless of ω, whch s exactly equal to the target payoff. Thus the target payoff can be acheved by askng player 1 to stay at the same state x(1) n the next perod,.e., we set β(c,z 1 ) = 0 for all z 1. On the other hand, f player 2 chooses D today, then player 2 s stage-game payoff s 4 regardless of ω, whch s hgher than the target payoff. To offset ths nstantaneous gan, player 1 wll swtch to the punshment state x(2) wth postve probablty β(d,z 1 ) > 0 n the 13

14 v ω 2 2 (4,4) (3,3) ( 1, 1) (2, 7 3 ) (0,0) v ω 1 2 Fgure 2: Payoffs next perod. Here the transton probablty β(d,z 1 ) > 0 s carefully chosen so that the nstantaneous gan by playng D today and the expected loss by the future punshment exactly cancel out gven any state ω;.e., we choose β(d,z 1 ) such that g ω 2 (C,D) vω 2 (1) = δ 1 δ π1 ω (z 1 C,D)β(D,z 1 )(v2 ω (1) vω 2 (2)) (2) z 1 Z 1 for each ω. (Note that the left-hand sde s the nstantaneous gan by playng D, whle the rght-hand sde s the expected loss n contnuaton payoffs.) The formal proof of the exstence of such β s gven n Appendx A, but the ntuton s as follows. Note that we have v ω 2 2 (1) vω 2 2 (2) < vω 1 2 (1) vω 1 2 (2), whch means that the magntude of the punshment at ω 2 s smaller than at ω 1. Thus, n order to offset the nstantaneous gan and satsfy (2) for each ω, the probablty of punshment, z1 Z 1 π1 ω(z 1 C,D)β(D,z 1 ), should be larger at ω 2 than at ω 1. Ths can be done by lettng β(d,z ω 2 1 ) > β(d,z 1) for all z 1 z ω 2 1, that s, n our equlbrum, player 1 swtches to the punshment state wth larger probablty when the output level s z ω 2 1, whch s an ndcaton of ω 2. Next, consder the case n whch player 1 s n the punshment state x(2) so that the target payoff s v 2 (2) = (2, 3 7 ). Snce player 1 chooses D today, player 2 s stage-game payoff s lower than the target payoff regardless of ω and of player 2 s acton today. Hence, n order to compensate for ths defct, player 1 needs to swtch to the reward state x(1) wth postve probablty γ(a 2,z 1 ) > 0 n the next 14

15 perod. The proof of the exstence of such γ s very smlar to that of β and s found n Appendx A. Wth such a choce of β and γ, player 2 s always ndfferent between C and D, and hence any strategy of player 2 s a best reply. Also, as explaned, we construct player 2 s two-state automaton n the same way so that player 1 s always ndfferent. Then the par of these strateges consttutes an equlbrum. In partcular, when both players begn ther play from the reward state x(1), the resultng equlbrum payoff s ((3,3),(3,3)), as desred. In ths effcent equlbrum, players choose (C,C) forever unless somebody devates to D. If player devates and chooses D, player punshes ths devaton by swtchng to x(2) wth postve probablty and startng to play D. However, always play D s too harsh compared wth the target payoff (2, 7 3 ), and hence player comes back to x(1) wth some probablty after every perod. In the long run, both players come back to x(1) and play (C,C) because ths s the unque absorbng automaton state. The above two-state automaton s a generalzaton of that of Ely and Välmäk (2002) for repeated prsoner s dlemma games wth almost-perfect montorng. The reason why ther equlbrum constructon drectly extends s that n ths example, the payoffs at dfferent states are smlar n the sense that the acton C can be used to reward the opponent and D to punsh, regardless of the true state ω. 8 When ths structure s lost, a player s not sure about what acton should be taken to reward or punsh the opponent; therefore, state learnng becomes more mportant. In the next example, we show what the equlbrum strateges look lke n such an envronment. 9 8 It s also mportant to note that the sgnal space n ths example s Σ = A Z, whch s larger than the canoncal sgnal space (Σ = A) consdered by Ely and Välmäk (2002). As dscussed n the ntroducton, ths larger sgnal space ensures that there be enough room to choose approprate randomzaton probabltes. 9 A referee ponted out that the followng two-state automaton can consttute an equlbrum n ths example. Players choose C n the automaton state x(1), whle they choose D n state x(2). Regardng the transton functon, f the state was x(1) last perod then the state s x(1) ths perod f and only f both players chose C last perod. If the state was x(2) last perod then the state s x(1) ths perod f and only f both players chose D last perod. It s easy to check that ths strategy profle s ndeed an ex-post equlbrum for all δ ( 1 3,1). However, ths profle s not belef-free and hence not robust to the ntroducton of prvate montorng. On the other hand, our equlbrum constructon works well even f the montorng structure s perturbed. See Remark 1. 15

16 3.2 Advertsng Consder two frms that sell (almost) homogeneous products. Each frm decdes whether to advertse ts product (Ad) or not (N), so the set of actons for frm s A = {Ad,N}. Suppose that each frm observes an acton profle a A and a sales level z Z n each perod. As n the prevous example, ths setup can be regarded as a specal class of the model ntroduced n Secton 2 by lettng Σ = A Z. Frm s stage-game payoff s u (a,z ) = ũ(z ) c(a ), where ũ(z ) s the revenue when the sales level s z and c(a ) s the cost of advertsement. The dstrbuton of sales levels (z 1,z 2 ) s nfluenced by the true state ω and the level of advertsement a and s denoted by π ω ( a). We assume that advertsement nfluences the frms sales levels n two ways: Frst, t ncreases the aggregate demand for the products so that advertsement by one frm has postve effects on both frms expected sales levels. Second, a frm that advertses takes some customers away from the rval frm, whch means that advertsement has a negatve effect on the rval frm s expected sales level. There s uncertanty about the magntude of these effects. We assume that n state ω 1, the frst effect s large (whle the second s small) so that advertsement greatly ncreases the aggregate demand. Hence, n state ω 1, the sum of the frms profts s maxmzed when both frms advertse. However, because advertsement s costly, frms have ncentves to free rde advertsement by the rval frm. Accordngly, n state ω 1, the game s a prsoner s dlemma, where (Ad,Ad) s Pareto-effcent whle N strctly domnates Ad. On the other hand, n state ω 2, we assume that the second effect of the advertsement s large, but the frst effect s small; so advertsement has a lmted effect on the aggregate demand. Ths mples that n state ω 2, the sum of the frms profts s maxmzed when both frms do not advertse. But the frms are tempted to advertse, as t allows them to take customers away from the opponent. As a result, n state ω 2, the game s a prsoner s dlemma n whch (N, N) s Pareto-effcent whle Ad strctly domnates N. Note that the roles of the two actons are reversed here. The followng tables summarze ths payoff structure: 16

17 Ad N Ad 1, 1 1, 2 N 2, 1 0, 0 Ad N Ad 0, 0 2, 1 N 1, 2 1, 1 We stress the numbers n the tables are expected payoffs g ω (a) = z π ω (a)u (a,z ), rather than actual payoffs, and hence the frms do not observe these numbers n the repeated game. As explaned, what frm can observe n each stage game s the acton profle a and the sales level z (and the resultng payoff u (a,z )). In ths example, the effcent payoff vector ((1,1),(1,1)) s not feasble n the one-shot game, as the frms need to choose dfferent acton profles at dfferent states to generate ths payoff (they need to play (Ad,Ad) at ω 1 and (N,N) at ω 2 ). 10 We assume that there s a partton {Z ω 1,Z ω 2 } of Z such that gven any acton profle a, the probablty of z Z ω s 3 2 at state ω and s 3 1 at state ω ω. That s, we assume that z Z ω π ω (z a) = 2 3 for each, ω, and a. We nterpret Zω 1 as the set of sales levels (or sgnals) z that ndcate that the true state s lkely to be ω 1, whle Z ω 2 s nterpreted as the set of sgnals that ndcate that the true state s lkely to be ω 2. As n the prevous example, ths lkelhood rato assumpton s not necessary for the exstence of asymptotcally effcent equlbra, but t smplfes our equlbrum constructon. We mpose no assumpton on the jont dstrbuton of z 1 and z 2, so these sgnals can be ndependent or correlated. In ths example, the payoff functons are totally dfferent at dfferent states so that state learnng s necessary to provde proper ntertemporal ncentves. However, snce the frms learn the true state from prvate sgnals, they may not know what the opponent has learned n the past play, and t s unclear how the frms create such ncentves. Our goal s to gve a smple and explct equlbrum constructon where the frms learn the state and adjust future actons. As n the prevous 10 Ths payoff structure can be nterpreted as a problem of an organzaton as well: Each player s the manager of one dvson n a mult-dvsonal frm, and each dvson manager s contemplatng one of two technologes/products. They can ether choose the same product (that corresponds to outcomes (Ad, Ad) or (N, N)) or dfferent products. The state ndcates whch product s proftable (ether product 1 or 2). Decson-makng and cost allocatons are as follows. Revenue from the fnal product s equally shared. If both players agree on the product, then that product s mplemented and costs are shared. If they dsagree on the product, then the center nvestgates the state and chooses the product that maxmzes profts (not worryng about ts dstrbutve effect). Crucally, the dvson that recommended that project takes the lead and bears all the costs; so the tenson s between coordnatng on the best technology and shrkng on the cost sharng. Wth an approprate choce of parameters, ths example can ft n the payoff matrces presented n ths subsecton. I thank Rcardo Alonso for suggestng ths nterpretaton. 17

18 example, our equlbrum s a BFXE, that s, each frm s ndfferent between the two actons gven any hstory and gven any state ω. In our equlbrum, each frm tres to learn the true state ω from prvate sgnals at the begnnng and then adjust the contnuaton play to choose an approprate acton; each frm chooses Ad when t beleves that the true state s ω 1 and chooses N when t beleves that the true state s ω 2. Formally, frm 1 s strategy s descrbed by the followng four-state automaton: Automaton States: Gven any perod t and after any hstory, frm 1 s n one of the four automaton states, x(1), x(2), x(3), or x(4). Frm 1 chooses Ad n states x(1) and x(2), whle t chooses N n states x(3) and x(4). As before, we denote by v 2 (k) = (v ω 1 2 (k),vω 2 2 (k)) frm 2 s ex-post payoffs of the repeated game when frm 1 s play begns wth the automaton state x(k). Set v 2 (1) = (v ω 1 2 (1),vω 2 2 (1)) = (1,0), v 2 (2) = (v ω 1 2 (2),vω 2 2 (2)) = (0.8,0.79), v 2 (3) = (v ω 1 2 (3),vω 2 2 (3)) = (0.79,0.8), v 2 (4) = (v ω 1 2 (4),vω 2 2 (4)) = (0,1). The nterpretaton of the automaton states s as follows. In state x(1), frm 1 beleves that the true state s ω 1 and wants to reward the opponent by playng Ad. As a result, the correspondng target payoff for frm 2 s hgh at ω 1 (v ω 1 2 (1) = 1), but low at ω 2 (v ω 1 2 (1) = 0). Lkewse, n state x(4) frm 1 beleves that the true state s ω 2 and wants to reward the opponent by playng N. In states x(2) and x(3), frm 1 s stll unsure about ω; frm 1 moves back and forth between these two states for a whle, and after learnng the state ω, t moves to x(1) or x(4). The detal of the transton rule s specfed below, but roughly speakng, when frm 1 gets convnced that the true state s ω 1, t moves to x(1) and chooses the approprate acton Ad. Lkewse, when frm 1 becomes sure that the true state s ω 2, t moves to x(4) and chooses N. Ths learnng and adjustment process by frm 1 yelds hgh expected payoffs to the opponent regardless of the true state ω; ndeed, as shown n Fgure 3, frm 2 obtans hgh payoffs at both ω 1 and ω 2, when frm 1 s ntal automaton state s x(2) or x(3). 18

19 ( 1, 2) v ω 2 2 (0,1) (0.79, 0.8) (0.8, 0.79) (1,0) v ω 1 2 (2, 1) Fgure 3: Payoffs Transtons After x(1): If frm 2 advertses today, then frm 1 stays at x(1) for sure. If frm 2 does not, then frm 1 swtches to x(4) wth probablty 1 δ δ and stays at x(1) wth probablty 1 1 δ δ. The dea of ths transton rule s as follows. When frm 2 advertses, the stagegame payoff for frm 2 s (1,0), whch s exactly the target payoff v 2 (1), so that frm 1 stays at x(1) for sure. On the other hand, when frm 2 chooses N, the stagegame payoff for frm 2 s (2, 1), whch s dfferent from the target payoff. Thus, frm 1 moves to x(4) wth postve probablty to offset ths dfference. Transtons After x(2): Suppose that frm 2 advertses today. If today s sgnal s z 1 Z ω 1 (1 δ)117 1, frm 1 goes to x(1) wth probablty δ and stays at x(2) wth the remanng probablty. If today s sgnal s z 1 Z ω 2 1, then go to x(3) wth probablty (1 δ)4740 δ and stay at x(2) wth the remanng probablty. That s, frm 1 moves to x(1) only when frm 1 observes z 1 Z ω 1 1 and gets more convnced that the true state s ω 1. Suppose next that frm 2 does not advertse today. If today s sgnal s z 1 Z ω 1 1, frm 1 goes to x(3) wth probablty (1 δ)61 δ and stays at x(2) wth the remanng probablty. If today s sgnal s z 1 Z ω 2 (1 δ)238 1, then go to x(3) wth probablty δ and stay at x(2) wth the remanng probablty. Note that frm 1 wll not move to x(1) n ths case regardless of her sgnal z 1. The reason s that when frm 2 does not advertse, ts stage-game payoff at ω 1 s 2, whch s too hgh compared 19

20 wth the target payoff. In order to offset ths dfference, frm 1 needs to gve lower contnuaton payoffs to the opponent by movng to x(3) rather than x(1). Transtons After x(3): The transton rule s symmetrc to the one after x(2). Suppose that frm 2 does not advertse today. If today s sgnal s z 1 Z ω 2 frm 1 goes to x(4) wth probablty (1 δ)117 δ probablty. If today s sgnal s z 1 Z ω 1 1 and stay at x(3) wth the remanng probablty. 1, and stays at x(3) wth the remanng (1 δ)4740, then go to x(2) wth probablty δ Suppose next that frm 2 advertses today. If today s sgnal s z 1 Z ω 2 1, frm 1 goes to x(2) wth probablty (1 δ)61 δ and stays at x(3) wth the remanng probablty. If today s sgnal s z 1 Z ω 1 1 stay at x(3) wth the remanng probablty., then go to x(2) wth probablty (1 δ)238 δ Transtons After x(4): The transton rule s symmetrc to the one after x(1). If frm 2 does not advertse today, then stay at x(4). If frm 2 advertses, then go to x(1) wth probablty 1 δ 2δ 1 δ, and stay at x(4) wth probablty δ. and State x(2) Play U State x(3) Play D Only f z 1 = z ω 1 1 Only f z 1 = z ω 2 1 State x(1) Play U Only f R Only f L State x(4) Play D Fgure 4: Automaton Smple algebra (lke (2) n the prevous example) shows that gven any ω and x(k), frm 2 s ndfferent between the two actons and ts overall payoff s exactly v ω 2 (k). Ths means that any strategy of the repeated game s optmal for frm 2 20

21 when frm 1 uses the above automaton strategy. We can construct frm 2 s automaton strategy n the same way, and t s easy to see that the par of these automaton strateges s an equlbrum of the repeated game. When both frms ntal automaton states are x(2), the equlbrum payoff s ((0.8, 0.8),(0.79, 0.79)), whch cannot be acheved n the one-shot game. Ths example shows that BFXE work well even when the payoff functons are totally dfferent at dfferent states. A remanng queston s whether there are more effcent equlbra, and n partcular, whether we can approxmate the payoff vector ((1,1),(1,1)). The reason why the above equlbrum payoff s bounded away from ((1,1),(1,1)) s that whle the frms can obtan arbtrarly precse nformaton about the true state ω n the long run, they do not use that nformaton effectvely. To see ths, note that n the above equlbrum, each frm s contnuaton strategy from the next perod depends only on the current automaton state and the outcome n the current perod; that s, each frm s prvate sgnals n the past play can nfluence the contnuaton play only through the current automaton state. But there are only four possble automaton states (x(1), x(2), x(3), or x(4)), whch means that they are less nformatve about ω than the orgnal prvate sgnals. (In other words, the automaton states can represent only coarse nformaton about ω.) Accordngly, the frms often end up wth neffcent acton profles. For example, even f the true state s ω 1, the probablty that they reach the state x(4) n the long run and play the neffcent acton profle (N,N) forever s bounded away from zero. Ths problem can be solved by consderng an automaton wth more states; f we ncrease the number of automaton states, then nformaton classfcaton becomes fner, whch allows us to construct more effcent equlbra. For example, 21

22 there s an automaton wth sx states that generates the followng payoffs: 11 v 2 (1) = (v ω 1 2 (1),vω 2 2 (1)) = (1,0), v 2 (2) = (v ω 1 2 (2),vω 2 2 (2)) = (0.93,0.9), v 2 (3) = (v ω 1 2 (3),vω 2 2 (3)) = (0.927,0.91), v 2 (4) = (v ω 1 2 (4),vω 2 2 (4)) = (0.91,0.927), v 2 (5) = (v ω 1 2 (5),vω 2 2 (5)) = (0.9,0.93), v 2 (6) = (v ω 1 2 (6),vω 2 2 (6)) = (0,1). We can show that as the number of automaton states ncreases, more effcent payoffs are achevable, and the effcent payoff ((1, 1),(1, 1)) s eventually approxmated. 12 Also there are asymptotcally effcent equlbra even when we consder a general sgnal dstrbuton; see Secton 5.3 for more detals. Remark 1. The equlbrum strategy presented above s related to organzatonal routnes, although the meanng of routnes s ambguous n the lterature on organzaton economcs. Blume, Franco, and Hedhues (2011) consder a problem where each player has prvate nformaton (sgnal) about the state of the world and study ordnal equlbra n whch players use only coarse nformaton. In ordnal equlbra, each player s sgnal space s parttoned nto subsets over whch 11 These payoffs are generated by the followng automaton. Actons: Frm 1 chooses Ad n states x(1), x(2), and x(3) and N n states x(4), x(5), and x(6). Transtons After x(1): If frm 2 advertses today, then frm 1 stays at x(1) for sure. If not, then frm 1 swtches to x(6) wth probablty 1 δ δ, and stays at x(1) wth probablty 1 1 δ δ. Transtons After x(2): Suppose that frm 2 advertses today. If z 1 Z ω 1 1 s observed, frm goes to x(1) wth probablty (1 δ)39 δ and stays at x(2) wth the remanng probablty. If z 1 Z ω 2 1 s observed, then go to x(3) wth probablty (1 δ)1890 δ and stay at x(2) wth the remanng probablty. Suppose next that frm 2 does not advertse today. If z 1 Z ω 1 1 s observed, frm 1 goes to x(3) wth probablty (1 δ)1570 δ3 and stays at x(2) wth the remanng probablty. If z 1 Z ω 2 1 s observed, then go to x(3) wth probablty (1 δ)70 δ3 and stay at x(2) wth the remanng probablty. Transtons After x(3): Suppose that frm 2 advertses today. If z 1 Z ω 1 1 s observed, frm 1 goes to x(2) wth probablty (1 δ)1146 δ and stays at x(3) wth the remanng probablty. If z 1 Z ω 2 1 s observed, then go to x(4) wth probablty (1 δ)7095 δ17 and stay at x(3) wth the remanng probablty. Suppose next that frm 2 does not advertse today. If z 1 Z ω 1 1 s observed, frm 1 goes to x(4) wth probablty (1 δ)236 δ17 and stays at x(3) wth the remanng probablty. If z 1 Z ω 2 1 s observed, then go to x(4) wth probablty (1 δ)2747 δ17 and stay at x(3) wth the remanng probablty. The specfcaton of the transtons after x(4), x(5), x(6) s symmetrc so that we omt t. 12 The proof s avalable upon request. 22

23 the player plays the same acton; thus, players equlbrum behavor does not change even f the ntal envronment (prvate sgnals) s slghtly perturbed. Ths means that an ordnary equlbrum nduces the same behavor pattern regardless of the detals of the envronment, and Blume, Franco, and Hedhues (2011) argue that ordnary equlbra have a natural nterpretaton as routnes n ths sense. Our equlbrum strategy shares the same feature, snce players equlbrum behavor remans the same even f the ntal pror about the state changes. Chassang (2010) studes a relatonal contractng problem when players learn the detals of cooperaton over tme. Long-run behavor of hs equlbrum strategy s routne n the sense that players often end up wth neffcent actons and do not explore more effcent ones. Ths feature s smlar to the fact that players are locked n neffcent actons wth postve probablty n our equlbra. Remark 2. In ths secton, we have looked at games wth observed actons, but ths assumpton s not crucal. That s, a smlar equlbrum constructon apples to games wth prvate and almost-perfect montorng, where each player does not observe actons drectly but receves prvate nformaton about actons wth small nose. The dea s that even f small nose s ntroduced to the montorng structure, we can slghtly perturb the target payoffs v (k) and the transton probabltes so that the resultng automaton stll satsfes all the ndfference condtons. The formal proof s very smlar to the one for belef-free equlbra (Ely and Välmäk (2002) and EHO) and hence omtted. 4 Belef-Free Ex-Post Equlbrum In the prevous secton, we have constructed equlbrum strateges where each player s ndfferent over all actons gven any state ω and gven any past hstory of the opponent. An advantage of ths equlbrum constructon s that we do not need to compute a player s belef for checkng the ncentve compatblty, whch greatly smplfes the analyss. In ths secton, we generalze ths dea and ntroduce a noton of belef-free ex-post equlbra, whch s a specal class of Nash equlbra. Gven a strategy s S, let s h t denote the contnuaton strategy nduced by s when player s past prvate hstory was h t Ht. 23

24 Defnton 1. Player s strategy s s a belef-free ex-post best reply to s f s h t s a best reply to s h t n the nfntely repeated game wth the true state ω for all ω, t, and h t. In other words, a strategy s s a belef-free ex-post best reply to s f n each perod, what player does today s optmal regardless of the true state ω and of the opponent s past hstory h t. Ths means that player s acton today must be optmal regardless of her belef about the true state ω and the opponent s hstory h t. Obvously ths requrement s stronger than the standard sequental ratonalty, whch says that player s play s optmal gven her belef about ω and h t. Note that the automaton strateges constructed n the prevous secton satsfy ths property, snce the two actons are ndfferent (and hence optmal) gven any state ω and gven any past hstory h t. Defnton 2. A strategy profle s S s a belef-free ex-post equlbrum, or BFXE, f s s a belef-free ex-post best reply to s for each. In BFXE, a player s belef about the state and the past hstory s payoffrrelevant, and hence we do not need to compute these belefs for the verfcaton of ncentve compatblty. Ths exactly captures the man dea of the equlbrum constructon n the prevous secton. BFXE reduce to belef-free equlbra of EHO n known-state games where Ω = 1. Note that repetton of a statc ex-post equlbrum s a BFXE. Note also that BFXE may not exst; for example, f there s no statc ex-post equlbrum and the dscount factor s close to zero, then there s no BFXE. Gven a BFXE s, let R t A denote the set of all (belef-free ex-post) optmal actons for player n perod t,.e., R t s the set of all a A such that s (h t 1 ) = a for some h t 1 and for some s S whch s a belef-free ex-post best reply to s. Let R t = I R t, and we call the set Rt the regme for perod t. Note that the regme R t s non-empty for any perod t; ndeed, f an acton a s played wth postve probablty after some hstory h t 1, then by the defnton of BFXE, a s an element of R t. The equlbrum strateges n the prevous secton are a specal class of BFXE where the correspondng regmes are R t = A for all t. Of course, there can be a BFXE where R t s a strct subset of A. For example, when the stage game has a strct ex-post equlbrum a, playng a n every perod s a BFXE of 24