Repeated Sales with Multiple Strategic Buyers

Transcription

1 Repeated Sales with Multiple Strategic Buyers NICOLE IMMORLICA, Microsoft Research BRENDAN LUCIER, Microsoft Research EMMANOUIL POUNTOURAKIS, Northwester Uiversity SAMUEL TAGGART, Northwester Uiversity I a market with repeated sales of a sigle item to a sigle buyer, prior work has established the existece of a zero reveue perfect Bayesia equilibrium i the absece of a commitmet device for the seller. This couter-ituitive outcome is the result of strategic purchasig decisios, where the buyer worries that the seller will update future prices i respose to past purchasig behavior. We first show that i fact almost ay reveue ca be achieved i equilibrium, but the zero reveue equilibrium uiquely survives atural refiemets. This establishes that sigle buyer markets without commitmet are subject to market failure. However, our mai result shows that this market failure depeds crucially o the assumptio of a sigle buyer. If there are multiple buyers, the seller ca approximate the reveue that is possible with commitmet. We costruct a ituitive equilibrium for multiple buyers that survives our refiemets, i which the seller lears from past purchasig behavior ad obtais a costat factor of the per-roud Myerso optimal reveue. Moreover, we describe a simple ad computatioally tractable pricig algorithm for the seller that achieves this approximatio whe buyers best-respod. 1 INTRODUCTION It is ow commoplace for regular, repeated purchases to be made through large olie platforms. New parets purchase diapers mothly through Amazo Prime. Firms buy olie advertisig space millios of times per day through Google, Microsoft ad other advertisig markets. City-dwellers use delivery services like Foodler ad Istacart to purchase their meals ad groceries. Each platform is a corucopia of data, sice they ca readily observe how pricig decisios affect the purchasig behavior of customers, both i aggregate ad idividually. It is temptig for a platform to exploit this historical data, by usig the past behavior of idividual users to tue prices ad maximize reveue. However, usig revealed preferece data i this way rus afoul of game-theoretic cosideratios. If a regular customer kows that their behavior will impact the prices they will be offered i the future, they will aturally respod by chagig their behavior. It is therefore crucial to uderstad how forward-lookig customers will respod to price-learig algorithms, ad the implicatios for how a seller should use historical data to make pricig decisios. Cosider the followig simple ad fudametal istatiatio of the repeated-sales problem, coied the fishmoger problem [Devaur et al., 2015]. There is a sigle seller, ad each day the seller has a sigle copy of a good to sell. There is a sigle buyer, who has a private value v 0 for obtaiig the good each day, draw from a distributio kow to the seller. Crucially, the value does ot chage from oe day to the ext; the buyer has the same value for cosumig the good o every day. Each day, the seller posts a take-it-or-leave-it price, ad the buyer ca choose to accept or reject. The seller is free to set each day s price however she chooses, give the past purchasig behavior of the buyer. O ay day that the buyer rejects, the good expires ad the seller must discard it. The game is played for ifiitely may rouds; the buyer wishes to maximize total time-discouted utility, ad the seller wishes to maximize total time-discouted reveue. Mauscript submitted for review to ACM Ecoomics & Computatio 2017 (EC 17).

2 Nicole Immorlica, Breda Lucier, Emmaouil Poutourakis, ad Samuel Taggart 2 How should the seller set her price? If there is oly a sigle roud, the well-kow solutio is to post the Myerso price for the buyer s distributio, which maximizes expected reveue. I the dyamic settig, however, we caot expect the seller to post the Mysero price each roud. After all, if the buyer chose ot to purchase o the first day, the seller would aturally wat to lear from this iformatio ad set a lower price o the followig day. It is temptig to guess that the seller ca beefit from this oppotuity to lear, by offerig a variety of prices to gai iformatio about the value v, the use this kowledge to set a aggressive price just below v. However, a surprisig folklore result implies that such techiques ca ever be beeficial to the seller: the average per-roud reveue ca ever be higher tha the oe-roud Myerso reveue. Ituitively, the issue is that a ratioal buyer would respod to a explore/exploit strategy by pretedig at first to have a low value, passig up some opportuities to buy the item, i order to secure a lower price later o. Ideed, this strategic demad-reductio behavior is the essece of bargaiig, ad is commoly observed i practice. So what ca the seller do? To disetagle the strategic behavior of the buyer ad seller, it is ecessary to study equilibria. Sice ours is a repeated game with private iformatio, the appropriate solutio cocept is perfect Bayesia equilibrium (PBE). A formal defiitio is give i Sectio 2, but roughly speakig a PBE requires that the decisio take by each player at each poit i time, for ay observed history of prices ad purchases, is a best respose to the aticipated future behavior of the other player, give the seller s belief about the private value (which will deped o the observed behavior of the buyer). Determiig how the seller should set prices the reduces to uderstadig the structure of PBE. Sadly, prior work o equilibria for repeated sales have mostly geerated egative results. I particular, there exist PBE i which the seller posts a price equal to the miimum value i the support of the buyer s distributio, o every roud [Devaur et al., 2015, Hart ad Tirole, 1988, Schmidt, 1993]. For example, if the buyer s value is supported o [0, 1], the there is a PBE with zero reveue for the seller. This extreme ad couterituitive equilibrium is drive by a self-fulfillig prophecy: the buyer ever accepts ay positive price out of fear that doig so will lead the seller to charge very high prices i the future; as a result, the seller ifers that oly a buyer with very high type would ever accept a positive price, so the seller would ideed charge very high prices i respose. The formal details of the equilibrium are described i Sectio 3. This costructio illustrates that i the absece of commitmet power, a seller might suffer extremely low reveue i log-term iteractio with a buyer. We ote that this coclusio is remiisciet of the Coase cojecture; the primary differece is that the Coase cojecture refers to a durable good that a buyer will purchase oly oce, whereas i the fishmoger problem the good is perishable ad ca be repurchased each day [Coase, 1972]. This result is quite egative, but also usatisfyig sice the low-reveue equilibrium does ot appear to be predictive of real-world outcomes. Why do t we see this behavior i practice? Oe simplifyig assumptio i the model is the presece of oly a sigle buyer. Ideed, because there is oly oe buyer, it is possible for the seller to exploit the buyer s revealed preferece i a very targetted way. I cotrast, if the seller cotiues to sell by postig a sigle price, but that price will be faced by multiple buyers, the the opportuity for price-discrimiatio is dimiished. Ituitively, i a market with multiple buyers, each buyer is less worried about beig exploited directly, ad competitio gives a extra icetive to purchase eve though this is revealig a sigal to the seller. We therefore ask: would the presece of multiple buyers chage the structure of equilibrium?

3 Nicole Immorlica, Breda Lucier, Emmaouil Poutourakis, ad Samuel Taggart 3 Our Results. The existece of a zero-reveue equilibrium is discouragig, but we begi by showig that the sigle-buyer situatio is eve more dire tha that. Oe might woder whether the low-reveue equilibrium is simply a edge case, ad that better ad more plausible equilibria exist. Ideed, we establish a folk theorem that implies that ay amout of reveue betwee the trivial lower boud (that of postig the miimum-supported value every roud) ad that of Myerso pricig every roud ca be realized at a PBE of the game. However, despite the rich space of equilibria, we prove that the zero-reveue equilibrium is the uique equilibrium that survives a atural refiemet of the set of PBE. Specifically, it is the uique equilibrium i which the buyer uses threshold strategies (i.e., o each roud ad for ay offered price, a buyer purchases if ad oly if their value is sufficietly high), strategies are Markovia o-path (meaig that o the equilibrium path, the players strategies deped oly o their beliefs ad the curret price, ad ot the full history of past play), ad the seller offers prices i the support of buyers distributios. These refiemets have bee studied previously i the cotext of repeated sales (see [Fudeberg ad Tirole, 1983] ad [Hart ad Tirole, 1988]), ad are atural coditios for simple strategies. We iterpret this as strog evidece that the zero-reveue equilibrium, ad the market failure it implies, is actually a plausible ad atural outcome of the sigle-buyer repeated game. Mai Result: Multiple Buyers. We ext tur to studyig a multi-buyer variat of the Fishmoger problem. Suppose ow that there are 2 buyers, each buyer s value is draw iid from a kow distributio, ad these values are agai fixed over all rouds. The seller still has a sigle copy of the good for sale, ad sells that good by postig a sigle price each day. Each buyer idepedetly chooses whether or ot to purchase each day. If multiple buyers wish to purchase at the offered price o a give day, the oe of the acceptig buyers is chose uiformly at radom to make the purchase. 1 I cotrast to the sigle-buyer variat, we show that the seller ca achieve a costat fractio of the bechmark optimal reveue i a PBE that employs threshold strategies ad is Markovia o-path, survivig our refiemets. The equilibrium we costruct has a atural form, based upo a explore-exploit structure. The seller starts by settig a low price, ad slowly raises the price over time as log as at least two buyers purchase i each roud. Oce all but oe (or zero) buyers have stopped purchasig, the seller switches to a exploitatio phase i which she posts the highest price at which she believes a aget is guarateed to buy. Sice a aget is guarateed to buy, the seller stops learig iformatio about the buyers values, ad will post the same price every roud from that poit oward. This equilibrium structure sets up a atural optimizatio problem for the seller: how quickly should prices be icreased, give the way that ratioal buyers will respod at equilibrium? Typical of explore/exploit algorithms, the seller must balace the rate of learig with the reveue ultimately geerated i the exploitatio phase. Differet pricig policies will correspod to differet equilibria, with potetially differet amouts of reveue. We provide a approximatio result: for two buyers, if the distributio over buyer valuatios satisfies the stadard mootoe hazard rate (MHR) coditio, the we show how to compute prices (ad the correspodig equilibrium thresholds for the buyers) that geerate a costat fractio (1/3e 2 ) of the Mysero optimal reveue. For 3 bidders, we obtai a stroger guaratee (.12-approximatio) for a broader class of distributios (regular with moopoly 1 We choose to model the fishmoger problem as a pricig problem, as this is a commo approach take i practice. We ote that oe could alteratively model it as a geeral mechaism desig problem, which we leave as a directio for future research.

4 Nicole Immorlica, Breda Lucier, Emmaouil Poutourakis, ad Samuel Taggart 4 quatile at least 1/). The reveue of the prices we aalyze are a lower boud o the seller s reveue i our equilibrium. 1.1 Related Work Hart ad Tirole [Hart ad Tirole, 1988] iitiated the study of repeated sales ( retal, i their terms) with a sigle buyer ad a large but fiite horizo. They cosider a special case with just two possible values. They show that i equilibrium the seller will always post the smaller value for all but a costat umber of fial rouds. Schmidt [1993] geereralized their result to geeral discrete distributios. For a survey of this work ad the large body of work o closely related models, see the survey of Fudeberg ad Villas-Boas [2006]. Some variats iclude Kea [2001] ad Battaglii [2005] who aalyze the settig where the value of the buyer is ot costat but evolves accordig to a Markov process, ad Coitzer et al. [2012] who study the case where the buyers are short-lived ad give the optio to aoymize at a cost. Closest to our work is Devaur et al. [2015], which was the first attempt by the CS commuity to attempt to move beyod the strog egative results i the settig of Hart ad Tirole [1988] ad Schmidt [1993], ad the first to cosider cotiuous distributios. Like us, they aalyzed threshold equilibria, provig that o such equilibria exist for large but fiite umbers of rouds. They go o to study the case of partial commtimet, where the seller ca commit to ever icrease the price i the future. They prove existece of PBE for power law distributios ad provide reveue guaratees for the uiform distributio U[0, 1]. Note that our results ca be directly compared to Devaur et al. [2015] where istead of relaxig the commitmet assumptios we itroduce a extra buyer, ad provide reveue guaratees for a much larger family of distributios. 1.2 Discussio Our results have several iterpretatios. First, the folk theorem ad subsequet elimiatio of learig equilibria via refiemets ca be thought of a extesio of the coclusios of Hart ad Tirole [1988] ad Schmidt [1993] to ifiite horizo ad cotiuous distributios. This provides further justificatio for the modified assuptios Devaur et al. [2015] use to derive their results. Our work is similar i that we show that sigle-buyer market failure is fragile - we use extra buyers rather tha partial commitmet to support otrivial equilibria. Fially, we ote the prescriptive flavor of our results - our equilibrium ad reveue aalysis together provide a approximately optimal solutio to the problem of dyamic mechaism desig i the presece of distrustful buyers. 2 MODEL Game Descriptio ad Timig: The dyamic pricig game takes place i T rouds, where T may be ifiite. Each roud, there is oe item for sale, which must be allocated usig a commo price amog buyers. Before the game begis, each buyer i draws their value v i for the goods idepedetly ad idetically from some cotiuous distributio F which is commo kowledge. The value for allocatio remais uchaged from roud to roud. Each roud k the proceeds i the followig way: (1) The seller chooses a price p k 0, which is posted to the buyers. (2) Buyers simultaeously decide whether to accept p k. (3) The item is allocated uiformly at radom amog the agets who accept.

5 Nicole Immorlica, Breda Lucier, Emmaouil Poutourakis, ad Samuel Taggart 5 Utilities: Agets are risk-eutral expected utility maximizers. Utilities are liear i moey, additive across rouds, ad discouted by a commo discout factor (0, 1) over time. Formally: Seller: The seller s utility for a outcome to the above game is k : p k accepted k p k. Buyer: The buyer s utility for a outcome is k : i wis k v i. Note that all of our results, except the reveue aalyses of Sectios 5.2 ad 6.2, hold without modificatio if the seller holds a differet discout factor from the buyers. Moreover, the reveue aalyses exted i a atural way. Iformatio: We assume all iformatio ad outcomes are commo kowledge, with the exceptio of buyers values, which are privately held ad ukow by all other agets. Histories: A history of play at roud k, deoted h k is differet for buyers ad the seller, but geerally cosists of all past pricig ad purchasig decisios. Formally, h k cosists of cosists of the vector p[k 1] = (p 1,..., p k 1 ) of past prices, as well as the purchasig decisios of agets i each past roud, deoted D[k 1] = (D 1,..., D k 1 ), where D j = (D j 1,..., Dj ) {A, R} is the vector of accept/reject decisios for each aget i i roud j. Beliefs: The seller does ot kow ay buyer s values, ad buyers oly kow their ow. As metioed earlier, this ucertaity is modeled with a Bayesia prior. After every roud of play, the actios of agets may reveal iformatio about their private values, ad hece agets beliefs must be updated. We cosider oly outcomes where agets posteriors after each roud are shared, which is possible because all actios are commoly observed. The prior for v i after history h k, deoted µ k i ( hk ), is a probability measure over the support of F. The joit posterior at roud k is deoted µ k = i µ k i. After roud k, the seller believes values are distributed accordig to µ k, ad buyer i believes other buyers values are distributed accordig to µ k i. Strategies: Geerally, strategies are maps from histories ad private iformatio to actios i roud k: A seller strategy σ k S (hk ) specifies for every history h k a oegative price p k. Buyer i s strategy σ k i (hk, p k ; v i ) specifies for every buyer history a respose to price p k for every possible value of buyer i. Equilibrium: Our solutio cocept is Perfect Bayesia Equilibrium (PBE). Perfect Bayesia Equilibrium imposes joit requiremets o beliefs ad strategies: beliefs must be updated accurately give strategies, ad give beliefs, strategies must form a subgame-perfect equilibrium. Formally, a profile of strategies σ = (σ k S ( ), σk 1 ( ),..., σ k ( )) ad beliefs µ k ( ) for k = 0,..., T is a PBE if: Bayesia updatig: For every history h k, if there is some v such that µ k i (v hk ) > 0 ad σ k i (hk, p k ; v) = D k i, the µk i (v hk ) is computed accordig to Bayes rule. Importatly, for histories which would ot occur accordig to (σ k S ( ), σk 1 ( ),..., σ k ( )) uder ay realizatio of buyers values, beliefs may be arbitrary. Subgame perfectio: Let u S (σ h k, µ k ) deote the expected utility of the seller from the cotiuatio of the game from stage k accordig to σ, give that buyers values are distributed accordig to µ k (h k ). We require that for every alterate strategy σ S of the seller, we have that u S(σ h k, µ k ) u S (σ S, σ S h k, µ k ). Similarly if u i (σ h k, µ k, p k ; v i ) is the expected utility of a buyer with value v i offered price p k

6 Nicole Immorlica, Breda Lucier, Emmaouil Poutourakis, ad Samuel Taggart 6 uder history h k uder beliefs µ k i (hk ) o other buyers values, u i (σ h k, µ k, p k ; v i ) u i (σ i, σ i h k, µ k ; v i ) for every alterate strategy σ i. Simple Equilibria: Equilibria may i geeral be extremely complicated. We focus o equilibria satisfyig two refiemets: Markovia o path: A equilibrium is Markovia o path if o the equilibrium path, agets coditio their actios oly o the public beliefs ad their private iformatio, rather tha the complete history. Formally, for ay profile of buyer values v ad strategy profile σ, let h k ad h k be the histories geerated by σ uder v. If µ k = µ k, the p k = p k ad D k = D k. Threshold equilibrium: If a buyer will buy whe their value is v i, they will also buy with ay higher value. Formally, a PBE is a threshold equilibrium if for each history h k ad price p k, there is a threshold t i (h k, p k ) such that for each aget i, i accepts p k if ad oly if v i t i (h k, p k ). Note that i threshold equilibria, updated beliefs derived from o-path histories will be the value distributio F coditioed to some iterval [a, b] for each aget. For such equilibria, we will therefore summarize beliefs over aget i s value with the otatio Fa b to deote F coditioed to the iterval [a, b]. We refer to threshold equilibria which are Markovia o path as simple. Note that simplicity is a refiemet rather tha a restrictio of the strategy space. 3 FOLK THEOREM We first explore the space of Markovia o path threshold equilibria with o further refiemets. It is well-kow from previous work o the subject that there exists a equilibrium for the oe-buyer case i which the seller gets o reveue ad does ot lear aythig about the buyer s value. The buyers refuse all positive prices, ad deviatio is puished by the seller with high prices i the future. We refer to this as the o-learig equilibrium, ad for completeess preset the equilibrium i Appedix A. Formally, we have: Theorem 3.1 (Devaur et al. [2015]). For 1/2 ad ay umber of buyers there is a simple PBE i which the seller does ot lear, ad posts a price of 0 every roud. All buyers accept each roud. The o-learig equilibrium is cosidered uatural ad upredictive. I this ad the ext sectio, we offer a more uaced view. We prove a folk theorem: the o-learig equilibrium ca be used to eforce other eve less ituitive equilibria, icludig postig ay fixed price every roud. I other words, PBE is ieffective at rulig out commitmet. We solve this problem i Sectio 4, by offerig a additioal, ituitive refiemet which surprisigly elimiates all equilibria but precisely the o-learig equilibrium. This suggests that such behavior is a reasoable outcome to the game. Theorem 3.2 (Folk theorem). If +1, the for ay price p, there is a Markovia o path threshold PBE of the dyamic pricig game with buyers where the seller offers price p every roud o the equilibrium path, regardless of the actio of the buyer. This holds regardless of the iitial prior over buyers values. We prove the theorem i Appedix B. Ituitively, we use the o-learig equilibrium to commit the seller to a strategy. Oe way to uderstad the space of PBE is i terms of pairs of attaiable payoffs for the buyers ad the seller. Theorem 3.2 implies that the Pareto frotier of attaiable payoffs uder our two simplicity refiemets is at least as strog as

7 Nicole Immorlica, Breda Lucier, Emmaouil Poutourakis, ad Samuel Taggart 7 that attaiable from postig the same price each roud. A atural questio is whether there are PBE which surpass this frotier. The best kow bouds o the performace of PBE is a theorem due to Devaur et al. [2015], which we rephrase below. Theorem 3.3 (Devaur et al. [2015]). For ay target total expected buyer utility U ad reveue R attaiable i a PBE, there is a mechaism for the sigle-roud game i which the buyers attai total expected utility (1 )U ad the seller attais expected reveue (1 )R. The proof is costructive: give the PBE attaiig R ad U, the mechaism desiger may i essece simulate the PBE o the reported values of the sellers. I other words, PBE resemble sigle-shot mechaisms with stroger icetive costraits. Theorem 3.3 implies that the utility-reveue Pareto frotier for PBE caot geerally exceed that of the sigle-shot mechaism desig problem. For oe buyer, Theorem 3.3 implies that the folk theorem is tight - the utility ad reveue guaratees are the best possible. Theorem 3.2 implies a troublig multiplicity of equilibria, all with very differet outcomes for both the seller ad the buyers. It implies that further study of PBE is ot worthwhile without a maer of refiig away equilibria. We provide such a selectio tool i the ext sectio. 4 NON-ROBUSTNESS OF ONE-BUYER LEARNING EQUILIBRIA We ow cosider the case of oe buyer ad oe seller. I this settig, Theorem 3.2 proves that there are Markovia o path threshold equilibria which are totally efficiet, totally iefficiet, ad reveue-maximizig, as well as everythig i betwee. We argue that these equilibria exhibit uatural seller behavior. I particular, i the equilibria of Theorem 3.2, there are cotiuatios i which the seller offers prices at the upper boudary of or outside the support of the curret beliefs. We prove i this sectio that every simple equilibrium of the oe-buyer case, except those i which o learig occurs o the part of the seller, requires such odd behavior. This leaves oly equilibria i which the seller posts a price at the bottom of the support each roud. We first formalize atural seller behavior. Defiitio 4.1. A perfect Bayesia threshold equilbrium σ of the sigle-buyer has atural prices o-path (or simply atural prices) if for every o-path history h k with beliefs supported o [a, b], the seller s price σ k S (hk ) lies i [a, b). Though this requiremet might seem mild, it i fact suffices to elimiate all otrivial equilibria. Theorem 4.2. I the sigle-buyer game, let the value distributio F be supported o [a, b], with a > 0. If > 1/2, the i ay simple PBE with atural prices, the seller posts a every roud, which is accepted by all buyers. I other words, o learig will occur. Drivig the proof of Theorem 4.2 will be a idea from sigle-dimesioal mechaism desig: i equilibrium, allocatios are mootoe i type. I the repeated pricig settig, agets are maximizig their discouted total utility, which is a fuctio of discouted total allocatio ad discouted total paymets. These quatities satisfy the usual icetive costraits from mechaism desig, ad hece ituitios from mechaism desig carry over. We ow defie these formally: Defiitio 4.3. Give a PBE of the sigle-buyer game with some fixed value distributio, let: x k (v) be a idicator variable of whether or ot the buyer with value v purchases i roud k o the equilibrium path, ad let p k (v) be the o-path price offered that roud. The we may defie the followig for ay fiite i 1 ad ay j {1,..., }:

8 Nicole Immorlica, Breda Lucier, Emmaouil Poutourakis, ad Samuel Taggart 8. The total discouted allocatio: The total discouted paymets: The total discouted utility: X(v) = P (v) = k x k (v) k=1 k p k (v)x k (v) k=1 U(v) = vx(v) P (v) Lemma 4.4. I ay PBE of the oe-buyer game, the total discouted allocatio, paymets, ad utility (respectively X(v), P (v), ad U(v)), are odecreasig i v. To prove this claim, we ivoke a theorem of Myerso [1981]: Theorem 4.5 ([Myerso, 1981]). Let f( ), g( ) ad be fuctios from some iterval [a, b] (a > 0) to the positive reals, ad assume the followig holds for all v ad v i [a, b]: The the followig must be true: (1) f( ) is odecreasig i v. (2) g(v) = vf(v) v f(s) ds + g(a). a vf(v) g(v) vf(v ) g(v ). (1) The classic applicatio of this theorem sets f( ) to be the equilibrium allocatio probability i a sigle-item auctio ad g( ) the equilibrium expected paymets. We take a similar approach to prove Lemma 4.4. Proof of Lemma 4.4. Cosider a buyer with value v, who must choose a strategy. Amog their optios are to preted to have a differet value, say v, ad play the actios that value would play. Doig so would yield total discouted allocatio X(v ), total discouted paymets P (v ), ad total discouted utility U(v ). Sice the buyer is best respodig, it must be that vx(v) P (v) vx(v ) P (v ). We may ow ivoke Theorem 4.5. Mootoicity of X( ) follows from part (1) of the theorem, ad mootoicity of P ( ) from part (2). Notig that U(v) = v X(s) ds P (a) shows U( ) to be odecreasig as well. a We will oly use mootoicity of allocatios here. I Appedix C, we make heavier use of Lemma 4.4 to derive alterate sufficiet coditios uder which the coclusios of Theorem 4.2 hold. We ow show that atural prices iduces o-mootoicity aroud ay threshold t other tha the bottom of the support. I particular, we will show that there must exist a type below t with high cumulative allocatio, while the threshold type gets allocated strictly less ofte. This cotradicts Lemma 4.4. Lemma 4.6. For ay > 1/2, cosider ay simple PBE of the sigle-buyer game satisfyig atural prices with distributio supported o [a, b] ad first-roud threshold t > a. There exists a type t < t such that X(t ) > 1.

9 Nicole Immorlica, Breda Lucier, Emmaouil Poutourakis, ad Samuel Taggart 9 Proof. We argue by cotradictio. We will assume that for all t < t, X(t ) 1 ad use atural prices, alog with simplicity of equilibrium, to show that there is at least oe type less tha t who would prefer to deviate from the equilibrium. We first argue that we may assume the existece of some M such that all types i [a, t) have rejected by roud M. Assume this is ot the case. The let k ɛ be the earliest roud such that all agets i [a, t ɛ) have rejected at least oce. If it is the case that k ɛ as ɛ 0, the because > 1/2, it must be the case that there exists some t < t with X(t ) > 1, which would prove the lemma. Hece we may assume that the umber of rouds before every type i [a, t) would reject at least oce o the equilibrium path is fiite. Let M a idex such that all agets i [a, t) have rejected before roud M. We ow claim that there is a roud M M such that a positive measure of types accept i every oe of rouds 1,..., M 1, but all such agets reject i roud M. If ot, the it must be that a positive measure of agets accept i every roud up to ad icludig M, a cotradictio. Let the iterval of such agets be [a, t). (The upper boud beig t is implied by the threshold property.) Fially, we show that the existece of M, combied with atural prices ad the Markovia o path property, implies a profitable deviatio for some buyer with type i [a, t). First ote that the beliefs coditioed o acceptace i rouds 1,..., M 1 do ot chage after roud M, as all agets who accepted i rouds 1,..., M 1 will reject i roud M. Because beliefs do t chage, the Markovia i path property implies that actios do t chage - hece, i this cotiuatio, o aget i [a, t) accepts after roud M 1. O the other had, the requiremet of atural prices o path implies that the seller offers a price p [a, t) i every roud after M 1. Some type i (p, t) would clearly prefer to accept at least oce rather tha reject forever, yieldig a cotradictio. Proof of Theorem 4.2. Fix a > 1/2, ad cosider a roud of the game i which the beliefs are supported o [a, b] ad for which the buyer has a otrivial threshold t (i.e. above the bottom of the support of the curret beliefs). Subgame perfectio implies that we may assume this roud is the first. We kow from Lemma 4.6 that there is a value t < t such that X(t ) > 1. We will show that we may break ties so that X(t) = 1, which cotradicts Lemma 4.4. By the defiitio of threshold equilibrium, the buyer with type t accepts this roud. Natural prices implies that upo seeig a acceptace, the seller will ever price below t. It follows that the buyer with value t will ot get utility from ay subsequet roud. We may therefore assume they reject i every roud without chagig their utility. Moreover, such tiebreakig does t chage the icetives of the seller, as the type t buyer has measure 0. Hece, there is a equilibrium with X(t) = 1 ad X(t ) > 1 for some t < t, cotradictig Lemma 4.4. I Appedix C we give a alterate refiemet which similarly elimiates learig equilibria. Theorem 4.2 ad Appedix C together strogly suggest that with just oe buyer, oe should ot expect the seller to lear from purchasig behavior. This stregthes the coclusios of [Hart ad Tirole, 1988, Schmidt, 1993] ad exteds them to cotiuous distributios. I Sectios 5.1 ad 6, we show that these coclusios are critically depedet o the presece of oly a sigle buyer. With multiple buyers, we give a simple PBE with atural prices i which the seller lears from buyers actios. Moreover, the seller is able to use this kowledge to obtai reveue comparable with the reveue of the optimal auctio ru i every roud.

10 Nicole Immorlica, Breda Lucier, Emmaouil Poutourakis, ad Samuel Taggart 10 5 EQUILIBRIUM WITH TWO BUYERS I what follows, we describe a simple equilibrium with two buyers whose values are idepedet ad idetically distributed, with distributio fuctio F, ad discout factor 2/3. This equilibrium has two desirable properties: first, it survives the refiemets proposed i Sectio 4, ad ca therefore be cosidered robust. Secod, the seller gets otrivial reveue, which stads i cotrast to the robust o-learig equilibrium of the sigle-buyer case. We describe the mai ideas of the equilibrium i sectio 5.1 ad leave the full formal descriptio to Appedix D. I Sectio 5.2, we derive reveue guaratees for our equilibrium. 5.1 Equilibrium Descriptio The equilibrium cosists of two phases: a exploratio phase, ad a exploitatio phase. I the exploratio phase, which starts i the very first roud ad lasts util oe or more agets reject, the seller offers prices which will be rejected with positive probability. Cosequetly, the buyers respose to the seller s exploratio prices cause otrivial updates to the seller s beliefs. Oce a aget rejects, the equilibrium eters the exploitatio phase, which lasts util the ed of the game. If a sigle aget triggered the phase by rejectig, the seller igores this aget, ad posts a price at the bottom of the support of the beliefs for the stroger aget. This price is offered ad accepted for the rest of the game. If both agets rejected to trigger exploitatio, the the seller posts a price at the bottom of the commo support of their beliefs. Below, we iformally describe the optimizatio problems of the seller ad the buyers to covey the mai ideas of the equilibrium. Buyers. I ay give roud of the exploratio phase, either buyer has rejected a price yet. The seller offers a ew price, say, p, ad the buyers, whose beliefs are distributed i.i.d. accordig to some posterior F supported o [a, b], behave accordig to a threshold t solvig the equatio: ( ) 1 F (t) 2 + F (t) (t p) = 2(1 ) (t a)f (t). (2) The lefthad side represets the utility of a buyer with type t who accepts the price p, which is t p times the probability of wiig i the curret roud, with a cotiuatio utility of zero. The righthad side represets the threshold buyer s utility from rejectio - if the other buyer accepts, the the seller will oly post prices above t i the subsequet game yieldig zero cotiuatio utility for the threshold buyer. If both buyers reject, the they share the item for the rest of the game at price a. I Appedix D, we show that as expected, higher-valued buyers will prefer to accept, ad lower-valued buyers will reject. I the exploitatio phase, the seller targets the buyers with the strogest value distributio coditioed o past behavior, ad prices at the bottom of their support. The buyer icetives i this phase of the game are similar to those of the o-learig equilibrium. Seller. I the explore phase, the seller s optimizatio problem is a algorithmic pricig problem. Each roud, the seller must joitly choose a threshold ad a price satisfyig the threshold equatio (2), for curret beliefs F supported o [a, b]. They kow from the buyers strategies that such prices will be met with a threshold respose. It therefore suffices for the seller to maximize the followig value fuctio: R(a, b, p) = (1 F (t(p))) 2 (p+r(t(p), b))+2(f (t(p)) F (t(p)) 2 )(p+ t(p) 1 )+F (t(p))2 ( a 1 ), (3)

11 Nicole Immorlica, Breda Lucier, Emmaouil Poutourakis, ad Samuel Taggart 11 where R(x, y) is the optimal cotiuatio reveue from the equilibrium with values distributed accordig to F coditioed to [x, y] ad t(p) is the threshold correspodig to price p. The three terms of this fuctio represet the three possible outcomes to the curret roud: both buyers accept, exactly oe accepts, ad both reject. I our presetatio of the equilibrium, we leave the specific price path selected by the seller as the implicit solutio to the above optimizatio, ad ote that ay policy for choosig prices ad correspodig thresholds satisfyig the threshold equatio will support threshold behavior by the buyers. To fid a policy arbitrarily close to optimal, oe may discretize value space ad solve the Markov decisio problem associated with the value fuctio (3) by value iteratio, though we make o claim as to the computatioal efficiecy of this method. For a computatioally costraied buyer, we give i the ext sectio a particular threshold-supported pricig policy which obtais provably high reveue. 5.2 Reveue Guaratees We ow argue that if distributios are well-behaved, the reveue of the equilibrium outlied i the previous sectio (ad discussed i full detail i Appedix D) has high reveue. f(v) Specifically, we assume that the hazard rate 1 F (v) of the distributio is icreasig i v - a stadard assumptio i mechaism desig. As a bechmark, we use the reveue that the seller would obtai if they used the optimal auctio i every roud. For example, with two U[0, 1] buyers (as is cosidered i Devaur et al. [2015]), the seller obtais 5/12 every roud 5 i expectatio, yieldig a bechmark of 12(1 ). By Theorem 3.3, this reveue is a upper boud o the seller s reveue i ay PBE. The result is the followig: Theorem 5.1. Assume the value distributio F of the two buyers has a mootoe hazard rate, ad assume 2/3. I the equilibrium described i Sectio 5.1 ad Appedix D, the 1 seller obtais reveue which is at least 3e of the reveue of the optimal auctio ru each 2 roud. To argue the theorem, we first observe that i the exploratio phase of the equilibrium, the seller may offer ay price which has a threshold respose, ad the argumets i the previous sectio esure that buyers will be icetivized to adhere to threshold resposes. It follows that we may aalyze ay sequece of prices for the exploratio phase, ad as log as each price has a threshold respose, the resultig reveue will be a lower boud o the actual reveue of the seller. As our upper boud o our bechmark, we use 2(1 F (p ))p 1, where p is the sigle-buyer moopoly price. This correspods to sellig two items optimally every roud. Because the optimal reveue for the sale of a sigle item is cocave i the umber of buyers, this upper boud will always exceed the reveue of the optimal sigle-item auctio every roud. To relate our equilibrium to this upper boud, we imagie the seller choosig a sequece of prices which icreases the threshold quickly util it reaches p, after which the seller volutarily eters the exploit phase. Assumig both agets have value above p, the seller will receive reveue of p i perpetuity startig as soo as the threshold reaches this poit. By upperboudig the time it takes for this to occur, we ca lowerboud the expected reveue from this sequece of prices, ad therefore the reveue from the price sequece actually selected by the seller i equilibrium. First, cosider a arbitrary step of the explore phase, where the curret beliefs are over a iterval [a, b] with CDF Fa. b We argue that there is always a way for the seller to iduce a threshold t which lears quickly. Formally:

12 Nicole Immorlica, Breda Lucier, Emmaouil Poutourakis, ad Samuel Taggart 12 Lemma 5.2. I the explore phase with beliefs supported o [a, b], there always exists a price p a iducig the threshold t which satisfies F b a(t) = 1. Proof. Note that the threshold equatio for this stage implies: (t a)fa(t) b 1 = (F a(t) b + 1)(t p). Substitutig i F b a(t) = 1 ad solvig for p yields p = t (t a). To obtai our boud, we will assume the seller offers the followig sequece of prices: If there exists some p [a, b] iducig threshold p, offer p. Otherwise, offer a price which iduces t satisfyig F b a(t) = 1. We ow argue that such a sequece of prices will evetually iduce a threshold of p, if buyers values are above p. Lemma 5.3. If both sellers have value at least p, the the above sequece of prices evetually iduces threshold p. Proof. By Lemma 5.2, the seller will evetually reach a stage where the threshold t satisfyig Fa(t) b = 1 is greater tha p. We show that i this case, there is a price iducig a threshold t = p. To see this, assume the curret beliefs for buyers who have t rejected are distributed accordig to Fa b with support [a, b]. Let t be the threshold for which Fa(t b ) = 1, ad assume t > p. Note that the threshold equatio ca be rearraged as: t p t a = F a(t) b Fa(t) b The righthad side is obviously icreasig i t. Sice t > p, we therefore have: 0 < F a(p b ) Fa(p b ) < F a(t b ) Fa(t b ) < 1. If we set t = p, we see that the lefthad side rages from 0 at p = p to 1 at p = a, hece, there is a price p that iduces the desired threshold. We ca ow argue that uder the above sequece of prices, the exploratio phase will reach threshold p quickly if both agets have values above p. Formally: Lemma 5.4. Let x = 1. If both sellers have value at least p ad F (p ) 1 1/e the after 1/x + 1 rouds of the exploratio phase usig the price sequece defied above, we will have that the lower boud of the support is p. Proof. Let t j be the threshold iduced i the jth stage of the exploratio phase, ad assume t j < p. We will first lowerboud F (t j ). Note that F (t j ) is exactly the probability that a aget will reject oe of the prices i first j stages of the learig phase. This probability ca also be writte as F (t j 1 ) + (1 F (t j 1 ))x, sice coditioed o a aget acceptig the first j 1 prices, the price i roud p j was chose to be accepted with probability x. This yields the recurrece: F (t j ) = F (t j 1 ) + (1 F (t j 1 ))x, which is solved by F (t j ) = 1 (1 x) j. If we set j = 1/x, we obtai F (t j ) = 1 (1 x) 1/x 1 1/e. (4)

13 Nicole Immorlica, Breda Lucier, Emmaouil Poutourakis, ad Samuel Taggart 13 It therefore must be that after at most 1/x rouds, the exploratio phase termiates with the threshold reachig p. I the subsequet roud, the lower boud of the support will be the previous roud s threshold, p. Lemma 5.5. If F (p 1 2 ) 1 1/e the the equilibrium obtais reveue at least 1 3e p (1 F (p )) 2. Proof. We lower boud the reveue with the reveue from the sequece of prices described above. The probability that both agets have values above p, ad therefore that the threshold reaches p, is (1 F (p )) 2. By Lemma 5.4 the discout factor after reachig p is at most e. After p is reached the seller prices the item at p for 1 all remaiig rouds ad the item is accepted with probability oe for a total of 1 d p reveue. Overall the reveue obtaied by the seller with this price sequece is therefore at 1 2 least 1 3e p (1 F (p )) 2. Proof of Theorem 5.1. Let OPT deote the total reveue from ruig the optimal auctio for two buyers o F. Our bechmark for reveue is OPT/(1 ). By cocavity of the reveue we have that OP T 2p (1 F (p )). By Lemma 5.5 the equilibrium gets reveue e p (1 F (p )) 2 OPT 1 1 3e (1 F (p )) For distributios satisfyig the mootoe hazard rate assumptio, it is a stadard fact that F (p ) 1 1/e. We therefore have that the reveue of our equilibrium is at least OPT 3(1 )e OPT 2 3e, provig the theorem. 2 6 EQUILIBRIUM WITH MANY BUYERS I Sectio 5, we showed that the dyamic pricig game with 2/3 supports a equilibrium with otrivial reveue ad learig. Moreover, i cotrast to the oe-buyer case, this equilibrium is robust to the refiemets laid out i Sectio 4 ad Appedix C. We ow geeralize these coclusios to games with 3 buyers ad. I particular, we give a recursively-costructed equilibrium, built o the two-buyer equilibrium as a base case, i which the seller obtais o-trivial reveue, learig occurs, ad which survives the refiemets of Sectio 4 ad Appedix C. We derive reveue guaratees i Sectio 6.2 ad Appedix F. 6.1 Equilibrium Descriptio Much like the two-buyer equilibrium, the multibuyer versio has a exploratio phase ad a exploitatio phase. I the exploratio phase, the seller posts a price which iduces a threshold respose amog the buyers. Those buyers that reject are priced out of the game, while the k buyers who accept cotiue i the k-buyer versio of the equilibrium. This cotiues util all or all but oe buyer rejects, at which poit the seller exploits the buyers who most recetly accepted, postig the bottom of their support i perpetuity. We give the full descriptio of the equilibrium i Appedix E. Buyers. I the exploratio phase, the seller targets the set of buyers S who have ot yet rejected a price. The ot i S, i.e. those who have rejected, are igored by the seller ad priced out of the market. We may therefore cotiue our discussio with S =. For the +1

14 Nicole Immorlica, Breda Lucier, Emmaouil Poutourakis, ad Samuel Taggart 14 sellers i S, the price offered by the seller iduces a threshold respose. For buyers, the threshold equatio is P (t)(t p) = F (t) 1 (t a), (5) 1 where P (t) = 1 ) j=0 F (t) j (1 F (t)) 1 j, ad F is the commo distributio of 1 j ( 1 j buyers i S, supported o [a, b]. The lefthad side agai correspods to the utility of a buyer with type t from acceptig, which is made more complicated ow by the fact that aywhere from 0 to 1 other buyers could accept. Meawhile, the righthad side is agai the utility of the threshold aget from rejectig, which is oly positive from the case where all other buyers also reject, i which case all buyers split the item i perpetuity. I the exploitatio phase, the seller cotiues targetig either the set of agets who have ever rejected, or the agets who oly rejected i the previous roud, if everyoe rejected that roud. As i the two-buyer equilibrium, the buyer icetives are similar to the o-learig equilibrium. Seller. As i the two buyer equilibrium, the seller is faced with a algorithmic pricig problem, where each roud they are restricted to prices which support threshold resposes. To do so, they optimize the value fuctio ( ) R(a, b, p, ) = F (t(p)) ( a 1 ) + (1 F (t(p)))f (t(p)) 1 p + t(p) 1 + j=2 ( j) (1 F (t(p))) j F (t) j (p + R(t(p), b, j)), where t(p) is the correspodig threshold price solvig (5) for p ad R(t(p), b, j) is the expected cotiuatio reveue from the j-buyer equilibrium with the belief distributio coditioed to [t(p), b]. Note that for 3 buyers, the recurrece has a additioal argumet, which is the umber of players who have ot yet rejected. As before, this problem ca be solved to arbitrary precisio usig discretizatio ad value iteratio. I Sectio 6.2, we exted the ideas of Sectio 5.2 to give a computatioally efficiet ad simple strategy which secures for the seller a costat fractio of the maximum reveue possible while lowerboudig the reveue of the seller s best respose. 6.2 Approximate Optimality of Multibuyer Equilibrium I this sectio, we state our reveue guaratees, assumig buyers value distributios are sufficietly well-behaved. As with 2 buyers, the proof demostrates a squece of prices which the seller could select i equilibrium, for which buyers would behave accordig to their threshold resposes. This lowerbouds the reveue of the seller if they best respod. Ituitively, our sequece of prices lears from accept decisios of the buyers as aggressively as possible, seekig high thresholds util it is possible to offer the price with quatile 1/ accordig to the origial distributio. Our aalysis shows that this sequece of prices quickly reaches this price, ad that offerig this price every roud approximates the optimal reveue. Formally: Defiitio 6.1. A distributio is regular if the Myerso virtual value fuctio φ(v) = 1 F (t) v f(t) is icreasig i v. Theorem 6.2. For all regular value distributios F with moopoly quatile at least 1/, the equilibrium described i Sectio 6 ad Appedix E ears the seller at least a l (2/3)1 l(1 1/ 2).1202-fractio of the reveue of the reveue-optimal auctio for F ru each roud.

15 Nicole Immorlica, Breda Lucier, Emmaouil Poutourakis, ad Samuel Taggart 15 We reiterate that all MHR distributios are regular with moopoly quatile at least 1/e. Hece, the requiremets for Theorem 6.2 are weaker tha for two buyers. The proof ca be foud i Appedix F. REFERENCES Saeed Alaei Bayesia Combiatorial Auctios: Expadig Sigle Buyer Mechaisms to May Buyers. I IEEE 52d Aual Symposium o Foudatios of Computer Sciece, FOCS 2011, Palm Sprigs, CA, USA, October 22-25, DOI: Marco Battaglii Log-term cotractig with Markovia cosumers. The America ecoomic review 95, 3 (2005), Yag Cai ad Costatios Daskalakis Extreme-Value Theorems for Optimal Multidimesioal Pricig. I IEEE 52d Aual Symposium o Foudatios of Computer Sciece, FOCS 2011, Palm Sprigs, CA, USA, October 22-25, DOI: R. H. Coase Durability ad Moopoly. The Joural of Law ad Ecoomics 15, 1 (1972), DOI: Vicet Coitzer, Curtis R Taylor, ad Liad Wagma Hide ad seek: Costly cosumer privacy i a market with repeat purchases. Marketig Sciece 31, 2 (2012), Nikhil R. Devaur, Yuval Peres, ad Balasubramaia Siva Perfect Bayesia Equilibria i Repeated Sales. I Proceedigs of the Twety-Sixth Aual ACM-SIAM Symposium o Discrete Algorithms, SODA 2015, Sa Diego, CA, USA, Jauary 4-6, DOI: Drew Fudeberg ad Jea Tirole Sequetial bargaiig with icomplete iformatio. The Review of Ecoomic Studies 50, 2 (1983), Drew Fudeberg ad J Miguel Villas-Boas Behavior-based price discrimiatio ad customer recogitio. Hadbook o ecoomics ad iformatio systems 1 (2006), Oliver D Hart ad Jea Tirole Cotract reegotiatio ad Coasia dyamics. The Review of Ecoomic Studies 55, 4 (1988), Joh Kea Repeated bargaiig with persistet private iformatio. The Review of Ecoomic Studies 68, 4 (2001), Roger Myerso Optimal Auctio Desig. Mathematics of Operatios Research 6, 1 (1981), Klaus M Schmidt Commitmet through icomplete iformatio i a simple repeated bargaiig game. Joural of Ecoomic Theory 60 (1993), A NO-LEARNING EQUILIBRIUM I this appedix, we give the full descriptio of the o-learig equilibrium. The seller s strategy ca be foud i Algorithm 1, ad the buyer s strategy i Algorithm 2. The beliefs which support this strategy profile are simple: if a buyer has ever accepted a positive price or rejected 0 (either of which is o-path), the seller believes the buyer s value is 1. Otherwise,

16 Nicole Immorlica, Breda Lucier, Emmaouil Poutourakis, ad Samuel Taggart 16 the seller lears othig about the buyer s value ad offers the item for free every roud. Moreover, it is clear from ispectio that this equilibrium survives the simplicity refiemet. ALGORITHM 1: Zero-Reveue Equilibrium - Seller s Strategy Iput : Purchasig history h k, belief support [a, b]. Output : Price p k+1 if Buyer has ever accepted a positive price the p k+1 = 1; if Buyer has ever rejected a price of 0 the p k+1 = 1; p k+1 = 0; ALGORITHM 2: Zero-Reveue Equilibrium - Buyer s Strategy Iput : Purchasig history h k, belief support [a, b], value v, price p k+1 Output : Purchasig decisio for roud k + 1 if p k+1 = 0 the Accept; if p k+1 > 0 the if Buyer has ever accepted a positive price the Accept; if Buyer has ever rejected a price of 0 the Accept; Reject; B PROOF OF THEOREM 3.2 We will explicitly costruct a equilibrium where the seller offers price p every roud, o matter the buyer s actio. We give the buyer s strategy i Algorithm 4, ad the seller s strategy i Algorithm 3. Beliefs are simple - o-path, they are updated after the first buyig decisio ad remai costat thereafter. If the seller has caused a off-path history by postig a price other tha p, the they expect positive prices to be rejected for the rest of time, as i the zero-reveue equilibrium. As i the latter equilibrium, if a buyer accepts a