Analyzing Scrip Systems

Transcription

1 Submitted to manuscript Pease, provide the manuscript number! Anayzing Scrip Systems Kris Johnson Operations Research Center, Massachusetts Institute of Technoogy, David Simchi-Levi Engineering Systems Division, Department of Civi & Environmenta Engineering and the Operations Research Center, Massachusetts Institute of Technoogy, Peng Sun The Fuqua Schoo of Business, Duke University, Scrip systems provide a non-monetary trade economy for exchange of resources. We mode a scrip system as a stochastic game and study system design issues on seection rues to match potentia trade partners over time. We show the optimaity of one particuar rue in terms of maximizing socia wefare for a given scrip system that guarantees payers incentives to participate. We aso investigate the optima number of scrips to issue under this rue. In particuar, if the time discount factor is cose enough to 1, or trade benefits one partner much more than it costs the other, the maximum socia wefare is aways achieved no matter how many scrips are in the system. When the benefit of trade and time discount are not sufficienty arge, on the other hand, inecting more scrips in the system hurts most participants; as a resut, there is an upper bound on the number of scrips aowed in the system, above which some payers may defaut. We show that this upper bound increases with the discount factor as we as the ratio between the benefit and cost of service. Finay, we demonstrate simiar properties for a different service provider seection rue that has been anayzed in previous iterature. Keywords: Repeated games, stochastic games, dynamic program. 1

2 Artice submitted to ; manuscript no. Pease, provide the manuscript number! 1 1. Introduction Scrips are coupons that are used in pace of currency to exchange goods and services; typicay, they cannot be exchanged for money, and therefore their soe use is for the good or service which they are intended for. In this paper we study scrip systems, which are markets that use scrips rather than money to exchange goods and services. Such markets are typicay impemented when the use of governmenta currency is impractica or undesirabe. One exampe of a scrip system is that of the Capito Hi babysitting co-op, documented in Sweeney and Sweeney A group of about 150 married coupes with chidren who ived in the Washington, D.C. area were tired of ooking for and hiring babysitters to watch their chidren every time they wanted to enoy a night out, so they decided to oin together to form a babysitting co-op, managed by a scrip system. Every coupe in the co-op was given an initia amount of coupons, or scrips, to pay for babysitting service by another coupe in the co-op who was wiing to provide the service. Free riding was mitigated in the system since a coupe coud ony enoy the service when they had coupons, and earning coupons required providing services. It turns out that this babysitting co-op experienced market crashes simiar to many other types of markets. Initiay, they distributed too few coupons and trade rarey occurred. This was ikey because either a coupe ran out of coupons to pay for service or they hoarded the few coupons for ater specia situations. In order to sove this issue, the group coectivey decided to give every coupe additiona coupons, to the point that each coupe vaued one additiona coupon too itte, and therefore was not wiing to provide services to earn an additiona coupon. This story was popuarized by Krugman 1999, who reated the scrip system s crashes to economic sumps and monetary poicies. Since crashes occurred due to having the wrong number of scrips in the system, a natura and important question to ask is, therefore, what is the right number of scrips in a system? There are many other exampes of scrip systems that have been impemented in resource exchange environments, such as onine peer-to-peer systems, to prevent free riding see, e.g., Vishnumurthy et a. 2003, Gupta et a. 2003, Ioannidis et a. 2003, Sirivianos et a. 2007, Peterson and Sirer 2009,

3 2 Artice submitted to ; manuscript no. Pease, provide the manuscript number! Beenkiy et a. 2007, and Satoshi akamoto The idea is simiar to that of the babysitting co-op exampe, where scrips are credited to users who provide products/services e.g., share fies and debited when users receive them e.g., access other s fies, so that in the ong-run, the amount of products/services participants can receive matches what they provide. Other common uses of scrip systems incude onine resource aocation environments; for exampe, grid computing networks see, e.g., Brunee et a. 2006, research testbeds see, e.g., Chun et a. 2005, AuYoung et a and AuYoung et a. 2007, distributed database systems Stonebraker et a. 1996, and privacy-enhancing technoogies, where a vounteer network of servers are needed to route Internet traffic in order to concea the user s IP address Humbert et a Some schoars suggest that the academic ourna refereeing process may aso be managed by a scrip system! Ceary scrip systems are important in a variety of settings, yet there has been reativey itte work done to anayze their behavior. In a of these exampes, the number of scrips inected into the system is a determinant of system performance. As seen from the babysitting co-op exampe, having too few or too many scrips in the system can cause market crashes. Another important question arises in the onine appications of scrip systems how shoud the service provider be chosen? In this paper, we anayze a cass of scrip systems and provide insights regarding system design: the way service providers shoud be seected as we as the optima number of scrips that shoud be used in the system. We show the optimaity of one particuar service provider seection rue for a given scrip system in terms of maximizing socia wefare, i.e., tota utiity of a payers in the system over time, whie making sure that payers have the incentive to foow the rues of the scrip system. For scrip systems where the time discount factor is cose enough to 1, or trade benefits one partner much more than it costs the other, the maximum socia wefare is aways achieved no matter how many scrips are in the system. As a resut, system optima performance can be achieved under individua incentive constraints. When the benefit of trade and time discount are not sufficienty arge, on the other hand, inecting more scrips in the system hurts most participants; in this case, there is an upper bound on the number of scrips aowed in the system, above which some payers may defaut. We

4 Artice submitted to ; manuscript no. Pease, provide the manuscript number! 3 show that this upper bound increases with the discount factor as we as the ratio between the benefit and cost of service. In the remainder of this section, we provide a iterature review on the modeing and anaysis of scrip systems. The basics of our mode, as we as the optima centraized contro poicy, are introduced in Section 2. We then study the stochastic game payed in the absence of a centra panner in Section 3 and demonstrate that the centra optima soution can be achieved in the game when the discount factor is sufficienty arge. Section 4 investigates the impact of the number of scrips in the system. Finay, Section 5 concudes the paper with a summary of our resuts and potentia areas for future work Literature Review As mentioned before, in the computer science iterature, severa papers have been written regarding the appication of scrip systems and their impementation. There have been, however, ony a few papers that formay mode and anayze scrip systems. Aperis and Johari 2006 is one of the eariest papers that studies a peer-to-peer fie sharing system as an exchange economy. They propose a static game where users decide upoading/downoading rates, and they study the market cearing prices in equiibrium. The papers that are most cosey reated to ours are Friedman et a. 2006, Kash et a. 2007, Kash et a. 2012a and Kash et a. 2012b. In fact, this stream of papers motivated our study. Friedman et a is one of the first papers that anayzed payers strategies as we as design characteristics of scrip systems in a stochastic setting. Their mode considers a homogeneous popuation of payers in a scrip system with a finite number of scrips. Services are provided at a fixed price in terms of the number of scrips and incur a fixed and identica utiity gain oss to each payer receiving providing the service. They consider an infinite time horizon game with discounting. In each period, a payer is chosen uniformy at random to request service, whie a other payers have the option to vounteer as a service provider, one of whom is seected randomy. Our mode, described in Section 2, is simiar to their mode with one maor departure. Instead

5 4 Artice submitted to ; manuscript no. Pease, provide the manuscript number! of having a payer chosen randomy to provide service, we aow the system designer to choose a service provider seection rue for a payers to foow; then we determine the number of scrips that shoud be inected in the system accordingy. At the end of our anaysis, we return to their assumption of randomy seecting a service provider, and we show simiar resuts regarding the number of scrips that shoud be inected into such a scrip system. A key assumption adopted in Friedman et a is that each payer chooses to vounteer to provide service foowing a threshod strategy. That is, a payer is wiing to vounteer to provide service ony if his scrip stock is ower than a threshod number of scrips. The paper shows that when the discount factor is cose enough to one, there exists an ϵ-ash Equiibrium in which each payer foows such a threshod strategy. One impication of the threshod strategy is that there exists a tota threshod number of scrips in the system, above which no trade occurs and therefore the system wi experience a market crash. Our mode, on the other hand, does not restrict payers strategies to be of threshod type; rather, we show the existence of a tota threshod number of scrips as a resut. Foow-on work in Kash et a. 2012b shows that socia wefare increases as the number of scrips in the system increases up to this threshod of tota scrips, after which the socia wefare drops to zero due to the market crash. Kash et a. 2012a, where the mode is further generaized to incude mutipe payer types, characterizes each payer s threshod that achieves the optima socia wefare. In Kash et a. 2012b, the authors further anayze the impact of atruists and hoarders in the scrip system. Motivated by the appication of scrip systems to privacy-enhancing technoogies, Humbert et a anayzes scrip systems where each service request requires n providers to satisfy. This mode directy extends the mode in Kash et a to require n service providers instead of one. The authors show simiar resuts to those in Kash et a incuding the existence of an ϵ-ash Equiibrium where a payers act according to a threshod poicy. Finay, recent iterature in economics aso studies scrip systems, motivated by the babysitting co-op, as the micro-foundation of monetary poicy. Hens et a provides an overview of this ine of work. There are quite a few differences between their mode and ours. The main difference

6 Artice submitted to ; manuscript no. Pease, provide the manuscript number! 5 is that they assume no cost to provide service, whie we assume providing service incurs a negative utiity, and the ratio between the benefit and cost of service pays an important roe in our mode. Simiar to Friedman et a. 2006, Hens et a aso focuses on the random service provider seection rue, which is, one may argue, simpe and reaistic in many economic settings. 2. Basic Mode Description Consider an economy with a non-empty set of payers. Each payer i has r i scrips, where we abuse notation to use to represent the number of payers as we. In each time period, one payer at random wi be the service requester with probabiity 1/, and a other payers types are 0. In any given period, the service requester is abe to obtain utiity u if the payer chosen as a service provider one of the type 0 payers is wiing to sacrifice utiity c < u in exchange for 1 scrip and if the service requester has a scrip and is wiing to pay 1 scrip for service. Assume a time discount factor for the system. Denote the state of the system to be s = r,, where r is the vector of scrip stocks r i i and is the service requester. Thus, the tota number of scrips in the system is R = r i i, which does not change over time. State space S is the coection of a possibe states s. We denote a stationary poicy π to map the state r, into a probabiity distribution on the remaining payers other than. The purpose of π is to seect a service provider for any possibe state s. Denote set Π to represent the set of admissibe poicies. At this point it is worth introducing a particuar service provider seection rue in Π, the minimum scrip seection rue π, where in each round, the type 0 payer with the east number of scrips is seected as the service provider. If more than one type 0 payer has the fewest scrips, one payer is chosen randomy from them with equa ikeihood to be the service provider. Another exampe of a seection rue in Π is the random provider seection rue, a common seection rue considered in previous iterature, where a payer is seected as the service provider uniformy at random, independent of her scrip stock. We first demonstrate that the minimum scrip seection rue, π, maximizes socia wefare among

7 6 Artice submitted to ; manuscript no. Pease, provide the manuscript number! a poicies in Π in a centra panner setting, where we assume that a hypothetica centra panner not ony chooses the service provider, but aso decides whether trade shoud occur Centra Panner Setting ow we consider a hypothetica centra panner who tries to maximize the tota socia wefare over an infinite time horizon with discount factor 0, 1. In each period, given state r,, the centra panner decides whether trade shoud occur when payer has at east one scrip, and if so, chooses a payer i to be the service provider. Let e k be a vector in R with every component equa to zero except the kth component equa to one. Jr, is the system socia wefare from state r,, and J r = =1 Jr, is the tota system socia wefare across a states with the same vector of scrip stocks. The corresponding Beman equation is: { max{maxi u c + Jr, = J r + e i e, J r}, r > 0 J r, r = 0, 1 where J r = Jr,. In the Beman equation 1, the outer maximization decides whether to trade a scrip for service, =1 whie the inner maximization seects the trading partner. Equivaenty, we can express the Beman equation in terms of J as J = ΓJ, where ΓJ r = r J r + { max u c + max J r + e i e, i:i : r 1 in which r is the number of payers with positive scrips in r. } J r, 2 ext we define the foowing properties for a generic function J defined on the integer simpex {r Z + : i r i = R}, and show that the optima system socia wefare function J satisfies them, which further impies that the minimum scrip seection rue is optima in the centra panner setting. C1 Symmetry: For any r and r with r k = r k for a k except i,, where r i = r and r = r i, J r = J r. 3 C2 For any r and payers i and such that r i > r, J r e i + e J r. 4

8 Artice submitted to ; manuscript no. Pease, provide the manuscript number! 7 C3 For any scrip distribution r and payer with r > 0, J r max J r + e i e u c. 5 i:i Furthermore, for a payer i, denote set I ri to contain a payers with at most r i scrips. If I ri r r i I ri 1 + 1, 6 then for any payer I ri we have J r J r + e i e u c. 7 Proposition 1. The soution J to the Beman equation J = ΓJ satisfies conditions C1, C2, and C3. The proof is based on showing that for any function J that satisfies these properties, so does ΓJ. The detaied proof is presented in the Appendix. Proposition 1 impies the foowing characterization of the centra panner s optima poicy. Theorem 1. In the centra panner setting, trade aways occurs, and the minimum scrip seection rue π is optima. Proof: Condition 5 for J suggests u c + max m:m J r + e m e J r, for r > 0, which impies that trade aways occurs if the service requester has at east one scrip to pay for service. Therefore, for a vector r with r > 0, Beman equation 1 becomes J r, = u c + max i:i J r + e i e. 8 Condition C2 further impies that the optima i that soves the maximization in 8 must be the one with the east number of scrips. Q.E.D.

9 8 Artice submitted to ; manuscript no. Pease, provide the manuscript number! 0 2 minimum scrip seection rue random scrip seection rue og Pr no trade R Figure 1 Probabiity of no trade with = 3 payers. The intuition behind Theorem 1 is cear. In order to maximize socia wefare, the centra panner aways prefers trading, which generates u c > 0, over not, whenever possibe. Trade cannot occur when the service requester has no scrips. The minimum scrip seection rue tries to baance the scrip hodings among payers, therefore minimizing the chance that a payer runs out of scrips. An important observation is that with more scrips in the system, there is a ower probabiity that a service requester wi have no scrips, thus resuting in a higher probabiity of socia wefare increasing by u c in each round; this impies that in the centra panner setting, socia wefare increases with the number of scrips in the system. In particuar, the soid curve in Figure 1 iustrates that the probabiity that trade does not occur, due to ack of scrips by the service requester, appears og ineary decreasing with the number of scrips R in the system when R is sufficienty arge; each point on this curve is obtained through standard numerica iteration approaches to compute steady state probabiities for a 3 payer Markov Chain. In comparison, the dotted curve represents the same probabiity foowing the random provider seection rue. It is cear that the chance of no trade is much ower with the minimum scrip seection rue than the random provider seection rue. More generay, we are abe

10 Artice submitted to ; manuscript no. Pease, provide the manuscript number! 9 to obtain a cosed form expression of the probabiity of no trade foowing the random provider seection rue see Proposition 7 in the Appendix, and Kash et a. 2012a Lemmas A.3 and A.4 for steady state probabiities in more genera settings. 3. Stochastic Game and Optimaity of Minimum Scrip Seection Rue In this section, we remove the existence of a centra panner, and we formay define the game in which the pair of payers seected to be potentia trade partners can decide whether to exchange a scrip for service. We then demonstrate that even in the absence of a centra panner, the minimum scrip seection rue, π, achieves the maximum socia wefare obtained in the centra panner setting under certain conditions, which corresponds to the Fok Theorem for stochastic games see, e.g., Fudenberg and Tiroe Furthermore, we show that in the game setting there exist threshods on the discount factor as we as the reative benefit of receiving a service above which a payers have the incentive to trade a scrip for service whenever the service requester has at east one scrip Stochastic Game ow we consider a game in which a payers foow a particuar service provider seection poicy π Π; we assume that no payers coude. We focus on a stochastic game setting where the panning horizon is infinite, due to the we known distinction between finite horizon stochastic games and infinite horizon stochastic games see, e.g., Fudenberg and Tiroe In our setting, if the panning horizon was finite, given that scrips have no savage vaue at the end of the horizon, no type 0 payer woud offer service and suffer a negative utiity c in the ast period. Using backward induction and foowing the same ogic throughout the time horizon, no trade ever occurs in any finite horizon game. In the infinite horizon setting, each payer s strategy may depend on the entire history of the game. In particuar, we denote vector θ = r,, i, Ω to represent the state of game at each period, in which r is the scrip distribution vector, is the service requester, i is the payer seected to be the service provider according to seection rue πr,, and set Ω contains a payers who i have never refused to trade a scrip for service with anyone within set Ω, and ii have never traded with anyone

11 10 Artice submitted to ; manuscript no. Pease, provide the manuscript number! not in set Ω at the time of trade 1. Obviousy, in the beginning of the time horizon, Ω contains a the payers in the game. Further denote set D k θ to represent payer k s action space at state of game θ. In particuar, if payer has positive scrips r > 0, she can choose whether or not to give one scrip to payer i in exchange for service d = 1, or not to spend the scrip for service d = 0; therefore, D r,, i, Ω = {1, 0}. Payer i, on the other hand, can choose whether to accept the scrip and serve payer d i = 1 or not d i = 0; therefore, D i r,, i, Ω = {1, 0}. Any payer other than i and must take no action, so D k r,, i, Ω = {0} for k i,. ote that at state of game θ = r,, i, Ω, trade of a scrip for service occurs ony if r d i d > 0. Denote the action profie Dθ = k D k θ, with eement dθ = d k θ k Dθ. Given state of game θ = r,, i, Ω and action profie d, singe period utiities for payers i and depend on whether or not trade occurred; u i θ, d = c and u θ, d = u if r d i d > 0 trade occurred, and u i θ, d = u θ, d = 0 if r d i d = 0 trade did not occur. In either case, for any payer k i,, the utiity u k θ, d = 0. Given service provider seection rue π, each payer tries to maximize her own tota utiity over an infinite horizon with discount factor 0, 1. Foowing Myerson 1997, it is sufficient to consider stationary strategy profies, i.e., strategies that ony depend on state of the game rather than the entire history. Specificay, consider stationary strategy τ k for payer k that maps state of the game θ to a particuar action d k D k θ and the corresponding poicy profie for a payers, denoted as τ = τ k. Let v k τ, θ denote payer k s expected -discounted average payoff if payers commit to stationary poicy profie τ and the current state of game is θ. Further denote Y k τ, d k, v k, θ to represent payer k s discounted average payoff starting from state θ if a payers commit to stationary poicy τ except payer k, who deviates in the first round with action d k, Y k τ, d k, v k, θ = 1 u k θ, dk, τ k θ + v k τ, r,, πr,, ωθ, d k, where ωθ, d k = Ω if d k = 1 or k i,, and ωθ, d k = Ω \ k if d k = 0 and k = i or.

12 Artice submitted to ; manuscript no. Pease, provide the manuscript number! 11 Theorem 7.1 of Myerson 1997 states that the stationary strategy τ is an equiibrium strategy profie of the stochastic game if for every payer k we have v k τ, θ = max d k D k θ Y k τ, d k, v k, θ. 9 In other words, if each payer s optima strategy is to not deviate from τ in a singe period, then τ is an equiibrium strategy profie. Using these notations, it is straightforward to verify the foowing resut, which we wi use to prove Lemma 3 to support the Fok Theorem resut. Lemma 1. Foowing any service provider seection rue π Π, it is an equiibrium for every payer i = πr, to aways refuse providing service. The corresponding equiibrium discounted average payoff for each payer is 0. In order to demonstrate the next resut, we define the aways trade strategy profie τ to be such that in each time period the service requester aways chooses to pay for service whenever she has positive scrips, and the seected type 0 payer aways chooses to provide service as ong as the service requester beongs to set Ω and refuses to provide service if the service requester does not beong to set Ω. We aso define unichain seection rues to be the set of service provider seection rues under which if payers foow the aways trade strategy profie, the resuting Markov chain on the state space S has a singe recurrent cass. It is easy to verify that both the minimum and random provider seection rues mentioned earier in the paper are exampes of unichain seection rues, aong with many others. Lemma 2. Foowing any unichain service provider seection rue π, if payers foow the aways trade strategy τ, the tota discounted payoff is positive for a payers at any state s for cose enough to 1. Proof: Since poicy π is unichain, the ong run average payoff is independent of the initia state r, Bertsekas In any time period, the chance that a random service requester has a positive number of scrips is at east 1/. As a resut, by foowing the aways trade strategy profie, the expected socia wefare gain per time period is at east u c/, which ower bounds the ong

13 12 Artice submitted to ; manuscript no. Pease, provide the manuscript number! run average socia wefare gain. Since the payers are indistinguishabe, the per payer ong run average payoff is ower bounded by u c/ 2 > 0. Foowing Proposition of Bertsekas 2007, the tota average discounted payoff v k τ, θ converges to the ong run average payoff as discount approaches 1, and therefore is aso positive. Q.E.D. Lemma 2 essentiay states that when the discount factor is cose enough to 1, the vaue function is positive at a states under any unichain service provider seection rue and the aways trade strategy. Foowing the idea behind the Fok Theorem, if a payer wants to refuse requesting or providing service in exchange for a scrip, and therefore deviate from the aways trade strategy, the entire group of payers can punish this payer by refusing to provide service in the future. This resuts in an inferior, zero, tota future utiity for the foca payer. This threat prevents a payer from deviating from the aways trade strategy. The foowing resut summarizes this idea. Lemma 3. Under any unichain service provider seection rue π, there exists a such that for any [, 1], the aways trade strategy profie τ is an equiibrium. This resut foows from the Fok Theorem for stochastic games Theorem 9, Dutta The compete proof in the Appendix verifies the conditions of Theorem 9 in Dutta 1995 based on Lemmas 1 and 2. Lemma 3 impies that as the discount factor is getting cose to 1, the centraized optima soution can be achieved in the stochastic game. This is by no means surprising, in ight of the Fok Theorem. The above sequence of emmas, however, motivates our next anaysis of the aways trade strategy when either the poicy π is not unichain or the discount factor is not sufficienty cose to Aways Trade Strategy In this section we show that under certain conditions the centra panner s optima socia wefare obtained in Section 2.1 can be achieved in the stochastic game. Motivated by Lemmas 1-3, we next focus on the case in which each payer foows the aways trade strategy.

14 Artice submitted to ; manuscript no. Pease, provide the manuscript number! 13 Without oss of generaity, denote V s to represent the tota discounted vaue function of payer 1 at state of the system s = r,, under service provider seection rue π and the aways trade strategy τ. Therefore, function V satisfies the foowing recursive equation, V = T V, 10 in which { / V r,, r T V r, 1 = 1 = 0 u + / Υ π r, 1 V r e i Υ π r,1 1 + e i,, r 1 > 0, 11 and / V r,, r = 0 T V r, = / Υ π r, i Υ V r e π r, + e i,, r > 0, 1 Υ π r, c + / V r e i Υ π r, + e i, / Υ π r,, r > 0, 1 Υ π r,. 12 Here the set Υ π r, represents the set of payers eigibe to be seected as the service provider, according to seection poicy π; we assume ties are broken randomy. For exampe, under the minimum scrip seection rue π, set Υ π r, 1 incudes a payers who hod the smaest number of scrips, excuding payer 1. If the cardinaity of the set Υ π r, > 1, each payer in the set has the same chance of being chosen to be the service provider. When the service provider seection rue is cear in the context, we remove the superscript in Υ π for simpicity. Using the recursive expression of vaue function V, we show the foowing resut. Proposition 2. For any given service provider seection rue π Π and mode parameters u, c, and R, there is a unique threshod 0, 1, such that V r, 0 for a r and if and ony if. Proof: Denote V to be the soution to recursive equations That is, V = T V. ow consider a sighty revised vaue iteration, { / T V r,, r V r, 1 = 1 = 0 u + / Υr, 1 V r e i Υr,1 1 + e i,, r 1 > 0,

15 14 Artice submitted to ; manuscript no. Pease, provide the manuscript number! / V r,, r = 0 T V r, = / Υr, i Υr, V r e + e i,, r > 0, 1 Υr, c + / V r e i Υr, + e i, / Υr,, r > 0, 1 Υr, Denote ˆV to be the soution to ˆV = T ˆV, which is aso the tota discounted vaue function of payer 1 with u and c as the benefit and cost of trade instead of u and c. Therefore, ˆV = V. ow consider a discount factor ˆ such that V ˆ 0, which impies ˆV ˆ 0. Consider any discount. factor such that > ˆ. We have T ˆV ˆ T ˆ ˆV ˆ = ˆV ˆ 0. Foowing the convergence of the vaue iteration agorithm and monotonicity of the operator T Coroary and Lemma in Bertsekas 2007, ˆV = im T t ˆV ˆ im T ˆ t ˆV ˆ = ˆV ˆ 0, t t which impies V = ˆV / 0. Q.E.D. ote that this resut is stronger than Lemma 2 because it hods for poicies π that are not unichain and shows a unique threshod. Parae to Proposition 2, we have the foowing intuitive resut. Proposition 3. For any given service provider seection rue π Π and mode parameters, and R, there is a unique threshod on u/c, such that V r, 0 for a r and if and ony if u/c is arger than this threshod. The proof is very simiar to the proof of Proposition 2 and thus is omitted here. Propositions 1, 2 and 3 impy the foowing main resut of this section, which is stronger than Lemma 3. Theorem 2. For any mode parameters u, c, and R, there is a unique threshod of the discount factor,, such that when >, the centraized optima socia wefare is achieved in equiibrium. That is, under the minimum scrip seection rue π, in equiibrium a payers foow the aways trade strategy. Simiary, for any given mode parameters, and R, there is a unique threshod on u/c, above which the centraized optima socia wefare is achieved in equiibrium by a payers foowing the aways trade strategy under the minimum scrip seection rue.

16 Artice submitted to ; manuscript no. Pease, provide the manuscript number! 15 The equiibrium resut is proved by appying the definition of the aways trade strategy to equiibrium condition umber of Scrips in the System In the previous section we demonstrated the optimaity of the minimum scrip seection rue in the stochastic game setting with a fixed number of scrips and when the discount factor is arge enough. In this section we investigate the appropriate number of scrips to ensure that aways trade is an equiibrium strategy under the minimum scrip seection rue. In particuar, we show that under fairy genera conditions, there is a unique threshod of the number of scrips in the system, beow which aways trade is an equiibrium. Furthermore, the threshod increases with the discount factor and the benefit of receiving service, u/c. First we present a condition under which no matter how many scrips are in the system, the vaue function for any state is non-negative, impying that aways trade is an equiibrium. Theorem 3. Under any service provider seection rue π Π, if u c, 13 no matter how many scrips are in the system, the vaue function V that soves the recursive equations is non-negative; that is, the aways trade strategy is an equiibrium. The proof is presented in the Appendix. The condition 13, rewritten as c u/, refects the trade-off between the cost of serving today versus the expected benefit of receiving service tomorrow. It is intuitive that if the cost of earning a scrip today is ess than the expected benefit of spending it in the next period, providing service never generates a negative net expected profit. Interestingy, the condition does not depend on the service provider seection rue. Since the condition is rather restrictive, we next anayze what happens under the minimum scrip seection rue when the condition c u/ is vioated. First, we present a technica characterization of recurrent states, which is somewhat interesting in its own right and usefu for proving our main resut.

17 16 Artice submitted to ; manuscript no. Pease, provide the manuscript number! Lemma 4. Consider the case when R, i.e., the number of scrips in the system is no ess than the number of payers. Under the minimum scrip seection rue and aways trade strategy, any state with more than one payer having 0 scrips is transient. The proof is based on induction on the number of payers with 0 scrips. The detaied proof is presented in the Appendix. Lemma 4 aows us to restrict attention to ony those states where no more than 1 payer has 0 scrips, which wi be usefu to prove Proposition 4, constituting the foundation of our main resut, Theorem 4. Anaogous to the symmetry condition C1 in the centra panner setting, Lemma 5 beow provides a symmetry argument needed for the proofs of Propositions 4 and 5. Lemma 5. Assume vaue function V satisfies recursive equations for a system with R scrips with the minimum scrip seection rue. For any nonnegative integer vectors r and r such that r = R and r = R with r k = r k for a k except 1, m 1, where r = r m and r m = r, V r, = The proof is presented in the Appendix. V r,. 14 Proposition 4. Assume vaue function V satisfies recursive equations for a system with R scrips with the minimum scrip seection rue, and vaue function V satisfies recursive equations for a system with the same parameters except with R + 1 scrips. Further, assume that u c [ ] For any nonnegative integer vector r in the recurrent cass such that r = R, and any payer index and k 1, we have V r + e k, V r,, 16 V r + e 1, V r, c, r : r 1 > 0, and 17 V r + e 1, V r, [ ] 1 c, r : r 1 = 0. 18

18 Artice submitted to ; manuscript no. Pease, provide the manuscript number! 17 Property 16 is a monotonicity property for vaue functions across different state spaces, and it states that if we inect one more scrip in the system, every payer, other than the one who receives the scrip, is worse off measured by the vaue function. The basic intuition behind it is that giving more scrips to others makes a payer more ikey to become the minimum scrip hoder, and therefore work sooner than otherwise. For the one who does receive the additiona scrip, whie it is intuitive that the person is better off, properties 17 and 18 show that the benefit is, in fact, upper bounded. The intuition is that even though the payer who receives the additiona scrip is better off by, at some point, spending it for service, such a trade is foowed by a state where the additiona scrip beongs to someone ese, therefore the payer wi be worse off afterwards, foowing 16. The basic ogic of the proof for Proposition 4 is that these properties are preserved through vaue iteration. The compete proof, however, needs to verify the properties under a possibe scenarios of scrip distribution among payers in both the R scrip system as we as the R + 1 scrip system. Furthermore, in order to estabish each one of properties 16-18, we need a the properties to hod to begin the vaue iteration, as we as condition 15. As a resut, the proof is rather invoved and is presented in the Appendix. Property 16 is the key property that we focus on. It impies that there is a threshod possiby infinity on the number of scrips, above which the vaue function at some state may become negative. When the vaue function does become negative, the threat of zero utiity does not work anymore, and the corresponding payer at this state is better off caiming bankruptcy by eaving the group. In order to prevent such an undesirabe outcome, the number of scrips issued in the system must be ower than this threshod. As discussed in Section 2.1, the socia wefare of the system increases with the number of scrips. Therefore, assuming payers foow the minimum scrip seection rue, the system designer wi choose the number of scrips in the system to be ust beow this threshod in order to achieve the greatest possibe socia wefare whie making it in each payer s best interest to not eave the group. The monotonicity property 16, however, hods ony under condition 15. Figure 2 depicts the condition. That is, in the area beow the soid curve, due to monotonicity there is a threshod on

19 18 Artice submitted to ; manuscript no. Pease, provide the manuscript number! / / [ / 1 ] 1 30 u/c B A C Figure 2 Conditions 13 and 15, with = the number of scrips possiby infinity, beow which aways trade is an equiibrium strategy. The dashed curve in Figure 2 corresponds to condition 13 in Theorem 3. In the area above the dashed curve, no matter how many scrips are in the system, aways trade is an equiibrium strategy. These curves partition Figure 2 into four ares. In area A, no matter how many scrips are in the system, aways trade is an equiibrium strategy. In area B, adding a scrip to the system decreases every payer s tota discounted vaue except that of the payer with the additiona scrip; however, we know that each payer s tota discounted vaue remains positive and thus no matter how many scrips are in the system, aways trade is an equiibrium. In area C, adding a scrip to the system decreases every payer s tota discounted vaue except that of the payer with the additiona scrip; in this case, there is a threshod possiby infinity on the number of scrips above which at east one of the payer s tota discounted vaue is negative. This eaves the shaded area depicted in the figure not covered by theoretica resuts. Later in Section 4.1, we conduct a numerica study on the shaded area, which indicates that athough the monotonicity property 16 does not hod, it is very ikey that there sti exists a unique upper bound on the number of scrips. Theorem 2 in the previous section states that for any given number of scrips R, there is a

20 Artice submitted to ; manuscript no. Pease, provide the manuscript number! 19 threshod on above which the vaue function V is aways positive. Proposition 4 further states that under condition 15, for a given discount, there is a threshod R beow which the vaue function V is aways positive. The combination of the two resuts impies that in the R space, the region that guarantees that vaue function V is aways positive is characterized by a threshod in R that is monotone in. The resut is formay stated in the foowing Theorem. The resut on u/c foows the exact same argument. Theorem 4. In a scrip system with payers and at east scrips, for any given set of mode parameters such that u c , or , 19 there is an upper bound R possiby infinity on the number of scrips, beow which aways trade is an equiibrium strategy under the minimum scrip seection rue, and the system optima socia wefare is achieved in the game. Furthermore, the upper threshod R increases with and u/c. Sufficient condition 19 is obtained by equating conditions 13 and 15. Iustrated in Figure 2, condition 19 covers the area to the eft and above of the intersection between the soid and dashed curves. As we wi demonstrate in numerica studies in Section 4.1, the monotone threshod structure presented in Theorem 4 ikey hods even without these conditions being met Shaded Area We do not have theoretica resuts when conditions 13 and 15 are both vioated, depicted by the shaded area in Figure 2. Therefore, we conducted numerica studies to check the structure of the vaue function in its minimum recurrent state. In particuar, we take a grid of vaues for u/c and in the shaded area when = 3, 3 and we see how the minimum vaue function s vaue over recurrent states changes with increasing R. We observe that in every case, the vaue function is unimoda and therefore monotonicay decreases as R increases to be arge enough. Figure 3 depicts one such exampe. The findings indicate that when condition 15 in Proposition 4 is vioated, a payer s vaue function does not aways decrease monotonicay with more scrips given to others. On the other

21 20 Artice submitted to ; manuscript no. Pease, provide the manuscript number! = 3, u/c = = 0.93 = 0.95 = minv R Figure 3 The non-monotone, unimoda structure of the vaue function in its minimum recurrent state. hand, in our numerica exampes, it aways first increases when the number of scrips R is sma, and then decreases. Therefore, as ong as the minimum vaue function over recurrent states is positive at the smaest scrip number R =, there sti is a unique upper bound possiby infinity on the number of scrips, beow which the vaue function is positive in a states. If so, the threshod on the number of scrips in the system increases with and u/c, even without the necessity of condition 19. The foowing resut indicates that when R = the vaue function is indeed positive in a recurrent states. Proposition 5. Assume R = and u c [ ] The vaue function V that soves the recursive equations under the minimum scrip seection rue is positive in a recurrent states. The proof is incuded in the Appendix.

22 Artice submitted to ; manuscript no. Pease, provide the manuscript number! Random provider seection rue It is important to note that there is a possibiity that a different service provider seection rue may permit a greater number of scrips than the minimum scrip seection rue. We have shown that for a given number of scrips in the system, the minimum scrip seection rue achieves the maximum socia wefare. It coud be possibe, however, that a different service provider seection rue outperforms the minimum scrip seection rue because it aows more scrips in the system without a payer defauting compared to the minimum scrip seection rue. Here we anayze the random provider seection rue, a common seection rue considered in previous iterature. More precisey, we consider a generaization of the minimum scrip seection rue that aso incudes the random provider seection rue. In particuar, consider the foowing service provider seection rue. First, K payers out of the 1 payers are randomy seected as potentia providers. Then the potentia provider who has the east number of scrips is seected as the service provider. Such a setting covers the possibiity that not every payer can provide service in each period, which, to a certain extent, reates to settings studied in Kash et a. 2007, Kash et a. 2012a, and Kash et a. 2012b. Obviousy, when K = 1, we have the random provider seection rue, whie the case K = 1 is the minimum scrip seection rue. The recursive equations are customized to V = T V, where T V r, = E κ [Ξ κ V r, ], 21 in which κ represents the set of K randomy seected potentia providers among the 1 payers who are not, and Ξ κ V foows { / V r,, r Ξ κ1 V r, 1 = 1 = 0 u + / Υ κ1 r, 1 V r e i Υ κ1 r,1 1 + e i,, r 1 > 0, 22 and / V r,, r = 0 Ξ κ V r, = / Υ κ r, i Υ V r e κ r, + e i,, r > 0, 1 Υ κ r, c + /. V r e i Υ κ r, + e i, / Υ κ r,, r > 0, 1 Υ κ r, 23

23 22 Artice submitted to ; manuscript no. Pease, provide the manuscript number! Here Υ κ r, represents the set of minimum scrip hoders among payers in the set κ. Simiar to Proposition 4 for the minimum scrip seection rue, the foowing resut hods for the genera service provider seection rue described above, incuding the random provider seection rue. Proposition 6. Assume vaue function V satisfies V = T V, where operator T is defined in equations 21-23, and vaue function V satisfies the same recursive equation in a system with the same parameters except with R + 1 scrips. Further, assume that u c For any nonnegative integer vector r such that r = R and any payer index and k 1, we have V r + e k, V r,, and 25 V r + e 1, V r, c. 26 The proof is simiar to that of Proposition 4 and is presented in the Appendix. For simiar reasons as discussed above for the minimum scrip seection rue, property 25 impies that there is a threshod possiby infinity on the number of scrips in the system, above which the vaue function at some state may become negative. umerica studies simiar to those described in Section 4.1 suggest that conditions 13 and 24 are sufficient, but not necessary, for the threshod structure to hod. Interestingy, therefore, both the minimum scrip seection rue and the random provider seection rue permit a threshod number of scrips in the system possiby infinity, above which the system crashes. Depending upon the system parameters u, c,, and, numerica resuts show that sometimes the minimum scrip seection rue permits at east as many scrips as the random provider seection rue; in this case, the minimum scrip seection rue is the preferabe service provider seection rue for the scrip system because it provides a greater socia wefare. For other parameters, though, the random provider seection rue permits more scrips than the minimum scrip seection

24 Artice submitted to ; manuscript no. Pease, provide the manuscript number! 23 rue, and in some cases the difference is enough to cause the random provider seection rue to outperform the minimum scrip seection rue in terms of socia wefare. Depending on the system parameters and appication, the system designer may choose to compare the performance of the minimum scrip seection rue and the random provider seection rue before creating the scrip system. Since both exhibit a threshod property on the number of permissabe scrips in the system, this shoud be reativey simpe to do. 5. Concusion In this paper we study design issues for managing a scrip system in a stochastic setting. In particuar, in each period one payer becomes a service requester and receives positive utiity if another payer is wiing to provide the service in exchange for a scrip. We first show that a centra panner woud aways prefer a trade of scrip for service to occur and woud seect the payer who has the east number of scrips to be the service provider. In a stochastic game setting with the absence of a centra panner, such a system optima soution can be achieved in equiibrium when the time discount factor is high enough or when the benefit of service is high enough compared with the cost to the service provider. When the time discount factor, or ratio between the benefit and cost of service, is not that high, we show that when using the minimum scrip seection rue or random provider seection rue there is an upper bound on the number of scrips that are aowed in the system, above which some payers may decide to defaut and exit the game when their scrip stock becomes ow. Furthermore, this upper bound increases with the time discount factor as we as the ratio between the benefit and cost of service. From a system design point of view, our resuts demonstrate that, assuming payers foow the minimum scrip seection rue, the number of scrips in the system shoud be at the upper bound, and a payers have the incentive to trade a scrip for service whenever the service requester has at east one scrip. We aso anayzed a commony used service provider seection rue, the random provider seection rue, and showed simiar threshod resuts as the minimum scrip seection rue. This makes it simpe for the system designer to compare the performance of the minimum scrip

25 24 Artice submitted to ; manuscript no. Pease, provide the manuscript number! seection rue and the random provider seection rue and choose the rue that resuts in the greatest socia wefare. One inherent assumption of our work is that each payer is abe to provide service when asked, and, furthermore, the detection and punishment of payers who do not contribute is possibe. In practice for arge scrip systems, this coud be difficut, especiay if the system reies on the service requester to report whether trade occurred, which coud open the possibiity of maicious payers getting others kicked out of the system. Previous work by Kash et a. 2007, Kash et a. 2012a and Kash et a. 2012b has not required the abiity of each payer to provide service nor the detection and punishment of payers who do not contribute. With a few additiona assumptions and mode differences as described in Section 1.1, they show the existence of an equiibrium in which payers foow a threshod poicy under the random provider seection rue. One usefu extension of our work woud be to possiby combine these two streams of work by using the minimum scrip rue without the severe punishment that the Fok Theorem equiibrium reies upon. There are a number of other possibe extensions to our paper that deserve further exporation. The most notabe one is that preferences are not homogeneous among payers. For exampe, some payers may vaue the service more than others, or providing service may cost some payers more than others. Simiary, the benefit and cost of service may change over time or be stochastic. We suspect that the reason why our system does not experience a market crash when there are too few scrips, as observed in some appications, is a resut of our current assumptions on the homogeneity of utiities across payers and over time. Future work wi hopefuy provide us with more insights on this type of market crash. Another extension that is worth studying is that the price of service may not be fixed at one scrip, but is instead determined according to the scrip distribution among payers. Endnotes 1. In the stochastic game, generay speaking, the service provider seection rue may aow seection of another payer in case a first seected payer reects to serve. Such a generaized seection rue

26 Artice submitted to ; manuscript no. Pease, provide the manuscript number! 25 π maps the distribution of scrips to a sequence of provider seections, contingent upon acceptance. A equiibrium resuts hod with this generaization. 2. When the poicy π is not unichain, Lemma 2 does not hod. Therefore Condition 1 of Dutta s theorem may be vioated. Furthermore, Dutta s Fok Theorem for stochastic games requires the discount factor to be sufficienty cose to Since the state space grows exponentiay with, which poses significant computationa chaenges, we did not check for higher vaues of. Acknowedgments We thank Ian Kash, Saed Aizamir, and He Wang for very vauabe comments and suggestions, which ed to improvements in the paper. We aso thank the referees, whose comments heped us significanty improve the presentation and some of our resuts. This research was partiay supported by SF Contract CMMI and Masdar Institute of Science and Technoogy MIST. References Aperis, C., R. Johari A peer-to-peer system as an exchange economy. Proceeding from the 2006 Workshop on Game Theory for Communications and etworks. ACM. AuYoung, A., B.. Chun, C. g, D. Parkes, A. Vahdat, A. C. Snoeren Practica Market-Based Resource Aocation. Technica report, UC San Diego, CS Beenkiy, M., M. Chase, C. C. Erway, J. Jannotti, A. Küpçü, A. Lysyanskaya, E. Rachin Making P2P accountabe without osing privacy. Proceedings of the 2007 ACM Workshop on Privacy in the Eectronic Society, WPES 2007, Aexandria, VA, USA, October 29, 2007, Bertsekas, D Dynamic Programming and Optima Contro, vo. II. Athena Scientific. Brunee, J., P. Hurst, J. Huth, L. Kang, C. G, D. C. Parkes, M. Setzer, J. Shank, S. Youssef Egg: An extensibe and economics-inspired open grid computing patform. Third Workshop on Grid Economics and Business Modes GECO, Chun, B.., P. Buonadonna, A. AuYoung, C. g, D. C. Parkes, J. Shneidman, A. C. Snoeren, A. Vahdat Mirage: A microeconomic resource aocation system for sensornet testbeds. Second IEEE Workshop on Embedded etworked Sensors,