Trading cookies with a random walk

Transcription

1 Graduate Theses and Dissertations Iowa State University Capstones, Theses and Dissertations 2017 Trading cookies with a random walk Yiyi Sun Iowa State University Follow this and additional works at: Part of the Applied Mathematics Commons Recommended Citation Sun, Yiyi, "Trading cookies with a random walk" (2017). Graduate Theses and Dissertations This Thesis is rought to you for free and open access y the Iowa State University Capstones, Theses and Dissertations at Iowa State University Digital Repository. It has een accepted for inclusion in Graduate Theses and Dissertations y an authorized administrator of Iowa State University Digital Repository. For more information, please contact digirep@iastate.edu.

2 Trading cookies with a random walk y Yiyi Sun A thesis sumitted to the graduate faculty in partial fulfillment of the requirements for the degree of MASTER OF SCIENCE Major: Applied Mathematics Program of Study Committee: Alexander Roitershtein, Major Professor Arka P. Ghosh Songting Luo The student author and the program of study committee are solely responsile for the content of this thesis. The Graduate College will ensure this thesis is gloally accessile and will not permit alterations after a degree is conferred. Iowa State University Ames, Iowa 2017 Copyright c Yiyi Sun, All rights reserved.

3 ii DEDICATION I would like to dedicate this thesis to my parents Mr. Zhiqiang Sun and Mrs. Hongyu Luo without whose support I would not have een ale to complete this work. I would also like to thank my friends and family for their loving guidance and financial assistance during the writing of this work.

4 iii TABLE OF CONTENTS LIST OF FIGURES iv ACKNOWLEDGEMENTS ABSTRACT v vi CHAPTER 1. INTRODUCTION General ackground and previous work Motivation and goals Overview of the model Game description CHAPTER 2. OPTIMAL PRICES c 1 AND c 2 FOR A GIVEN STORE LO- CATIONS CHAPTER 3. OPTIMAL STORE LOCATIONS n 1 AND n CHAPTER 4. CONCLUSION BIBLIOGRAPHY

5 iv LIST OF FIGURES Figure 2.1 The relationship etween c 1 and c

6 v ACKNOWLEDGEMENTS I would like to take this opportunity to express my thanks to those who helped me with various aspects of conducting research and the writing of this thesis. First and foremost, Dr. Alexander Roitershtein for his guidance, patience and support throughout this research and the writing of this thesis. His insights and words of encouragement have often inspired me and renewed my hopes for completing my graduate education. I would also like to thank my committee memers for their efforts and contriutions to this work: Dr. Arka P. Ghosh and Dr. Songting Luo.

7 vi ABSTRACT The mathematical prolem of determining a gamler s risk of ruin involves analyzing decisions of only one agent, namely the gamler. In this work we consider an extension that introduces two additional players, so called sellers. These two new agents can oost the proaility of success for the gamler y selling to him (using a jargon orrowed from the theory of excited random walks) a cookie which is used to increase the proaility of moving forward in the next step. The generalized gamler s ruin scenario considers an excited random walk on a finite interval of integer line with two cookie store locations and unlimited supply of cookies at each. Each time when the uyer (walker) visits a store location, he has an opportunity to decide whether he is willing to consume the cookie or not. We wish to determine the equilirium prices and cookie store locations in a formal game associated with this generalized gamler s ruin scenario.

8 1 CHAPTER 1. INTRODUCTION 1.1 General ackground and previous work Excited random walks (ERW) or random walks in a cookie environment on Z d is a modification of the nearest neighor simple random walk such that in several first visits to each site of the integer lattice, the walk s jump kernel gives a preference to a certain direction and assigns equal proailities to the remaining (2d 1) directions. If the current location of the random walk has een already visited more than a certain numer of times, then the walk moves to one of its nearest neighors with equal proailities. The model was introduced y Benjamini and Wilson in [3] and extended y Zerner in [13]. In the cookies jargon, upon first several visits to every site of the lattice, the walker consumes a cookie providing them a oost toward a distinguished direction in the next step. Many important aspects of the asymptotic ehavior of excited random walks on Z are y now well-understood [11]. An application of the theory of excited random walks to the physics of DNA molecular motors is discussed in [1, 4]. This work continues to investigate a class of models introduced in [10]. In [10] several variations of a two-person (Stackelerg) game etween a uyer and a seller, whose major component is a random walk of the uyer on a finite interval of integers were considered. The key element of the game is a gamler s ruin prolem [6, 7], where in contrast to the classical version, the walker (uyer) has the option of consuming cookies, which when used, increase the proaility of moving in the desired direction for the next step. The cookies are supplied to the uyer y the second player (seller). The ultimate goal is to determine an equilirium price policy for the seller and the equilirium cookie store location. The optimization prolem which the seller faces is somewhat similar to that of a monopoly whose market is a spatially non-homogeneous Hotelling each with demand curve varying randomly across the population

9 2 [8, 9]. The original Hotelling each (linear city) model was introduced to illustrate Hotelling s law in economics, namely a general paradigm that in many markets it is rational for producers to make their products as similar as possile. 1.2 Motivation and goals The game can serve as a simplified model to explore the relationship etween economic agents in a risky environment, for instance a firm in an innovative and competitive segment of a hi-tech industry and an experienced consulting company. The firm (uyer) seeks to reduce uncertainty and increase the expected profit y investing in the consulting service at a ottleneck point of its production line, while the consultant (seller) wants to optimize the configuration and the price of its service package. From the proaility theory point of view, the models introduced in [10] and in this work attempt 1. To measure the gain of the walker from exploiting a reinforcing mechanism represented y cookies. It is natural to study this type of prolems using a gamler s ruin scenario and within a game-theoretic framework, where exact features of the reinforcing mechanism are determined through the interaction etween the walker and sellers. This is in contrast to the usual excited random walk, where the walker, as a price-taker in a large market, has no effect on determining the parameters of the cookie environment. 2. To further contriute to the asic understanding of one-dimensional excited random walks y considering a suitale variation of gamler s ruin prolem. It is well-known, see for instance [11], that the asymptotic ehavior of a random walk can e inferred from the solution to the corresponding gamler s ruin prolem. In particular, the asymptotic ehavior of excited random walk is largely governed y a single parameter, its average local drift. Curiously, the main result of this work suggests that the same parameter solely determines the optimal cookie prices for a fixed store location (see Chapter 2 elow). 3. The optimization methods employed in this work are, up to a certain point, methods of continuous convex optimization. One could expect that, similarly to the Hotelling linear

10 3 city model, the equilirium configuration will place the cookie stores at the same location. However, it is easy to see that this solution is not availale when the cookie s prices can vary and are determined y the sellers. Thus the actual equilirium design is expected to e affected y a not-so-intrinsic to the prolem discrete optimization. A part of our initial motivation was to see whether the discrete design can force the risk-neutral uyer to ecome effectively risk-adverse. Remarkaly, in some particular sense the answer to this question turns out to e affirmative (see the discussion of the main result in Chapter 3). We are planning to consider in the future an extension of this work to a continuous time model ased on the excited stochastic process considered in [12]. 1.3 Overview of the model This thesis introduces a generalization of the model of [10] to a three-person game with two competing sellers and a uyer. The model is significantly more involved from the technical point of view and computationally extensive ecause of the required rather detailed analysis of the underlying finite-state Markov chain. More specifically, in this work we introduce the following modification of the classical two-person gamler s ruin scenario, where the uyer has the option to consume cookies supplied y the sellers at two different cookie locations. The cookie serves to instantly increase the proaility of moving forward in the next step. Set the starting point of uyer as n N located etween 0 and N, 2, and treat the direction from 0 to as the forward direction. Assume that the uyer performs a nearest-neighor random walk on the integer line with asortion at 0 and. If the uyer reaches the point efore 0 he is rewarded with R dollars, otherwise he receives a zero payoff. Simultaneously and independently each of other, two sellers set up the cookie stores at integer sites n 1 and n 2 within the interval (0, ). The two sellers sell the cookies at fixed prices c 1 and c 2, respectively. At a regular site, the uyer moves one step forward with a fixed proaility p (0, 1), and ackward with a fixed proaility q 1 p. If he consumes a cookie at the store locations, then he moves one step forward with a larger proaility p + ɛ 1 (p, 1) from n 1 and p + ɛ 2 (p, 1) from n 2. The uyer can choose either accept the cookie for the suggested price or reject it in order to reach the ultimate purpose, which is maximizing the total revenue at the asortion

11 4 time. The goal of this work is to determine an equilirium price for each cookie and location for the cookie store. In this work, we are focus on the a special situation, similar to the one-seller counterpart which is referred to in [10] as a asic game, that is we assume p q 1 2. We use the Markov property of the underlying excited random walk and a sugame perfect Nash equilirium to solve the relationship etween the price and location. In Section 2 we treat the location of cookie stores as a fixed variale to find out the equilirium price, while the contrary situation will e considered in the Section 3 where the equilirium store locations are determined. We assume that the uyer is risk-neutral and maximizes its expected game payoff. The proof of the main result is concluded in the last section. 1.4 Game description In this and next sections, we discuss the asic game. We consider the following scenario with a fixed proaility p q 1 2. First fix any N, 2, and set the forward direction as from 0 to. Then let the uyer starting the random walk at a fixed point n N located etween 0 and. On the other hand, the two sellers, who are seeking for the maximum expected revenue, need to make a decision for the store s location n 1 and n 2 and the price of each cookie c 1 and c 2 independently. In this scenario, there is no product cost for oth of the two sellers and the numer of cookies η that the sellers provided to the uyer can e infinite many. The uyer can accept the cookie as a instant proaility oost strategy, however, he has an option to refuse if he consider it is not worthwhile. Denote ɛ 1 (0, 1 6 ) and ɛ 2 (0, 1 6 ) as the cookie strategy for the two sellers separately. If the uyer decides to accept the cookie at the first/second cookie store (located at n 1 /n 2 ) he moves forward on Z according to P n1 p + ɛ 1 /P n2 p + ɛ 2,otherwise his motion is ased on P p. Notice that, the uyer can only moves step y step. Once the uyer arrives any of the sides, the game is end. If the uyer reaches the point first he is rewarded with R dollars, in contrast, he gets nothing. Consider the money he paid for cookies as the cost, the uyer seeks to maximize his expected earnings. The main purpose of this section is to calculate the explicit result for the value of optimal price for each cookie.

12 5 Definition 2.1 Game Γ n Γ n is a three-person game ased on the Stackelerg model (the first two players take action independently, the third player oserves their action and then decides his own moves). All of the players in the game need to consider a strategy that maximizes their corresponding expected payoff given the chosen strategy pf two other players.. The first two players are the sellers, and the third player is the uyer. The two sellers move first and inform their action to the uyer separately. Then the uyer determines his game plan and starts a random walk. Let S : [0, ) {1,..., 1} e the set of strategies of the two sellers. Each pair (c 1, n 1 ) S specifies the cookie s price c 1 > 0 and the store s location n 1 {1,..., 1} determined y the first seller. Similarly, each pair (c 2, n 2 ) S specifies the cookie s price c 2 > 0 and the store s location n 2 {1,..., 1} determined y the second seller. Notice that, we default n 1 n 2 in the whole set. Let B : S 2 {e n1, e n2 } where e k {0, 1} e the strategy of the uyer. The uyer can choose to reject the cookie or consume it at the two different stores with the certain price. Definition 2.2 The uyer s random walk Let X k (0, ) denote uyer s location on the integer line at time k Z +. Let M k Z + e the numer of cookies availale at the walk s current location at time k Z +. Since the uyer can only move one step at a time, the Markov chain transition kernel of uyer s random walk at the cookie store n 1 is given y P n1 (X k+1 i + 1, m k+1 m 1 X k i, M k m) p + 1 in1,m>0 ɛ P n1 (X k+1 i 1, m k+1 m 1 X k i, M k m) q 1 in1,m>0 ɛ

13 6 Similarly, the Markov chain transition kernel of uyer s random walk at the cookie store n 2 is given y P n2 (X k+1 i + 1, m k+1 m 1 X k i, M k m) p + 1 in2,m>0 ɛ P n2 (X k+1 i 1, m k+1 m 1 X k i, M k m) q 1 in2,m>0 ɛ where 1 A is the indicator.

14 7 CHAPTER 2. OPTIMAL PRICES c 1 AND c 2 FOR A GIVEN STORE LOCATIONS The goal of this section is to determine optimal prices. If the price of each cookie is attractive enough, the uyer will choose to consume it. Indeed, if the cookies are free, the uyer will definitely want them, and the claim follows y the continuity. Therefore, the uyer would consume the cookies at oth of the two sellers when the price is optimal (does the optimal exist?). For n I, we denote y R n the expected reward of the uyer who starts their random walk at the location n I. By the strong Markov property, R n1 c 1 + (q ε 1 )αr n1 + (p + ε 1 )βr n1 + (p + ε 1 )(1 β)r n2, where α : P n1 1(T n1 < T 0 ) is the proaility that the uyer returns to n 1 from the ackward direction and β : P n1 +1(T n1 < T n2 ) is the proaility that the uyer returns to n 1 from the forward direction. Similarly, R n2 c 2 + (q ε 2 )γr n2 + (q ε 2 )(1 γ)r n1 + (p + ε)δr n2 + (p + ε)(1 δ)r, where γ : P n2 1(T n2 < T 0 ) is the proaility that the uyer returns to n 2 from the ackward direction and δ : P n2 +1(T n2 < T ) is the proaility that the uyer returns to n 2 from the forward direction. The solution to the classical gamler s ruin prolem [6] yields: α n 1 1, β n 2 n 1 1 n 1 n 2 n 1 γ n 2 n 1 1, δ n 2 1 n 2 n 1 n 2

15 8 Therefore, R n1 (p + ɛ 1)(1 β)r n2 c 1 (2.1) 1 (q ɛ 1 )α (p + ɛ 1 )β [(q ɛ 1 )(p + ɛ 2 )(n 2 n 1 ) + (p + ɛ 1 )(p + ɛ 2 )n 1 )] R n2 R (q ɛ 1 )(q ɛ 2 )( n 2 ) + (q ɛ 1 )(p + ɛ 2 )(n 2 n 1 ) + (p + ɛ 1 )(p + ɛ 2 )n 1 (q ɛ 2 )( n 2 )n 1 c 1 (q ɛ 1 )(q ɛ 2 )( n 2 ) + (q ɛ 1 )(p + ɛ 2 )(n 2 n 1 ) + (p + ɛ 1 )(p + ɛ 2 )n 1 [(q ɛ 1 )(n 2 n 1 )( n 2 ) + (p + ɛ 1 )( n 2 )n 1 ] c 2 (q ɛ 1 )(q ɛ 2 )( n 2 ) + (q ɛ 1 )(p + ɛ 2 )(n 2 n 1 ) + (p + ɛ 1 )(p + ɛ 2 )n 1 Definition 2.3 Sugame perfect Nash equilirium[8] A sugame perfect Nash equilirium means that the strategy serves est for each player and it satisfied that every player is playing in a Nash equilirium in every sugame. Since oth of the two sellers seek for the maximum profit simultaneously, the optimal c 1 and c 2 have to satisfied the following conditions. Without loss of generality, when the first seller fix the price of each cookie at c 1, the second seller would not change the price c 2 for a igger enefit. Therefore,if we fixed n 1 and n 2, then the revenue R n1 and R n2 should e larger if the uyer consumes at oth of the two cookie stores. This condition is equivalent to imposing the following set of four inequalities. R n1 (ɛ 1, c 1, ɛ 2, c 2 ) R n1 (0, 0, ɛ 2, c 2 ) (2.2) R n1 (ɛ 1, c 1, ɛ 2, c 2 ) R n1 (ɛ 1, c 1, 0, 0) (2.3) R n2 (ɛ 1, c 1, ɛ 2, c 2 ) R n2 (0, 0, ɛ 2, c 2 ) (2.4) R n2 (ɛ 1, c 1, ɛ 2, c 2 ) R n2 (ɛ 1, c 1, 0, 0) (2.5) For convenient, we first rewrite R n2 (ɛ 1, c 1, ɛ 2, c 2 ) and R n2 (0, 0, ɛ 2, c 2 ) in (2.4) as R n2 (ɛ 1, c 1, ɛ 2, c 2 ) X 1R X 2 c 1 X 3 c 2 K 1 R n2 (0, 0, ɛ 2, c 2 ) X1 1 R X1 3 c 2 K 1 1

16 9 with X 1 (q ɛ 1 )(p + ɛ 2 )(n 2 n 1 ) + (p + ɛ 1 )(p + ɛ 2 )n 1 (2.6) X 2 (q ɛ 2 )( n 2 )n 1 (2.7) X 3 (q ɛ 1 )(n 2 n 1 )( n 2 ) + (p + ɛ 1 )( n 2 )n 1 (2.8) X1 1 q(p + ɛ 2 )(n 2 n 1 ) + p(p + ɛ 2 )n 1 (2.9) X3 1 q(n 2 n 1 )( n 2 ) + p( n 2 )n 1 (2.10) K 1 (q ɛ 1 )(q ɛ 2 )( n 2 ) + (q ɛ 1 )(p + ɛ 2 )(n 2 n 1 ) (2.11) + (p + ɛ 1 )(p + ɛ 2 )n 1 (2.12) > 0 (2.13) K1 1 q(q ɛ 2 )( n 2 ) + q(p + ɛ 2 )(n 2 n 1 ) + p(p + ɛ 2 )n 1 (2.14) > 0 (2.15) Then (2.4) can e write as (X 1 R X 2 c 1 X 3 c 2 )K 1 1 (X 1 1R X 1 3c 2 )K 1 0 (X 1 K 1 1 X 1 1K 1 )R X 2 K 1 1c 1 (X 3 K 1 1 X 1 3K 1 )c 2 0 where X 1 K 1 1 X 1 1K 1 (q ɛ 2 )( n 2 )n 1 (p + ɛ 2 )ɛ 1 X 2 K 1 1 (q ɛ 2 )( n 2 )n 1 [q(q ɛ 2 )( n 2 ) + q(p + ɛ 2 )(n 2 n 1 ) + p(p + ɛ 2 )n 1 ] X 3 K 1 1 X 1 3K 1 (q ɛ 2 )( n 2 )n 1 ( n 2 )ɛ 1 By algeraic simplification, we get following result: (p + ɛ 2 )ɛ 1 R [q(q ɛ 2 )( n 2 ) + q(p + ɛ 2 )(n 2 n 1 ) + p(p + ɛ 2 )n 1 ]c 1 ( n 2 )ɛ 1 c 2 We then use the method to deal with (2.5), let R n2 (ɛ 1, c 1, ɛ 2, c 2 ) X 1R X 2 c 1 X 3 c 2 K 1 R n2 (ɛ 1, c 1, 0, 0) X2 1 R X2 2 c 1 K 2 1

17 10 with X1 2 p(q ɛ 1 )(n 2 n 1 ) + p(p + ɛ 1 )n 1 (2.16) X2 2 q( n 2 )n 1 (2.17) K1 2 q(q ɛ 1 )( n 2 ) + p(q ɛ 1 )(n 2 n 1 ) + p(p + ɛ 1 )n 1 (2.18) > 0 (2.19) and X 1, X 2, X 3, K 1 has een defined in (2.6),(2.7),(2.8)and(2.11). Hence,(2.5) is (X 1 R X 2 c 1 X 3 c 2 )K1 2 (X1R 2 X2c 2 1 )K 1 0 (X 1 K1 2 X1K 2 1 )R (X 2 K1 2 X1K 2 1 )c 1 X 3 K1c where X 1 K 2 1 X 2 1K 1 [(q ɛ 1 )(n 2 n 1 + (p + ɛ 1 )n 1 ]( n 2 ) (q ɛ 1 )ɛ 2 X 2 K1 2 X1K 2 1 [(q ɛ 1 )(n 2 n 1 ) + (p + ɛ 1 )n 1 ]( n 2 ) n 1 ɛ 2 X 3 K1 2 [(q ɛ 1 )(n 2 n 1 ) + (p + ɛ 1 )n 1 ]( n 2 ) [q(q ɛ 1 )( n 2 ) + p(q ɛ 1 )(n 2 n 1 ) + p(p + ɛ 1 )n 1 ] Therefore, (q ɛ 1 )ɛ 2 R + n 1 ɛ 2 c 1 [q(q ɛ 1 )( n 2 ) + p(q ɛ 1 )(n 2 n 1 ) + p(p + ɛ 1 )n 1 ]c 2 After solving the last two inequalities (2.4) and (2.5), we need figure out their the relationship with the first two (2.2) and (2.3). Plug the value of R n1 defined in (2.1) into (2.2), we have (p + ɛ 1 )(1 β)r n2 (ɛ 1, c 1, ɛ 2, c 2 ) c 1 1 (q ɛ 1 )α (p + ɛ 1 )β (p + ɛ 1)n 1 R n2 (ɛ 1, c 1, ɛ 2, c 2 ) (n 2 n 1 )n 1 c 1 (q ɛ 1 )(n 2 n 1 ) + (p + ɛ 1 )n 1 R n2 (ɛ 1, c 1, ɛ 2, c 2 ) R n2 (0, 0, ɛ 2, c 2 ) Similarly, for (2.3), p(1 β)r n 2 (0, 0, ɛ 2, c 2 ) 1 qα pβ pn 1 q(n 2 n 1 ) + pn 1 R n2 (0, 0, ɛ 2, c 2 ) (p + ɛ 1 )(1 β)r n2 (ɛ 1, c 1, ɛ 2, c 2 ) c 1 1 (q ɛ 1 )α (p + ɛ 1 )β R n2 (ɛ 1, c 1, ɛ 2, c 2 ) R n2 (ɛ 1, c 1, 0, 0) (p + ɛ 1)(1 β)r n2 (ɛ 1, c 1, 0, 0) c 1 1 (q ɛ 1 )α (p + ɛ 1 )β

18 11 Therefore, the inspection of (2.1) shows that in fact the first two inequalities imply the last two in the aove system. By using the closed form expressions for R 1 and R 2 the system is reduced to the following set of four inequalities. l 1 :c 1 0 with l 2 :c 2 0 l 3 :A 1 R + B 1 c 1 D 1 c 2 l 4 :A 2 R + B 2 c 1 D 2 c 2 A 1 (p + ɛ 2 )ɛ 1 (2.20) A 2 (q ɛ 1 )ɛ 2 (2.21) B 1 [q(q ɛ 2 )( n 2 ) + q(p + ɛ 2 )(n 2 n 1 ) + p(p + ɛ 2 )n 1 ] (2.22) B 2 n 1 ɛ 2 (2.23) D 1 ( n 2 )ɛ 1 (2.24) D 2 q(q ɛ 1 )( n 2 ) + p(q ɛ 1 )(n 2 n 1 ) + p(p + ɛ 1 )n 1 (2.25) The set of solutions in the (c 1, c 2 )-plane is non-empty and ounded y two (in general, olique) straight straight and two axes. The intersection point of two slanting lines is N(c N1, c N2 ), where c N1 A 2D 1 A 1 D 2 B 1 D 2 B 2 D 1 R and c N2 A 2B 1 A 1 B 2 B 1 D 2 B 2 D 1 R Since we are focusing on the p q 1 2, then we can compute out the value of c N1 and c N2. c N1 1 2 ( 1 2 ɛ 2)( 1 2 ɛ 1)ɛ 1 + ( 1 2 ɛ 1)ɛ 1 ɛ 2 n 2 + ( ɛ 2)ɛ 2 1 n 1 ( 1 2 ( 1 2 ɛ 2) + ɛ 2 n 2 )( 1 2 ( 1 2 ɛ 1) + ɛ 1 n 1 ) + ɛ 1 ɛ 2 ( n 2 )n 1 R (2.26) 2ɛ 1R (2.27) 1 2 c N2 2 ɛ 1)( 1 2 ɛ 2)ɛ 2 + ( 1 2 ɛ 1)ɛ 2 2 n 2 + ( ɛ 2)ɛ 1 ɛ 2 n 1 ( 1 2 ( 1 2 ɛ 2) + ɛ 2 n 2 )( 1 2 ( 1 2 ɛ R 1) + ɛ 1 n 1 ) + ɛ 1 ɛ 2 ( n 2 )n 1 (2.28) 2ɛ 2R (2.29) For a graphical illustration we refer to Fig. 2 elow. Note that for any fixed c 2 < c N2 there is a constant c 1 < c l1 such that the value of R n1 at c 1 is igger then at c l1 (see Fig. 2). Similarly,

19 12 for an aritrary c 1 one can find c 2 < c N2 such that the value of R n2 at c 2 is igger then at c 2. Hence, neither of two sellers would enefit form changing the price unilaterally if and only if c 1 c N1 and c 2 c N2. Figure 2.1 The relationship etween c 1 and c 2. As illustrated in the figure, the plot can e classified into four areas y the straight lines l 3 and l 4. Furthermore, the shadows D in the plot represents the price that the uyer would accept. Otherwise, the uyer would reject the cookies at least one of the two stores. Let point M e the projection of point N on c 1 axis. If the pair of price (c 1, c 2 ) lies on l 3, then the first seller can always increase c 1 and push the price point into area D. And once (c 1, c 2 ) appears in D, the second seller will definitely choose to rise his price, which leads two results. If the price point is on the right of NM, then the action would force the point to l 4 line. Otherwise, the two sellers would reach the agreement of price on point N. On the other hand, if the price point is on l 4, increasing the price is reasonale only for the second seller. Finally, turns out that the only price point satisfied Nash equilirium for the two sellers is point N (c 1, c 2 ). Since R n1 can e treated as a function of R n2, then the only thing left in this system is that we need to check if R n2 (ɛ 1, c 1, ɛ 2, c 2) R n2 (0, 0, 0, 0)

20 13 Proof Since c 1 and c 2 satisfied the inequality (2.5), then we know that R n2 (ɛ 1, c 1, ɛ 2, c 2) R n2 (ɛ 1, c 1, 0, 0) The only thing left that need to e proved is R n2 (ɛ 1, c 1, 0, 0) R n2 (0, 0, 0, 0) Plug the value of c 1 into (2.1), we get R n2 (ɛ 1, c 1, 0, 0) ( 1 2 ɛ 1) + 2ɛ 1 n 1 n 2 R ( 1 2 ɛ 1) 2ɛ 1 n 1 4ɛ 1 n 1 (1 + ( 1 2 ɛ ) n 2R 1) 2ɛ 1 n 1 (1 + 4ɛ 1n 1 ( 1 2 3ɛ 1) )n 2R n 2R R n2 (0, 0, 0, 0) Therefore, R n2 (ɛ 1, c 1, ɛ 2, c 2 ) R n 2 (0, 0, 0, 0).

21 14 CHAPTER 3. OPTIMAL STORE LOCATIONS n 1 AND n 2 In this section we continue to investigate the game-theoretic framework introduced somewhere efore. The main purpose of this section is to explicitly identify the optimal location for each cookie store. In the last section we have figured out the optimal prices c 1 and c 2 as a function of the two store locations. This will e used here to determine the optimal store location n 1 and n 2 for the two sellers according to the optimal price. We denote y W n1 the revenue of the seller (located at n 1 ) if the uyer starts at n 1 (that is, n n 1 ), and y W n2 the revenue of the seller (located at n 2 ) if the uyer starts at n 2 (that is, n n 2 ). Then we can write, W n1 η n 1 n 1 c 1 and W n2 η n 2 n 2 c 2, where η x y stands for the numer of visits of the uyer to the store located at x when he starts at y. If the uyer starts at n 1, then the strong Markov property implies Thus we get η n 1 n (q ɛ 1 ) n 1 1 η n 1 n n 1 + (p + ɛ 1 ) n 2 n 1 1 η n 1 1 n 1 n 2 n 1 + (p + ɛ 1 ) η n 2 n 1 n 2 n 1 1 η n 1 2 n 1 (q ɛ 2 ) η n 1 n n 2 n 1 + (p + ɛ 2 ) n 2 1 η n 2 n 1 n 1 + (q ɛ 2 ) n 2 n 1 1 η n 2 n 2 n 2 n 1 1 We then sustitute η n 2 n 1 And thus, η n 1 n 1 η n 2 n 1 (1 (q ɛ 1) n 1 1 n 1 into this equation: (p + ɛ 1 ) n 2 n 1 1 n 2 n 1 ) η n 1 n 1 1 p+ɛ 1 n 2 n 1 (1 (q ɛ 2 ) n 2 n 1 1 (p + ɛ 2 ) n 2 1 ) η n 2 n n 2 n 1 n 1 q ɛ 2 η n 1 n 2 n 2 n 1 1 (p + ɛ 2 )(n 2 n 1 )n 1 + (q ɛ 2 )( n 2 )n 1 (q ɛ 1 )(p + ɛ 2 )(n 2 n 1 ) + (q ɛ 1 )(q ɛ 2 )( n 2 ) + (p + ɛ 1 )(p + ɛ 2 )n 1

22 15 Similarly, if the uyer starts at n 2, Then, we have η n 2 n (q ɛ 2 ) n 2 n 1 1 η n 2 n n 2 n 2 + (p + ɛ 2 ) n 2 1 η n 1 2 n 1 n 2 + (q ɛ 2 ) η n 1 n 2 n 2 n 2 1 η n 1 n 2 (q ɛ 1 ) n 1 1 η n 1 n n 2 + (p + ɛ 1 ) n 2 n 1 1 η n 1 1 n 1 n 2 n 2 + (p + ɛ 1 ) η n 2 n 1 n 2 n 2 1 Sustituted into the equation: Thus, η n 2 n 2 η n 1 n 2 (1 (q ɛ 2) n 2 n 1 1 n 2 n 1 (p + ɛ 2 ) n 2 1 n 2 ) η n 2 n 2 1 q ɛ 2 n 2 n 1 (1 (q ɛ 1 ) n 1 1 (p + ɛ 1 ) n 2 n 1 1 ) η n 1 n n 1 n 2 n 2 p + ɛ 1 η n 2 n 1 n 2 n 2 1 (p + ɛ 1 )( n 2 )n 1 + (q ɛ 1 )(n 2 n 1 )( n 2 ) (q ɛ 1 )(p + ɛ 2 )(n 2 n 1 ) + (q ɛ 1 )(q ɛ 2 )( n 2 ) + (p + ɛ 1 )(p + ɛ 2 )n 1 Considering this prolem in the reality situation, we have the following cases: Case 1: When the uyer starts at n, where n 1 n < n 2 When the uyer starts at n n 1, we first define W 1 n 1 as the actual enefit of the first seller(located at n 1 ) and W 1 n 2 as the actual enefit of the second seller(located at n 2 ). Then we can write: W 1 n 1 W n1 η n 1 n 1 c 1 W 1 n 2 η n 2 n 1 c 2 Hence, Wn 1 ( ɛ 2)(n 2 n 1 )n 1 + ( 1 2 ɛ 2)( n 2 )n 1 ( 1 2 ɛ 1)( ɛ 2)(n 2 n 1 ) + ( 1 2 ɛ 1)( 1 2 ɛ 2)( n 2 ) + ( ɛ 1)( ɛ 2)n 1 2ɛ 1R Wn 1 ( ɛ 2)( n 2 )n 1 ( 1 2 ɛ 1)( ɛ 2)(n 2 n 1 ) + ( 1 2 ɛ 1)( 1 2 ɛ 2)( n 2 ) + ( ɛ 1)( ɛ 2)n 1 2ɛ 2R

23 16 Case 2: When the uyer starts at n, where n 1 < n 2 n When the uyer starts at n n 2, we first define W 2 n 1 as the actual enefit of the first seller(located at n 1 ) and W 2 n 2 as the actual enefit of the second seller(located at n 2 ). Therefore we can write: W 2 n 1 W n1 η n 1 n 2 c 1 W 2 n 2 η n 2 n 2 c 2 We can get the value of η n 1 n 2 and η n 2 n 2 directly from the aove analysis. Therefore, Wn 2 ( ɛ 1)( n 2 )n 1 ( 1 2 ɛ 1)( ɛ 2)(n 2 n 1 ) + ( 1 2 ɛ 1)( 1 2 ɛ 2)( n 2 ) + ( ɛ 1)( ɛ 2)n 1 2ɛ 1R Wn 2 ( ɛ 1)( n 2 )n 1 + ( 1 2 ɛ 1)( n 2 )(n 2 n 1 ) ( 1 2 ɛ 1)( ɛ 2)(n 2 n 1 ) + ( 1 2 ɛ 1)( 1 2 ɛ 2)( n 2 ) + ( ɛ 1)( ɛ 2)n 1 2ɛ 2R Case 3: When the uyer starts at n, where n 1 < n < n 2 When the uyer starts at n, we first define W 3 n 1 as the actual enefit of the first seller(located at n 1 ) and W 3 n 2 as the actual enefit of the second seller(located at n 2 ). Therefore we can write: W 3 n 1 α n W n1 + (1 α n )βw n1 where α n is the proaility that the uyer starts at n and reaches to n 1 efore n 2, and β is the proaility that that the uyer starts from n 2 and gets to n 1 efore. W 3 n 2 (1 α n )W n2 + α n (1 γ)w n2 where γ is the proaility that that the uyer starts from n 1 and gets to 0 efore n 2. Based

24 17 on the property of Markov chain, we know that: Therefore, α n n 2 n n 2 n 1 β (p + ɛ 2 ) n β + (q ɛ 2 ) + (q ɛ 2 ) n 2 n 1 1 β n 2 n 2 n 1 n 2 n 1 (q ɛ 2 )( n 2 ) β (q ɛ 2 )( n 2 ) + (p + ɛ 2 )(n 2 n 1 ) γ (q ɛ 1 ) 1 + (q ɛ 1 ) n 1 1 γ + (p + ɛ 1 ) n 2 n 1 1 γ n 1 n 1 n 2 n 1 (p + ɛ 1 )n 1 1 γ (q ɛ 1 )(n 2 n 1 ) + (p + ɛ 1 )n 1 Wn 3 ( ɛ 2)( n 2 )n 1 + ( ɛ 2)(n 2 n)n 1 ( 1 2 ɛ 1)( ɛ 2)(n 2 n 1 ) + ( 1 2 ɛ 1)( 1 2 ɛ 2)( n 2 ) + ( ɛ 1)( ɛ 2)n 1 2ɛ 1R Wn 3 ( ɛ 1)(n n 1 )( n 2 ) + ( ɛ 1)( n 2 )n 1 ( 1 2 ɛ 1)( ɛ 2)(n 2 n 1 ) + ( 1 2 ɛ 1)( 1 2 ɛ 2)( n 2 ) + ( ɛ 1)( ɛ 2)n 1 2ɛ 2R In order to figure out the relationship etween W 3 n 1 and n 1, we assume n 2 is fixed, then Wn 3 ( ɛ 2)( n 2 )n 1 + ( ɛ 2)(n 2 n)n 1 ( 1 2 ɛ 1)( ɛ 2)(n 2 n 1 ) + ( 1 2 ɛ 1)( 1 2 ɛ 2)( n 2 ) + ( ɛ 1)( ɛ 2)n 1 2ɛ 1R ( 1 2 ɛ 2)( n 2 ) + ( ɛ 2)(n 2 n) 2ɛ 1 R ( 1 2 ɛ 1)( 1 2 +ɛ 2)n 2 +( 1 2 ɛ 1)( 1 2 ɛ 2)( n 2 ) n 1 + ( ɛ 1)( ɛ 2) Therefore, W 3 n 1 is monotonic increasing as n 1 < n increased. The trend of W 3 n 1 is logically force the n 1 goes to n as close as possile, which force the case 3 into the case 2. Similarly, if we fixed n 1 and focus on W 3 n 2 and n 2, then Wn 3 ( ɛ 1)(n n 1 )( n 2 ) + ( ɛ 1)( n 2 )n 1 ( 1 2 ɛ 1)( ɛ 2)(n 2 n 1 ) + ( 1 2 ɛ 1)( 1 2 ɛ 2)( n 2 ) + ( ɛ 1)( ɛ 2)n 1 2ɛ 2R ( 1 2 ɛ 1)(n n 1 ) + ( ɛ 1)n 1 2ɛ 2 R ( 1 2 ɛ 1)( 1 2 +ɛ 2)(n 2 n 1 )+( 1 2 +ɛ 1)( 1 2 +ɛ 2)n 1 n 2 + ( 1 2 ɛ 1)( 1 2 ɛ 2)

25 18 Then for the part we can rewrite it as Z ( 1 2 ɛ 1)( ɛ 2)(n 2 n 1 ) + ( ɛ 1)( ɛ 2)n 1 n 2 Z ( 1 2 ɛ 1)( ɛ 2)n 2 + 2ɛ 1 ( ɛ 2)n 1 n 2 ( 1 2 ɛ 1)( ɛ 2) + ( 1 2 ɛ 1)( ɛ 2) + 2ɛ 1 ( ɛ 2)n 1 n 2 + It it oviously that Z is decreasing as n 2 decreased, so that W 3 n 2 is monotone increasing as n 2 decreased. Then to make W 3 n 2 reaches a higher enefit, we want to push n 2 to n as close as possile, which actually turns case 3 into the case 1. Case 4: When the uyer starts at n, where n 1 < n 2 < n When the uyer starts at n, we first define W 4 n 1 as the actual enefit of the first seller(located at n 1 ) and W 4 n 2 as the actual enefit of the second seller(located at n 2 ). Therefore we can write: W 4 n 1 α n βw n1 where α n is the proaility that the uyer starts at n and reaches to n 2 efore, and β is the proaility that that the uyer starts from n 2 and gets to n 1 efore. W 4 n 2 α n W n2 Here, we have Thus, α n n n 2 and β (q ɛ 2 )( n 2 ) (q ɛ 2 )( n 2 ) + (p + ɛ 2 )(n 2 n 1 ) Wn 4 ( ɛ 2)( n)n 1 ( 1 2 ɛ 1)( ɛ 2)(n 2 n 1 ) + ( 1 2 ɛ 1)( 1 2 ɛ 2)( n 2 ) + ( ɛ 1)( ɛ 2)n 1 2ɛ 1R Wn 4 ( ɛ 1)(n 2 n 1 )( n) + ( ɛ 1)( n)n 1 ( 1 2 ɛ 1)( ɛ 2)(n 2 n 1 ) + ( 1 2 ɛ 1)( 1 2 ɛ 2)( n 2 ) + ( ɛ 1)( ɛ 2)n 1 2ɛ 2R

26 19 Since n 1 < n 2 < n, we can focus on the relationship etween W 4 n 2 and n 2. Fixed n 1, we have, Wn 4 ( ɛ 1)(n 2 n 1 )( n) + ( ɛ 1)( n)n 1 ( 1 2 ɛ 1)( ɛ 2)(n 2 n 1 ) + ( 1 2 ɛ 1)( 1 2 ɛ 2)( n 2 ) + ( ɛ 1)( ɛ 2)n 1 2ɛ 2R ( 1 2 ɛ 1)n 2 + 2ɛ 1 n 1 2ɛ 2 ( n)r 2( 1 2 ɛ 1)ɛ 2 n 2 + ( 1 2 ɛ 1)( 1 2 ɛ 2) + 2ɛ 1 ( ɛ 2)n 1 ( 1 Z ) 2ɛ2 ( n)r + 2ɛ 2 2( 1 2 ɛ 1)ɛ 2 n 2 + ( 1 2 ɛ 1)( 1 2 ɛ 2) + 2ɛ 1 ( ɛ 2)n 1 where Z 1 2ɛ 2 ( ( 1 2 ɛ 1)( 1 2 ɛ 2) + 2ɛ 1 ( ɛ 2)n 1 ) + 2ɛ 1 n 1 < 0 This indicates that W 4 n 2 is monotone increasing as n 2 increased. In order to maximize the earning of the second seller, we will require n 2 goes to n, which turns case 4 into case 2 Case 5: When the uyer starts at n, where n < n 1 < n 2 When the uyer starts at n, we first define W 5 n 1 as the actual enefit of the first seller(located at n 1 ) and W 5 n 2 as the actual enefit of the second seller(located at n 2 ). Therefore we can write: W 5 n 1 α n W n1 where α n is the proaility that the uyer starts at n and reaches to n 1 efore 0. W 5 n 2 α n (1 γ)w n2 where γ is the proaility that that the uyer starts from n 1 and gets to 0 efore n 2. As what we discussed efore, α n n (p + ɛ 1 )n 1 and 1 γ n 1 (q ɛ 1 )(n 2 n 1 ) + (p + ɛ 1 )n 1

27 20 Thus, Wn 5 ( ɛ 2)( n 2 )n + ( ɛ 2)(n 2 n 1 )n ( 1 2 ɛ 1)( ɛ 2)(n 2 n 1 ) + ( 1 2 ɛ 1)( 1 2 ɛ 2)( n 2 ) + ( ɛ 1)( ɛ 2)n 1 2ɛ 1R Wn 5 ( ɛ 1)( n 2 )n ( 1 2 ɛ 1)( ɛ 2)(n 2 n 1 ) + ( 1 2 ɛ 1)( 1 2 ɛ 2)( n 2 ) + ( ɛ 1)( ɛ 2)n 1 2ɛ 2R Since n < n 1 < n 2, the only thing left is the relationship etween W 5 n 1 n 2, and n 1. We then fixed where Wn 5 ( ɛ 2)n 1 + ( 1 2 ɛ 2) + 2ɛ 2 n 2 2ɛ 1 nr 2( 1 2 ɛ 1)ɛ 2 n 2 + ( 1 2 ɛ 1)( 1 2 ɛ 2) + 2ɛ 1 ( ɛ 2)n 1 ( 1 Z ) 2ɛ1 nr + 2ɛ 1 2( 1 2 ɛ 1)ɛ 2 n 2 + ( 1 2 ɛ 1)( 1 2 ɛ 2) + 2ɛ 1 ( ɛ 2)n 1 Z ( 1 2ɛ 1 )(2( 1 2 ɛ 1)ɛ 2 n 2 + ( 1 2 ɛ 1)( 1 2 ɛ 2)) + ( 1 2 ɛ 2) + 2ɛ 2 n 2 > 0 Therefore, W 5 n 1 is monotone increasing as n 1 decreased. In order to maximize the earning of the first seller, we will require n 1 goes to n., which turns case 5 into case 1 In conclude, case 1 and case 2 are the est choice for the aove 5 different cases. And we still need to consider a following special case. Special Case: When the uyer starts at n, where n 1 n 2 n Under this situation, oth of the two stores coincide with the starting point,n 1 n 2 n. Therefore the numer of visits to the two stores should e same, write as η n n. Meanwhile, define Wn 6 1 as the actual enefit of the first seller(located at n 1 ) and Wn 6 2 as the actual enefit of the second seller(located at n 2 ). ηn n 1 + (q ɛ ) n 1 n ηn n + (p + ɛ ) n 1 n W 6 n ηn nc 1 ηn n where W 6 n ηn nc 2 ɛ 1 2 (ɛ 1 + ɛ 2 )

28 21 then, we get η n n 2( n)n + (ɛ 1 + ɛ 2 )(2n ) Therefore, Wn 6 ( n)n 2ɛ 1 R 1 + (ɛ 1 + ɛ 2 )(2n ) Wn 6 ( n)n 2ɛ 2 R 2 + (ɛ 1 + ɛ 2 )(2n ) In order to find out the est location for the two stores ased on the Nash equilirium, we need to compare their payoff under case 1 (n 1 n < n 2 ), case 2 (n 1 < n 2 n) and special case (n 1 n n 2 ). As we have mentioned efore, a Nash equilirium prolem is to make oth of the two seller want to stay at their location. Under case 1,we want to know if the second seller is willing to stay. From the aove analysis, we know that W 1 n 2 is increasing as n 2 decreased, which indicates that n 2 should get as close as possile to n. Since the location are integers, then the maximum W 1 n 2 happens when n 2 n+1. Hence we want to compare the actual payoff of the second seller when n 1 n, n 2 n + 1 and n 1 n 2 n. By sustitution and calculation, one can easily show that W 6 n 2 > W 1 n 2 (n 2 n+1), which indicates that the second seller would like to move from n 2 n + 1 to n 2 n. This result force the case 1 to e the special case. However, this special case is actually not a Nash equilirium result. Because if the two seller share the same store location, then any one of them can slightly reduce their price to gain all the trading opportunity with the uyer. Under case 2, we want to know if the first seller is still want to keep the original location. As we have discussed efore, W 2 n 1 increased as n 1 increased, which means that the maximum Wn 2 1 happens when n 1 n 1. Then compare the actual payoff of the first seller when n 1 n 1, n 2 n and n 1 n 2 n. Through calculation, one can show that Wn 2 1 (n 1 n 1) > Wn 6 1, which indicates that the first seller would like to stay at n 1 n 1. As for the second seller, if he move forward to n 2 n + 1, then the profit of the first seller would decrease, and so that the first guy would move forward to n 1 n, which is actually stay in the same case. If the second seller want to move ackward to n 2 n 1, then this situation changes into special case, which is not a Nash prolem. Hence, the second seller would also stay at the original

29 22 setting. From what we discussed aove, we can conclude that the est store location for oth of the two sellers are actually n 1 n 1 and n 2 n, where n is the starting point of the uyer.

30 23 CHAPTER 4. CONCLUSION We discussed a 2-dimensional modification gamler s ruin scenario, which has some common performances as the excited regular nearest-neighor random walk on Z,except eing localized to the two certain point. In the whole Markov chain, the states 0 and are recurrent states (with the transition proaility equal to 1) and other states are transient states(with the transition proaility smaller than 1). General speaking, the deformation of transition kernel at two different point can e descried as two sellers that providing an instantaneously increased proaility(smaller than 1 6 ) in the forward direction when the uyer visits the stores. In this game, the goal of the uyer is to maximize his expected earning which can e expressed in terms of a difference etween the revenue and the cost. The cost of the uyer is determined y the price of a cookie, which has een negotiated etween the uyer and the sellers. Through this paper, we discussed a special situation, a fair moving proaility for the uyer when he face the parts without modification. Based on the analysis, the equilirium price and the stores location are two independent variales. Since the starting point of the uyer can e anywhere etween 0 and, the sellers need to choose the stores location to maximize their expected enefit. For conclusion, we include all the reasonale relationship etween the starting point and the stores location, which turns out that the nash equilirium store location for the two sellers are n 1 n 1 and n 2 n, where n is the starting point of the uyer. However, since the result is just estalished on the certain case that p q 1 2 and ɛ 1, ɛ 2 < 1 6, this assumption may not e true for all the situations. In fact, we do analysis the different situations, and some results indicates that the equilirium price is somehow depending on the store s location. The further results need an even deeper analysis.

31 24 BIBLIOGRAPHY [1] T. Antal and P. L. Krapivsky, Molecular spiders with memory, Phys. Review E 76 (2007), [2] T Antal and S. Redner, Excited random walk in one dimension, J. Phys. A 38 (2005), [3] I. Benjamini and D. B. Wilson, Excited random walk, Elect. Comm. Proa. 8 (2003), [4] M. Buchanan, Attack of the cyerspider, Nature Physics , p [5] B. Davis, Brownian motion and random walk pertured at extrema, Proa. Theory Related Fields 113 (1999), [6] R. Durrett, Proaility: Theory and Examples, 2nd edn., Duxury Press, Belmont, CA, [7] M. A. El-Shehawey, On the gamler s ruin prolem for a finite Markov chain, Stat. & Proa. Letters 79 (2009), [8] R. Gions, Game Theory for Applied Economists, Princeton University Press, [9] H. Hotelling, Staility in competition, Econ. J. 39 (1929), [10] K. Jungjaturapit, T. Pluta, R. Rastegar, A. Roitershtein, M. Tema, C. N. Vidden, B. Wu, Trading cookies in a gamler s ruin scenario, Involve 6 (2013), [11] E. Kosygina and M. Zerner, Excited random walks: results, methods, open prolems, Bulletin of the Institute of Mathematics. Academia Sinica (New Series) 8 (2013),

32 25 [12] O. Raimond and B. Schapira, Excited Brownian Motions, ALEA, Lat. Am. J. Proa. Math. Stat. 8 (2011), [13] M. P. W. Zerner, Multi-excited random walks on integers, Proa. Theory Related Fields 133 (2005),