Admissioncontrolwithbatcharrivals

Transcription

1 Admissioncontrolwithbatcharrivals E. Lerzan Örmeci Department of Industrial Engineering Koç University Sarıyer İstanbul-Turkey Apostolos Burnetas Department of Operations Weatherhead School of Management Case Western Reserve University Euclid Avenue Cleveland OH 44106, USA Abstract We consider the problem of dynamic admission control in a multi-class Markovian loss system receiving random batches, where each admitted class-i job demands an exponential service with rate µ, and brings a reward r i. We show that the optimal admission policy is a sequential threshold policy with monotone thresholds. Keywords: Dynamic admission control, threshold policies, loss systems The corresponding author where her address is lormeci@ku.edu.tr. 1

2 1 Introduction We consider a system, which consists of c identical parallel servers with no waiting room and K classes of jobs. Arrivals occur according to a Poisson process with rate λ. At each arrival epoch, a random number of jobs from each class arrive at the system. We denote an arriving batch by j =(j 1,...,j K ), where j i is the number of class-i jobs. The system receives a batch j =(j 1,...,j K ) with probability p j. In other words, the probability that j i class-i jobs with i =1,...,K arrive at an arrival epoch is p j. Whenever a class-i job is admitted to the system, it brings a reward of r i > 0 upon its arrival, and requires an exponential service time with rate µ. Hence, jobs are differentiated only by their rewards. We are interested in dynamic admission policies that maximize the total expected discounted reward with a continuous discount rate β over an infinite horizon as well as the long-run average net profit. The system may employ batch acceptance, in which it can either accept or reject the entire batch, or partial acceptance, where some of the jobs in a batch can be admitted and the remaining ones rejected. For the case of batch acceptance, we present a numerical example which violates any expected monotonicity properties of an optimal policy. Apart from this discussion, the paper concentrates on systems following a partial acceptance policy, for which an optimal policy can be completely characterized: If the job classes are ordered such that r K <...<r 1, then it is always optimal to accept class-1 jobs. Moreover, there exists an optimal threshold policy that is based on a sequence of monotone thresholds. Capacity control in revenue management addresses optimal allocation of a fixed amount of resources to different demand segments. The underlying assumption is the perishability of the resources at a certain time. The effect of revenue management on the performance of car rental firmsarediscussedin[3] and[4]. However, for many rental businesses, in particular for car rental, perishability assumption is not appropriate, since the resources are rented for a random amount of time, after which they are available again for future customers. Savin, Cohen, Gans and Katalan [13] are the first to formulate such systems as multiple-server loss models with uncertain customer arrivals and uncertain rental durations. We consider another uncertain component in rental businesses: Geraghty and Johnson [4] indicate that the major car rental companies depend largely on corporate customers, which brings the issue of arrivals demanding more than one resource at a time. Hence, we use batch arrivals to model the demand coming from corporate firms. Admission control problems in stochastic knapsacks have been studied by many authors, see Chapter 4 of [12] for a comprehensive review. A stochastic knapsack is defined as a system which consists of c identical parallel servers, no waiting room and K job classes. Each class is distinguished by its size b i, its arrival rate and its mean service time. If a class-i job is accepted to the system, it seizes b i servers, and occupies all of them till the end of its service time, upon which it releases all b i servers at the same time. This is substantially different from the system with batch arrivals: In a system with batch arrivals, each admitted job of an arriving batch behaves independently, i.e., each job occupies and releases one server according to its own service time, regardless of all other admitted jobs. Section 4 presents numerical examples to point out the differences of the three systems, namely stochastic knapsacks, and systems employing batch or partial acceptance. 2

3 There have been earlier studies which investigate the structural properties of optimal admission policies for certain stochastic knapsacks: A stochastic knapsack with K different job classes, each of which has the same mean service time, µ, and a unit size, i.e., b i = 1, is studied in [9]: The class with the highest reward is identified as the preferred class, and the existence of optimal thresholds is established for all other classes. In fact, we extend these results to systems with batch arrivals, since our system would be identical to this knapsack if the arrivals were single. Lippman and Ross [8] analyze the optimal admission rule for a system with one server and no waiting room which receives offers from jobs according to a joint service time and reward probability distribution (this model is usually referred to as the streetwalker s dilemma). They also consider this system when it receives batch arrivals. Several authors, among others, study the structural properties of optimal dynamic admission policies in Markovian stochastic knapsacks with two classes of jobs demanding different service rates: Altman, Jiménez and Koole [2] show that these policies are of threshold type, whereas Örmeci, Burnetas and van der Wal [11] analyze the issue of preferred classes and establish the monotonicity of thresholds under certain conditions, while assuming b i = 1. Recently, there is a new research direction in control of loss systems, which aims to find the structure of optimal policies when the state of the system is not completely observable: Optimal routing of customers in a system with c non-identical servers and no waiting room is considered in [6] and [1]. This paper is organized as follows: In the next section, we present the Markov decision process (MDP) model of the system described above. The third section gives a complete description of an optimal policy. The fourth section compares the structure of optimal policies for stochastic knapsacks, systems employing batch or partial acceptance through numerical examples. Finally, we conclude and point out possible extensions of this work in the last section. 2 Markov decision model In this section, we build a discrete-time Markov decision process (MDP) for systems employing a partial acceptance policy with the objective of maximizing total expected discounted returns over a finite time horizon with β as the discount rate. Let the state of the system be x, the number of jobs in the system. We interpret discounting as exponential failures, i.e., the system closes down in an exponentially distributed time with rate β (for the equivalence of the process with discounting and the process without discounting but with an exponential deadline, see e.g., [14]). Arrivals occur according to a Poisson process with rate λ. Then, the maximum possible rate out of any state is λ + cµ + β. Since the maximum rate of transitions is finite, we can use uniformization (introduced in [7]) and normalization to build a discrete time equivalent of the original system. Specifically, we assume that the time between two successive transitions is exponentially distributed with rate λ + cµ + β, and using the appropriate time scale, that λ + cµ + β =1. Ateachtransitionepochthereiseitheranarrivalofabatchofjobswith probability λ, a service completion with probability xµ, afictitious service completion with probability (c x)µ due to uniformization, or a transition to the terminal state with probability β due to discounting. Furthermore, we will refer to the instantaneous states at the arrival epochs as (x, j) = (x; j 1,...,j K ) to indicate that a batch of j has arrived to find x jobs in the system. Note 3

4 that admission and rejection decisions are assumed to be made upon an arrival so that these states are observed only at arrival epochs. Immediately after those epochs the system moves to another state according to the decision made. Now, let a n i (x; j) be the optimal number of class-i jobs admitted to the system in state (x; j) when there are n more observation points in future. Finally, we let u n (x) be the maximal expected β-discounted net benefit ofthesystem whichstartsinstatex when n observation points remain in the horizon. Then the optimality equations are given by: K K K v n (x; j) = max u n (x + a i )+ a i r i :0 a i j i ; x + a i c u n+1 (x) = λ j p j v n (x; j)+xµu n (x 1) + (c x)µu n (x), where v n (x; j) can be interpreted as the maximal expected β-discounted reward for a system starting in state x with a batch of j seeking admittance, i.e., for a system starting in the instantaneous state (x; j) whenn observation points remain in the horizon. Moreover, we set u n ( 1) = u n (0). If the nth event occurred is an arrival of a batch j =(j 1,...,j K ), which happens with probability λp j,thena n i (x; j) ofj i class-i jobs are accepted so that the system moves to the state x + K a n i (x; j) with a total reward of K a n i (x; j)r i. If a job finishes its service, with probability xµ, the system state changes to x 1. The fictitious service completions, which occur with probability (c x)µ, affect neither the state nor the total reward of the system. Finally, the system closes down with probability β, receiving no more reward. All our results are shown under the objective of maximizing total expected β-discounted reward for a finite number of transitions, n, (including the fictitious transitions due to uniformization). Since the state and action spaces are finite sets and all rewards are bounded, standard arguments of Markov decision theory can be used to extend the results to the infinite horizon discounted reward as well as the expected average reward per unit time. Specifically, let v(x; j) andu(x) denote the value functions for the infinite horizon expected discounted reward. Thus, for β > 0: v(x; j) = lim n vn (x; j), u(x) = lim n un (x), and {a i (x; j)} K is the optimal action in state (x; j). For β =0,u(x), and instead the relative value functions and the gain can be considered. 3 Structure of the optimal policy In determining the structure of an optimal policy, the effect of an additional job is an important issue: If the arrivals were single, it is easy to see that a class-i job would be accepted only if u n (x) <u n (x +1)+r i. Now, we can interpret the quantity u n (x) u n (x + 1) as the expected burden of an additional job in state x, so that whenever the reward of a job is higher than its expected burden, it is admitted to the system. Our first result derives bounds on the expected burden of an additional job: 4

5 Lemma 1 For all n 1 and for all x<c, 0 u n (x) u n (x+1) <r 1, whenever the inequalities are true for n =0. Proof. The initial value function u 0 (x) = 0 for all x c satisfies both inequalities. We use induction to prove the second inequality in detail, whereas the proof of the first inequality is omitted since it is very similar to the following. Assume that u n (x) also satisfies the second inequality. Now consider period n +1. Let system A be in state x and system B be in state x + 1. We couple all jobs in both systems except for the additional job in system B, so that if a job leaves system A, the coupled job in system B also leaves. We let system A follow the optimal policy, and system B move to the same state with system A whenever system A accepts at least one job, i.e., K a n i (x; j) > 0, losing a maximum reward of r 1, otherwise, i.e., if system A rejects all the incoming jobs, system B also rejects them. Whenever the additional job in system B departs with probability µ, the two systems couple with no difference in reward, and in all other service completions including the fictitious service completions, the difference between the two systems due to an additional job does not change. Then: u n+1 (x) u n+1 (x +1) u n+1 (x) u n+1 B (x +1) λ p j max {r 1,u n (y) u n (y +1)} y<c j +(c 1)µ max y<c {un (y) u n (y +1)} < r 1, by the definition of the policy followed by system B and by the induction hypothesis and uniformization. 2 The first inequality in Lemma 1 shows that it is always preferable to be in a state where there are fewer jobs: Recall that u n (x) is equal to the expected discounted total reward of the system under an optimal policy when there are n more transitions. Since rewards are collected in the beginning of service, jobs that are already in the system do not contibute to u n (x), i.e., u n (x) isaffected only by the future rewards. Therefore, the jobs initially in the system bring only more burden by blocking acceptance of future jobs. The second inequality, on the other hand, proves that the maximum expected burden that a job can bring cannot exceed the largest reward, r 1. Therefore, this lemma immediately establishes the following result: Corollary 1 Class 1 is preferred. With the batch arrivals, we can admit a number of jobs at the same time. Then, the meaning of the expected burden of an additional job is not clear as the reference state x is not fixed. However,theburdenofallclassesofjobsinacertainstatex, u n (x) u n (x+1), are the same due to equal service rates, which allows us to decide on the admission of jobs sequentially: Whenever both class-i and class-k jobs are available with i>k,i.e.,r i <r k, class-k jobs are considered for admission before class-i jobs. Hence, there exists an index κ in each state (x; j) defined by: κ =max{k : a n k(x; j) > 0}, (1) 5

6 such that for all k<κ, a n k (x; j) =j k, a n i (x; j) = 0 for all i>κ, and1 an κ(x; j) j κ.wewill use this observation in our subsequent results. Intuitively, we expect that it should be less profitable to accept jobs when there are many jobs already in the system, or equivalently the benefit of additional jobs should decrease in the number of jobs already in the system. In the system we consider, this expectation is equivalent to the concavity of the value functions, which is our next result: Lemma 2 For all x c 2, we have: whenever the inequality is true for n =0. u n (x) 2u n (x +1)+u n (x +2) 0 n 1, (2) Proof. The claim is true for n =0withu 0 (x) = 0 for all x c 2. Assume that it is also true for n. Now we rewrite inequality (2) for n and use Lemma 1: 0 u n (x) u n (x +1) u n (x +1) u n (x +2). (3) The second inequality implies that if it is optimal to reject a class-i jobinstatex, thenitis optimal to reject him for all y x. Moreover, we can iterate the second inequality in (3) to have: u n (x) u n (x +1) u n (x + a) u n (x +1+a), which can be rewritten as follows: Now we can iterate again to obtain: u n (x) u n (x + a) u n (x +1) u n (x +1+a). u n (x) u n (x + a) u n (x + b) u n (x + a + b), (4) which is valid for all a 0andforallb 0. Now we show that the functions v n s also satisfy inequality (2): Let systems A, B, C and D correspond to systems in states (x; j), (x+1;j), (x+1;j) and(x+2;j) inperiodn, respectively. We set x and x as the optimal states to move into for system A and system D, respectively, so that x = x + k a i and x = x + k a i,wherea i = a n i (x; j) anda i = an i (x +2;j). Note that k and k are the largest indices of classes which have jobs that are accepted in system A and D, respectively. We also let: δ n = v n (x; j) 2v n (x +1;j)+v n (x +2;j). First, we note that it is enough to consider the cases when x x :Ifx >x,bothx and x are reachable from both states (x; j) and(x +2;j). Then either there is a contradiction due to the optimality of x and x or u n (x )=u n (x )sothatwecanchooseoneofthetwostates as an optimal state to move into for both systems. Now we differentiate the cases: Case I: x = x 6

7 In this case, system D rejects two of the jobs that system A accepts, so k k. Let κ and κ betheclassesofthesetwojobssothat k a i r i + r κ + r κ = k a i r i with k κ κ k. Now we let system B and C admit all jobs system A admits except for class-κ and class-κ jobs, respectively, so that both systems move to state x : δ n u n (x )+ u n (x ) k a i r i u n (x ) k a i r i + r κ k a i r i + r κ + u n (x )+ where the first inequality is due to the optimality of v n s. Case II: x <x k a i r i r κ r κ =0, Now, we let system B imitate the decisions of system A so that it accepts the same jobs that system A accepts moving to state x + 1, whereas system C accepts the same jobs that system D accepts moving to state x 1. Notice that both states x +1andx 1arefeasiblesince x <x. Hence: δ n u n (x )+ k a i r i u n (x +1) k k a i r i u n (x 1) a ir i + u n (x )+ a ir i 0, where the first inequality is due to the description of the policies followed by systems A, B, C and D, and the second follows by inequality (4) with x = x, a =1andb = x x 1 0. Now we can consider u n+1 s: We couple each of the additional jobs in system B and C with one of the additional jobs in system D, i.e., if the additional job in system B or C departs, which happens with probability µ, then one of the additional jobs in system D also departs. Hence: u n+1 (x) 2u n+1 (x +1)+u n+1 (x +2) = λ p j (v n (x; j) 2v n (x +1;j)+v n (x +2;j)) j +xµ (u n (x 1) 2u n (x)+u n (x +1)) +µ (u n (x) u n (x) u n (x +1)+u n (x +1)) +µ (u n (x) u n (x +1) u n (x)+u n (x +1)) +(c x 2)µ (u n (x) 2u n (x +1)+u n (x +2)) 0, where the first term is non-positive since we have proved the inequality for v n s, the second and last terms are non-positive by the induction hypothesis, and all the rest is 0. Thus, u n (x) s are concave in x. 2 k 7

8 The results of [5] can also be used to conclude Lemma 1 and 2. However, the corresponding proofs are more technical: Koole [5] proves certain monotonicity properties of several operators in queueing systems. The system under consideration can be represented by a rather complicated combination of three operators, one for admission control, one for multiserver queues, and another for modelling no-waiting-room systems. The complexity stems from having batch arrivals including different kinds of jobs in a loss system. Lemma 1 and 2 together describe an optimal threshold policy with monotone thresholds: Theorem 1 An optimal policy can be described as follows: For all n 1: (i) a n 1 (x; j) =min{j 1,c x}, and (ii) for all k 2, there exist numbers lk n such that: k 1 + a n k(x; j) =min j k, lk n x j i, where (b) + =max{0,b} for any real number b. Moreoverl n k+1 ln k. Proof. (i) The statement follows directly from Corollary 1. Note that a n 1 (x; j) j 1 and a n 1 (x; j) c x for an 1 (x; j) to be feasible. (ii) Wefirst define the numbers l n k as follows: l n k =min{y <c: u n (y) u n (y +1)+r k }, where lk n = c if there is no such y. Thus, ln k is the minimum number of jobs in the system for which it is better to reject a class-k job. From (1), we know that if an optimal policy is considering to accept class-k jobs, it should have accepted all jobs of classes 1,..,k 1 in the arriving batch j, i.e., a n i (x; j) =j i for i =1,..,k 1. Then for an action a n k (x; j) to be feasible, we need an k j (x; j) min k, c x k 1 + j i. If lk n = c for class k, then it is optimal to accept as many jobs as possible of class k, sothat a n k j (x; j) =min k, c x k 1 + j i, agreeing with part (ii) of the theorem. Note that even if lk n = c, there may not be room for class-k jobs since the jobs of smaller indices from the same batch may have filled out all the empty servers. This is taken care of by sequential decision making. If lk n <c, it is better to reject class-k jobs in every state y with y ln k due to Lemma 2: If lk n x k 1 j i j k, i.e., if the corresponding threshold value is not reached, then it is still optimal to accept as many class-k jobs as possible, i.e., a n k (x; j) =j k so that all class-k jobs in the batch are accepted. However, when lk n x k 1 j i <j k, it is optimal to reject some or all of these jobs not to exceed the threshold lk n for class k, hence an k (x; j) =max 0,lk n x k 1 j i. In both cases, a n k j (x; j) =min k, lk n x k 1 + j i,asclaimed. 8

9 To show the monotonicity, we differentiate two cases: If lk n = c, thenln k+1 ln k = c by definition of li n s. When ln k <c: r k+1 <r k u n (l n k ) u n (l n k +1), by definition of l n k, ordering of classes and Lemma 2, implying that ln k+1 ln k. 2 Theorem 1 states that an optimal policy accepts jobs sequentially in the order of class indices, which in fact reflects the order of the rewards. Moreover, jobs of a class will be accepted to the system until the corresponding threshold on the number of jobs in the system is reached, where the optimal thresholds are monotone. Thus, for these systems, there exists an optimal sequential threshold policy with monotone thresholds. When the batches consist of only one class of jobs, the structure of the optimal policy has a much simpler form: Corollary 2 In a system which receives batches containing only one class of jobs, class 1 is preferred so that a n 1 (x; j) =min{j 1,c x}. Moreover, there exist numbers lk n such that: a n k(x; j) =min j k, (lk n x) +. with ln k+1 ln k 4 Comparison of three loss systems In this section, we present a very simple example, which points out the unstructural characteristic of the batch acceptance. Consider a system with 8 servers, which receive only one class of jobs, but in either 5-unit batches or 1-unit batches. The parameter values are as follows before normalization: λ = 10, p 5 =0.7, p 1 =0.3, 1/µ =2,r = 10, and β = 1, where we optimize over an infinite horizon. Obviously, systems with partial acceptance will always admit as many jobs as possible. The optimal policy for batch acceptance, on the other hand, accepts all batches whenever it can, except for state x = 3, in which it rejects 1-unit batches. The intuition is clear: System waits for a batch, which will fully use its resources. If one small job is accepted in state 3, most of the capacity will stay idle until many 1-unit jobs arrive at the system. However, this intuition cannot be represented by any monotonicity property of optimal value functions. The system in this example can be modelled as a stochastic knapsack with two classes of jobs: The 5-unit batches, when admitted, will occupy five servers until the end of a common servicetime, andsotheyhavetobedifferentiated from the 1-unit jobs. Hence, the state of the system will be (x 1,x 2 ), where x 1 is the number of 5-unit jobs, and x 2 is the number of 1-unit jobs in the system. Then, the size of the two jobs change to b 1 =5andb 2 =1,whilethe arrivalstreamaswellastherewardsandtheservicetimesarethesameasabove. Notethat (x 1,x 2 ) has to satisfy 5x 1 + x 2 8. The optimal policy is very similar to that for the system with batch acceptance: All batches are accepted, whenever possible, except for state (0, 3), in which 1-unit jobs are rejected to wait for a 5-unit job. However, due to the differences in system dynamics which changes the state definition, this policy can be represented as a threshold policy (see [2]). The difference between the optimal policies of these two systems shows that the effect of independently-behaving jobs is rather remarkable. 9

10 5 Possible Extensions A natural extension of this work is to consider a system which receives jobs demanding different service rates. In fact, Örmeci and Burnetas [10] have considered such systems with two classes of jobs. They derive sufficient conditions for each of the classes to be always accepted to the system, and establish the submodularity of the value functions. References [1] E. Altman, S. Bhulai, B. Gaujal, and A. Hordijk. Open-loop routing to m parallel servers with no buffers. Journal of Applied Probability, 37: , [2] E. Altman, T. Jiménez, and G. M. Koole. On optimal call admission control in a resourcesharing system. IEEE Trans. on Communications, 49: , [3] W. Carrol and R. Grimes. Evolutionary change in product management: Experiences in the car rental industry. Interfaces, 25:84 104, [4] M. K. Geraghty and E. Johnson. Revenue management saves national car rental. Interfaces, 27: , [5] G. M. Koole. Structural results for the control of queueing systems using event-based dynamic programming. Queueing Systems, 30: , [6] G. M. Koole. On the static assignment to parallel servers. IEEE Trans. Automat. Control, 44: , [7] S. A. Lippman. Applying a new device in the optimization of exponential queueing systems. Operations Research, 23: , [8] S. A. Lippman and S. M. Ross. The streetwalker s dilemma: A job shop model. SIAM J. Appl. Math., 20: , [9] B. Miller. A queuing reward system with several customer classes. Management Science, 16: , [10] E. L. Örmeci and A. Burnetas. Admission Policies for a Two Class Loss System with Batch Arrivals. EURANDOM Technical Report , submitted to Adv. Appl. Prob., Eindhoven, [11] E. L. Örmeci, A. Burnetas, and J. van der Wal. Admission policies for a two class loss system. Stochastic Models, 17: , [12] K. W. Ross. Multiservice Loss Models for Broadband Telecommunication Networks. Springer-Verlag, Great Britain, [13] S. Savin, M. Cohen, N. Gans, and Z. Katalan. Capacity Management in Rental Businesses with Heterogeneous Customer Bases. Operations Research, under revision,

11 [14] J. Walrand. Introduction to Queueing Networks. Prentice Hall, Englewood Cliffs, N.J.,