3: Inventory management

Transcription

1 INSE6300 Ji Yun Yu 3: Inventory mngement Concordi Februry 9, 2016 Supply chin mngement is bout mking sequence of decisions over sequence of time steps, fter mking observtions t ech of these time steps. We illustrte this with the problem of mnging n inventory of nonperishble goods when demnd is stochstic. We first need to introduce the notion of Mrkov chins (e.g., wether model, coupon collector, etc.). Figure 1: From wether-model.png 1 Numericl exmple Consider the lst horse-drwn crrige deler in the world, which just took lese of N = 4 months on showroom. At time t = 1, 2, 3, 4, the strting inventory level is s t, the order size is t, nd the rndom demnd is d t. The rndom vribles {d t } re i.i.d. with the following distribution: 0 w.p. 0.1, d t = 1 w.p. 0.7, 2 w.p

2 Figure 2: From exportersindi.com The inventory level t the next time step t + 1 evolves ccording to the following Mrkov process: 0 if s t + t d t < 0, s t+1 = s t + t d t if 0 s t + t d t 2, (1) 2 if s t + t d t > 2. Suppose tht the ordering cost of ech unit of inventory is 1, nd tht both the holding nd bckorder costs re qudrtic, the overll cost is c(s t, t, d t ) = t + (s t + t d t ) 2. Unsold inventory hs no slvge vlue. Bckwrd induction strts t t = N = 4, nd ssigns vlue to ech inventory stte. Since unsold inventory hs no slvge vlue, we hve V 4 (0) = V 4 (1) = V 4 (2) = 0. For t = 1, 2, 3, we ssign the following vlues to the sttes: V t (s) = min E{c(s,, d t ) + V t+1 (s t+1 )} = min E{c(s,, d t ) + V t+1 ([s + d t ] 2 0)}, where we used the definition of s t+1 in (1) nd the nottion [ ] 2 0 to clmp to stte to the llowed rnge. Nmely, for t = 3, using the distribution of d t bove, we obtin V 3 (s) = min E{c(s,, d 3 ) + V 4 (s 4 )} = min E{ + (s + d t ) 2 + 0} ( = min + 0.1(s + 0) (s + 1) (s + 2) ), 2 2

3 nd The optiml order sizes re Repet for t = 2 nd t = 1: 1.3 if s = 0, V 3 (s) = 0.3 if s = 1, 1.1 if s = 2. 3(s) = 0 if s = 1, 2.5 if s = 0, V 2 (s) = 1.5 if s = 1, 1.68 if s = 2. 2(s) = 0 if s = 1, 3.7 if s = 0, V 1 (s) = 2.7 if s = 1, if s = 2. 1(s) = 0 if s = 1, Observe tht the optiml order size is 1 if the current inventory is 0, nd 0 otherwise. 2 Stochstic inventory mngement Consider single product (e.g., crs), nd discrete time steps (e.g., months 1, 2, etc.). Every time step (e.g., every month), the decision mker oberserves the current inventory level, nd decides how much inventory to order from the supplier. There re costs for holding inventory. The demnd is rndom, but we know the distribution of the rndom vrible. The gol is mximize the expected vlue of the profit (revenue minus costs) over number N of months. Assumptions: Delivery is instntneous (no led-time); The demnd tke integer vlues; 3

4 The demnd is i.i.d. with given distribution p j = P(D t = j) for j = 0, 1,...; Inventory hs cpcity M. For time steps t = 1, 2,..., let s t denote the inventory level, t the order size, nd D t the demnd t time t these re ll integer-vlued. The inventory level from one time step to the next follows this dynmics: The rewrd or profit t time t is s t+1 = mx{s t + t D t, 0}. r t (s t, t ) = F (s t + t ) O( t ) h(s t + t ), for t = 1,..., N 1, present vlue of inventory order holding r N (s N, N ) = g(s N, N ). slvge vlue where the expected present vlue of inventory is F (z) = z 1 j=0 f(j) revenue from j sles p j + j z f(z) p k, for z = 0, 1,... revenue cpped to z sles The order nd holding cost function cn be rbitrry; for instnce, O(z) = [K + c(z)]1 [z>0]. Remrk 1. Bckorder costs (missed sles) re implicitly ccounted for in the profit. 2.1 MDP We cn describe the stochstic inventory mngement problem s n MDP. The inputs re: Holding cost function h, order cost O, sles revenue f, slvge revenue g; Probbilities p 0, p 1,...; Time horizon: {1, 2,..., N}; Stte spce: S = {0, 1,..., M}; Action spce: A = {0, 1,..., M}; Expected rewrd: r 1, r 2,..., r N ; Stte trnsition probbilities: P (s s, ) = 0 if s (s +, M], p s+ s if s (0, s + ] nd s + M, k>s+ p k if s = 0 nd s + M. This is the probbility of hving n inventory level s t the next time step when the inventory level t the current time step is s nd we order units of inventory. 4

5 The output is optiml sequence of policies σ 1, σ 2,..., where σ j : S A. These policies re used to pick the optiml ction to tke t ech time step: suppose tht t time t = 1, 2,..., N, we observe the stte s t ( rndom vrible), then the optiml ction is σ t (s t ). Remrk 2 (Rewrd vs cost). We cn define the MDP in terms of costs by replcing the expected rewrd by expected cost, s in the numericl exmple bove, nd by replcing the mx by min. 2.2 Solving finite-horizon MDP by bckwrd induction How do we compute the optiml policies σ 1, σ 2,...? We propose method of dynmic progrmming clled bckwrd induction. The bckwrd induction lgorithm for MDPs proceeds s follows. 1. Set j = N, nd V N (s) = mx A r N (s, ) = g(s) for ll s S; 2. For j = N 1, N 2,..., 1: () For s S: i. Compute V j (s) = mx A { r j (s, ) + s S P (s s, )V j+1 (s) } ; ii. Output σ j (s) rg mx A { rj (s, ) + s S P (s s, )V j+1 (s) }. The output policies σ 1,..., σ N re optiml (cf. Putermn, Section 4.3). 3 References Mrkov Decision Processes, M. Putermn, Chpter 1. 5