Dscrete Event Dynamc Systems: Theory and Applcatons, 10, 51 70, 000. c 000 Kluwer Academc Publshers, Boston. Manufactured n The Netherlands. Smulaton Budget Allocaton for Further Enhancng the Effcency of Ordnal Optmzaton CHUN-HUNG CHEN, JIANWU LIN Department of Systems Engneerng, Unversty of Pennsylvana, Phladelpha, PA 19104-6315 ENVER YÜCESAN Technology Management Area INSEAD Fontanebleau, France STEPHEN E. CHICK Department of Industral and Operatons Engneerng, Unversty of Mchgan, Ann Arbor, MI 48109-117 Abstract. Ordnal Optmzaton has emerged as an effcent technque for smulaton and optmzaton. Exponental convergence rates can be acheved n many cases. In ths paper, we present a new approach that can further enhance the effcency of ordnal optmzaton. Our approach determnes a hghly effcent number of smulaton replcatons or samples and sgnfcantly reduces the total smulaton cost. We also compare several dfferent allocaton procedures, ncludng a popular two-stage procedure n smulaton lterature. Numercal testng shows that our approach s much more effcent than all compared methods. The results further ndcate that our approach can obtan a speedup factor of hgher than 0 above and beyond the speedup acheved by the use of ordnal optmzaton for a 10-desgn example. Keywords: dscrete-event smulaton, stochastc optmzaton, ordnal optmsaton queung network 1. Introducton Dscrete-event systems (DES) smulaton s a popular tool for analyzng systems and evaluatng decson problems snce real stuatons rarely satsfy the assumptons of analytcal models. Whle DES smulaton has many advantages for modelng complex systems, effcency s stll a sgnfcant concern when conductng smulaton experments (Law and Kelton, 1991). To obtan a good statstcal estmate for a desgn decson, a large number of smulaton samples or replcatons s usually requred for each desgn alternatve. Ths s due to the slow convergence of a performance measure estmator relatve to the number of smulaton samples or replcatons. The ultmate accuracy (typcally expressed as a confdence nterval) of ths estmator cannot mprove faster than O(1/ N), the result of averagng..d. nose, where N s the number of smulaton samples or replcatons (Faban, 1971; Kushner and Clark, 1978). If the accuracy requrement s hgh, and f the total number of desgns n a decson problem s large, then the total smulaton cost can easly become prohbtvely hgh. Ordnal Optmzaton has emerged as an effcent technque for smulaton and optmzaton. The underlyng phlosophy s to obtan good estmates through ordnal comparson whle the value of an estmate s stll very poor (Ho et al., 199). If our goal s to fnd the good desgns rather than to fnd an accurate estmate of the best performance value, whch s true n many practcal stuatons, t s advantageous to use ordnal comparson for selectng
5 CHEN ET AL. a good desgn. Further Da (1996) shows that the convergence rate for ordnal optmzaton can be exponental. Ths dea has been successfully appled to several problems (e.g., Cassandras et al., 1998; Gong et al., 1999; Patss et al., 1997). Whle ordnal optmzaton could sgnfcantly reduce the computatonal cost for DES smulaton, there s potental to further mprove ts performance by ntellgently controllng the smulaton experments, or by determnng the best number of smulaton samples among dfferent desgns as smulaton proceeds. The man theme of ths paper s to further enhance the effcency of ordnal optmzaton n smulaton experments. As we wll show n the numercal testng n secton 4, the speedup factor can be another order of magntude above and beyond the exponental convergence of ordnal optmzaton. Intutvely, to ensure a hgh probablty of correctly selectng a good desgn or a hgh algnment probablty n ordnal optmzaton, a larger porton of the computng budget should be allocated to those desgns that are crtcal n the process of dentfyng good desgns. In other words, a larger number of smulatons must be conducted wth those crtcal desgns n order to reduce estmator varance. On the other hand, lmted computatonal effort should be expended on non-crtcal desgns that have lttle effect on dentfyng the good desgns even f they have large varances. Overall smulaton effcency s mproved as less computatonal effort s spent on smulatng non-crtcal desgns and more s spent on crtcal desgns. Ideally, one would lke to allocate smulaton trals to desgns n a way that maxmzes the probablty of selectng the best desgn wthn a gven computng budget. We present a new optmal computng budget allocaton (OCBA) technque to accomplsh ths goal. Prevous researchers have examned varous approaches for effcently allocatng a fxed computng budget across desgn alternatves. Chen (1995) formulates the procedure of allocatng computatonal efforts as a nonlnear optmzaton problem. Chen et al. (1996) apply the steepest-ascent method to solve the budget allocaton problem. The major drawback of the steepest-ascent method s that an extra computaton cost s needed to teratvely search for a soluton to the budget allocaton problem. Such an extra cost could be sgnfcant f the number of teratons s large. Chen et al. (1997) ntroduce a greedy heurstc to solve the budget allocaton problem. Ths greedy heurstc teratvely determnes whch desgn appears to be the most promsng for further smulaton. However, the budget allocaton selected by the greedy heurstc s not necessarly optmal. On the other hand, Chen et al. (000) replace the objectve functon wth an approxmaton and the use of Chernoff s bounds, and present an analytcal soluton to the approxmaton. The approach n Chen et al. (000) provdes a more effcent allocaton than the greedy approach and the steepest-ascent method. In ths paper, we develop a new asymptotcally optmal approach for solvng the budget allocaton problem. The presented approach s even more effcent than the one gven by Chen et al. (000). Ths s accomplshed by replacng the objectve functon wth a better approxmaton that can be solved analytcally. Further, Chernoff s bounds are not used n the dervaton and fewer assumptons are mposed. The hgher effcency of ths new allocaton approach s also shown n the numercal testng. In addton to presentng a new and more effcent approach to determne the smulaton budget allocaton, n ths paper, we wll ) compare several dfferent budget allocaton
THE EFFICIENCY OF ORDINAL OPTIMIZATION 53 procedures through a seres of numercal experments; ) demonstrate that our budget allocaton approaches are much more effcent than the popular two-stage Rnott procedure; ) show that our approach s robust and the most effcent n dfferent settngs; v) show that the speedup factor becomes even larger when the number of desgns ncrease; v) demonstrate that addtonal sgnfcant speedup can be acheved above and beyond the exponental convergence of ordnal optmzaton. The paper s organzed as follows: In the next secton, we formulate the optmal computng budget allocaton problem. Snce our approach s based on the Bayesan model, we also provde a bref dscusson of that model for completeness. Secton 3 presents an asymptotc allocaton rule for OCBA. The performance of the technque s llustrated wth a seres of numercal examples n Secton 4. Secton 5 concludes the paper.. Problem Statement Suppose we have a complex dscrete event system. A general smulaton and optmzaton problem for such a DES system can be defned as mn J(θ ) E[L(θ,ξ)] (1) θ where, the search space, s an arbtrary, huge, structureless but fnte set; θ s the system desgn parameter vector for desgn, = 1,,...,k; J, the performance crteron whch s the expectaton of L, the sample performance, as a functonal of θ, and ξ, a random vector that represents uncertan factors n the systems. Note that for the complex systems consdered n ths paper, L(θ, ξ) s avalable only n the form of a complex calculaton va smulaton. The system constrants are mplctly nvolved n the smulaton process, and so are not shown n (1). The standard approach s to estmate E[L(θ,ξ)] by the sample mean performance measure J 1 N N j=1 L(θ,ξ j ), where ξ j represents the j-th sample of ξ and N represents the number of smulaton samples for desgn. Denote by σ : the varance for desgn,.e., σ = Var(L(θ,ξ)). In practce, σ s unknown beforehand and so s approxmated by sample varance. b: the desgn havng the smallest sample mean performance measure,.e., J b mn J, δ b, J b J. As N ncreases, J becomes a better approxmaton to J(θ ) n the sense that ts correspondng confdence nterval becomes narrower. The ultmate accuracy of ths estmate cannot mprove faster than 1/ N. Note that each sample of L(θ,ξ j ) requres one smulaton run. A large number of requred samples of L(θ,ξ j ) for all desgns may become very
54 CHEN ET AL. tme consumng. On the other hand, ordnal optmzaton (Ho et al., 199) concentrates on ordnal comparson and acheves a much faster convergence rate. Da (1996) shows that an algnment probablty of ordnal comparson can converge to 1.0 exponentally fast n most cases. Such an algnment probablty s also called the probablty of correct selecton or P{CS}. One example of P{CS} s the probablty that desgn b s actually the best desgn. To take advantage of such an exponental convergence, our approach s developed under the framework of ordnal comparson. Furthermore, nstead of equally smulatng all desgns, we wll further mprove the performance of ordnal optmzaton by determnng the best numbers of smulaton samples for each desgn. Statng ths more precsely, we wsh to choose N 1, N,...,N k such that P{CS} s maxmzed, subject to a lmted computng budget T, max P{CS} N 1,...,N k s.t. N 1 + N + +N k = T. N N, = 1,...,k. () Here N s the set of non-negatve ntegers and N 1 + N + + N k denotes the total computatonal cost assumng the smulaton tmes for dfferent desgns are roughly the same. To solve problem (), we must be able to estmate P{CS}. There exsts a large lterature on assessng P{CS} based on classcal statstcal models (e.g., Goldsman and Nelson, 1994; Banks, 1998 gve an excellent survey on avalable approaches). However, most of these approaches are only sutable for problems wth a small number of desgns. Recently, Chen (1996) ntroduced an estmaton technque that approxmates P{CS} for ordnal comparson when the number of desgns s large based on a Bayesan model (Bernardo and Smth, 1994). Ths technque has the added beneft of provdng senstvty nformaton that s useful n solvng problem (). We wll ncorporate ths technque wthn our budget allocaton approach. Many performance measures of nterest are taken over some averages of a sample path or a batch of samples. Thus, the smulaton output tends to be normally dstrbuted n many applcatons. In ths paper we assume that the smulaton output, L(θ, ξ), s normally dstrbuted. However, we wll demonstrate that our approach works equally well when for non-normal dstrbutons. After the smulaton s performed, a posteror dstrbuton of J(θ ), p(j(θ ) L(θ,ξ j ), j = 1,...,N ), can be constructed based on two peces of nformaton: () pror knowledge of the system s performance, and () current smulaton output. If we select the observed best desgn (desgn b), the probablty that we selected the best desgn s P{CS} = P{desgn b s actually the best desgn} = P{J(θ b )<J(θ ), b L(θ,ξ j ), j = 1,...,N, = 1,,...,k}. (3) To smplfy the notaton used, we rewrte Eq. (3) as P{ J b < J, b}, where J denotes the random varable whose probablty dstrbuton s the posteror dstrbuton for desgn. Assume that the unknown mean J(θ ) has the conjugate normal pror dstrbuton. We consder non-nformatve pror dstrbutons. Ths mples that no pror knowledge s gven
THE EFFICIENCY OF ORDINAL OPTIMIZATION 55 about the performance of any desgn alternatve before conductng the smulaton. In that case, DeGroot (1970) shows that the posteror dstrbuton of J(θ ) s ( J N J, σ ). N After the smulaton s performed, J can be calculated, σ can be approxmated by the sample varance; P{CS} can then be estmated usng a Monte Carlo smulaton. However, estmatng P{CS} va Monte Carlo smulaton s tme-consumng. Snce the purpose of budget allocaton s to mprove smulaton effcency, we need a relatvely fast and nexpensve way of estmatng P{CS} wthn the budget allocaton procedure. Effcency s more crucal than estmaton accuracy n ths settng. We adopt a common approxmaton procedure used n smulaton and statstcs lterature (Brately et al., 1987; Chck, 1997; Law and Kelton, 1991). Ths approxmaton s based on the Bonferron nequalty. Let Y be a random varable. Accordng to the Bonferron nequalty, P{ =1 k (Y < 0)} 1 k =1 [1 P{Y < 0}]. In our case, Y s replaced by ( J b J ) to provde a lower bound for the probablty of correct selecton. That s, { k P{CS} = P = 1 =1, b =1, b ( ) } J b J < 0 1 { } P J b > J = APCS. =1, b [ { }] 1 P J b J < 0 We refer to ths lower bound of the correct selecton probablty as the Approxmate Probablty of Correct Selecton (APCS). APCS can be computed very easly and quckly; no extra Monte Carlo smulaton s needed. Numercal tests show that the APCS approxmaton can stll lead to hghly effcent procedures (e.g., Chen, 1996; Inoue and Chck, 1998). We therefore use APCS to approxmate P{CS} as the smulaton experment proceeds. More specfcally, we consder the followng problem: max 1 N 1,...,N k s.t. =1, b { } P J b > J N = T and N 0. (4) =1 In the next secton, an asymptotc allocaton rule wth respect to the number of smulaton replcatons, N wll be presented. 3. An Asymptotc Allocaton Rule Frst, we assume the varables, N s, are contnuous. Second, our strategy s to tentatvely gnore all non-negatvty constrants; all N s can therefore assume any real value. Let N = α T. Thus, k =1 α = 1. Before the end of ths secton, we wll show how all α s
56 CHEN ET AL. become postve and hence all N s are postve. Based on ths dea, we frst consder the followng: { } max 1 P J b > J N 1,...,N k s.t. =1, b N = T. (5) =1 For the objectve functon, P{ J b > J } = =1, b = =1 b =1 b 0 (x δ b, ) 1 σ b, dx πσb, 1 δ b, σ b, π e t dt, where a new varable s ntroduced, σb, = σ b N, for notaton smplfcaton (δ b, s defned n secton ). Then, let F be the Lagrangan relaxaton of (5): ( ) F = 1 e t dt λ N T. π =1 =1 b 1 δ b, σ b, N b + σ Furthermore, the Karush-Kuhn-Tucker (KKT) (Walker, 1999) condtons of ths problem can be stated as follows. ( ) F F δ b, σ b, σ b, = ( ) λ N δ b, σ b, N σ b, [ ] δ b, = 1 π exp F = 1 N b π σ b, [ δ b, exp =1 b σ b, δ b, σ N (σ b, ] ( ) λ N T = 0, and λ 0. =1 )3/ λ = 0, for = 1,,...,k, and b. (6) δ b, σb Nb (σ b, λ = 0, (7) )3/ We now examne the relatonshp between N b and N for = 1,,...,k, and b. From Eq. (6), [ ] 1 δ π exp b, δ b, N = λ, for = 1,,...,k, and b. (8) )3/ σ b, (σ b, σ
THE EFFICIENCY OF ORDINAL OPTIMIZATION 57 Pluggng (8) nto (7), we have =1 b λn σ b N b σ λ = 0. Then N b = σ b k =1, b N σ or α b = σ b k α =1, b σ. (9) We further nvestgate the relatonshp between N and N j, for any, j {1,,...,k}, and j b. From Eq. (6), exp δ b, δ ( ) b, σ σ /N δ b N b + σ ( ) σ b N N b + σ 3/ = exp b, ( ) σ N b N b + σ δ b, j σj /N j ( ). (10) 3/ j σ b N j N b + σ j N j To reduce the total smulaton tme for dentfyng a good desgn, t s worthwhle to concentrate the computatonal effort on good desgns. Namely, N b should be ncreased relatve to N, for = 1,,...,k, and b. Ths s ndeed the case n the actual smulaton experments. And ths assumpton can be supported by consderng a specal case n Eq. (9): When σ 1 = σ = =σ k, N b = k =1, b N. Therefore, we assume that N b N, whch enables us to smplfy Eq. (10) as exp δ b, ( ) σ δb,σ /N δ ( ) σ 3/ = exp b, j ( ) σ δb, jσj /N j ( ). N N 3/ j σ j N j N j On rearrangng terms, the above equaton becomes exp 1 δb, N 1/ j = δ b, jσ. δ b, σ j δ b, j σ j N j σ N N 1/ Takng the natural log on both sdes, we have δ b, j σ j N j + log(n j ) = δ b, σ N + log(n ) + log ( δb, j σ δ b, σ j ),
58 CHEN ET AL. or δ b, j σ j α j T + log(α j T ) = δ b, σ α T + log(α T ) + log ( δb, j σ δ b, σ j ), whch yelds δ b, j σ j α j T + log(α j ) = δ b, σ α T + log(α ) + log ( δb, j σ δ b, σ j ). (11) To further facltate the computatons, we ntend to fnd an asymptotc allocaton rule. Namely, we consder the case that T. Whle t s mpossble to have an nfnte computng budget n real lfe, our allocaton rule provdes a smple means for allocatng smulaton budget n a way that the effcency can be sgnfcantly mproved, as we wll demonstrate n numercal testng later. As T, all the log terms become much smaller than the other terms and are neglgble. Ths mples δ b, j σ j α j = δ b, σ α Therefore, we obtan the rato between α and α j or between N and N j as: N = α ( ) σ /δ b, = for = 1,,...,k, and j b. (1) N j α j σ j /δ b, j Now we return to the ssue of nonnegatve constrant for N, whch we temporarly gnored. Note that from Eq. (9) and Eq. (1), all α s have the same sgn. Snce k =1 α = 1 and N = α T, t mples that all α s 0, and hence N s 0, where = 1,,...,k. In concluson, f a soluton satsfes Eq. (9) and Eq. (1), then KKT condtons must hold. Accordng to the KKT Suffcent Condton, ths soluton s a local optmal soluton to Eq. (4). We therefore have the followng result: THEOREM 1 Gven a total number of smulaton samples T to be allocated to k competng desgns whose performance s depcted by random varables wth means J(θ 1 ), J(θ ),..., J(θ k ), and fnte varances σ1,σ,...,σ k respectvely, as T, the Approxmate Probablty of Correct Selecton (APCS) can be asymptotcally maxmzed when (1) ( ) N N j = σ /δ b, σ j /δ b, j,, j {1,,...,k}, and j b, () N b = σ b k =1, b N σ. where N s the number of samples allocated to desgn, δ b, = J b J, and J b mn J.
THE EFFICIENCY OF ORDINAL OPTIMIZATION 59 Remark 1. In the case of k = and b = 1, Theorem 1 yelds N N 1 = σ 1 σ, Therefore, N 1 = σ 1. N σ Ths evaluated result s dentcal to the well-known optmal allocaton soluton for k =. Remark. To gan better nsght nto ths approach, consder another case where k = 3 and b = 1, whch yelds N = σ δ 1,3 N 3 σ3. δ1 1, We can see how the number of smulaton samples for desgn, N, s affected by dfferent factors. For a mnmzaton problem, when J 3 ncreases (we are more confdent of the dfference between desgn 1 and desgn 3) or σ ncreases (we are less certan about desgn ), N ncreases as well. On the other hand, when J ncreases (we are more confdent of the dfference between desgn 1 and desgn ) or σ 3 ncreases (we are less certan about desgn 3), N decreases. The above relatonshp between N and other factors s consstent wth our ntuton. Wth Theorem 1, we now present a cost-effectve sequental approach based on OCBA to select the best desgn from k alternatves wth a gven computng budget. Intally, n 0 smulaton replcatons for each of k desgn are conducted to get some nformaton about the performance of each desgn durng the frst stage. As smulaton proceeds, the sample means and sample varances of each desgn are computed from the data already collected up to that stage. Accordng to ths collected smulaton output, an ncremental computng budget,, s allocated based on Theorem 1 at each stage. Ideally, each new replcaton should brng us closer to the optmal soluton. Ths procedure s contnued untl the total budget T s exhausted. The algorthm s summarzed as follows. A Sequental Algorthm for Optmal Computng Budget Allocaton (OCBA) Step 0. Perform n 0 smulaton replcatons for all desgns; l 0; N1 l = N l = = N k l = n 0. Step 1. If k =1 N l T, stop. Step. Increase the computng budget (.e., number of addtonal smulatons) by and compute the new budget allocaton, N l+1 1, N l+1,...,n l+1 k, usng Theorem 1. Step 3. Perform addtonal max(0, N l+1 l l + 1. Go to Step 1. N l ) smulatons for desgn, = 1,...,k.
60 CHEN ET AL. In the above algorthm, l s the teraton number. As smulaton evolves, desgn b, whch s the desgn wth the largest sample mean, may change from teraton to teraton, although t wll converge to the optmal desgn as the l goes to nfnty. When b changes, Theorem 1 s drectly appled n step. However, the older desgn b may not be smulated at all n ths teraton n step 3 due to extra allocaton to ths desgn n earler teratons. In addton, we need to select the ntal number of smulatons, n 0, and the one-tme ncrement,. Chen et al. (1999) offers detaled dscussons on the selecton. It s well understood that n 0 cannot be too small as the estmates of the mean and the varance may be very poor, resultng n premature termnaton of the comparson. A sutable choce for n 0 s between 5 and 0 (Law and Kelton, 1991; Bechhofer et al., 1995). Also, a large can result n waste of computaton tme to obtan an unnecessarly hgh confdence level. On the other hand, f s small, we need to the computaton procedure n step many tmes. A suggested choce for s a number bgger than 5 but smaller than 10% of the smulated desgns. 4. Numercal Testng and Comparson wth Other Allocaton Procedures In ths secton, we test our OCBA algorthm and compare t wth several dfferent allocaton procedures by performng a seres of numercal experments. We also apply our OCBA to a buffer resource allocaton problem, whch has 10 desgn alternatves. 4.1. Dfferent Allocaton Procedures In addton to the OCBA algorthm, we test several procedures and compare ther performances. Among them, equal allocaton represents the sole use of ordnal optmzaton, the greedy allocaton and CCY are developed based a same Bayesan framework gven n secton, and Rnott s hghly popular n smulaton lterature. We brefly summarze the compared allocaton procedures as follows. Equal Allocaton Ths s the smplest way to conduct smulaton experments and has been wdely appled. The smulaton budget s equally allocated to all desgns. Namely, all desgns are equally smulated and then we focus on ordnal comparson. Such a way s equvalent to the sole use of ordnal optmzaton. As we noted n prevous sectons, ordnal optmzaton can ensure that P{CS} converges to 1.0 exponentally fast even f we smulate all desgn alternatves equally, that s, N = T/k for each. The performance of equal allocaton wll serve as a benchmark for comparson. Greedy Allocaton The greedy procedure offered by Chen et al. (1997) s developed based on the Bayesan framework presented n secton. They ntroduce an approach for approxmately estmatng
THE EFFICIENCY OF ORDINAL OPTIMIZATION 61 the gradent of P{CS} wth respect to N. Snce we ntend to maxmze the resultng P{CS}, a greedy approach selects and smulates a subset of promsng desgns n each teraton, then repeats the process untl the total budget s exhausted. Snce the gradent of P{CS} wth respect to N s an ndcaton of how much P{CS} can be mproved f we perform addtonal smulatons on desgn, the promsng desgns are defned as those whch have large gradents. More specfcally, n each teraton, we select the set of desgns wth topm largest gradents. Then the computng budget for ths teraton s equally allocated to these m desgns. Obvously, the budget allocaton selected by the greedy heurstc s not necessarly optmal. Chen, Chen and Yücesan Procedure (CCY) Ths s also developed based on the Bayesan framework presented n secton and s proposed by Chen et al. (000). Smlar to the OCBA algorthm, CCY s an asymptotc soluton to an approxmaton problem. However, Chernoff s bounds are used and further assumptons are mposed n the development of CCY. At a theoretcal level, the OCBA algorthm s superor to CCY. As we show later n the numercal experments, the OCBA algorthm ndeed performs better than the CCY procedure, although both outperform other compared procedures. CCY allocates smulaton budget accordng to: (1) () N N s = N b N s = σ b σ s ( ) σ /δ b, for = 1,...,k and s b, σ s /δ b,s ( ) k δ b,s 1/, =1 b δ b, where s s the desgn havng the second smallest sample mean performance measure. Two-Stage Rnott Procedure The two-stage procedure of Rnott (1978) has been wdely appled n the smulaton lterature (Law and Kelton, 1991). Unlke the OCBA approach, the two-stage procedures are developed based on the classcal statstcal model. See Bechhofer et al. (1995) for a systematc dscusson of two-stage procedures. In the frst stage, all desgns are smulated for n 0 samples. Based on the sample varance estmate (S ) obtaned from the frst stage, the number of addtonal smulaton samples for each desgn n the second stage s determned by: N = max ( 0, (S h /d n 0 ), for = 1,,...,k, where s the nteger round-up functon, d s the ndfference zone, h s a constant whch solves Rnott s ntegral (h can also be found from the tables n Wlcox, 1984). In short, the computng budget s allocated proportonally to the estmated sample varances.
6 CHEN ET AL. Fgure 1. P{CS} vs. T usng fve dfferent allocaton procedures for experment 1. Normal dstrbutons wth 10 desgns. The computaton costs n order to attan P{CS} =99% are ndcated. The major drawback s that only the nformaton on varances s used when determnng the smulaton allocaton, whle the OCBA algorthm, the CCY procedure and the greedy approach utlze the nformaton on both means and varances. As a result, the performance of Rnott s procedure s not as good as others. We do, however, nclude t n our testng due to ts popularty n the smulaton lterature. 4.. Numercal Experments The numercal experments nclude a seres of generc tests plus a test on the buffer allocaton problem. In all the numercal llustratons, we estmate P{CS} by countng the number of tmes we successfully fnd the true best desgn (desgn 0 n ths example) out of 10,000 ndependent applcatons of each selecton procedure. P{CS} s then obtaned by dvdng ths number by 10,000, representng the correct selecton frequency. Experment 1. Normal Dstrbuton There are ten desgn alternatves. Suppose L(θ,ξ)) N(, 6 ), = 0, 1,...,9. We want to fnd a desgn wth the mnmum mean. It s obvous that desgn 0 s the actual best desgn. In the numercal experment, we compare the convergence of P{CS} for dfferent allocaton procedures. We have n 0 = 10 and = 0. Dfferent computng budgets are tested. Fgure 1 shows the test results usng OCBA and the other four dfferent procedures dscussed n secton 4.1. Note that the smulaton varance of each desgn s 36, whle the dfference of two adjacent desgns means s only 1.
THE EFFICIENCY OF ORDINAL OPTIMIZATION 63 Gven such a hgh nose rato, we see that wth the total computaton cost as low as 700 smulaton samples, the probablty of correctly selectng the best desgn s already hgher than 80% even usng the smple equal allocaton. Ths demonstrates the advantage of applyng ordnal optmzaton. We see that all procedures obtan a hgher P{CS} as the avalable computng budget ncreases. However, OCBA acheves a same P{CS} wth a lower amount of computng budget than other procedures. In partcular, we ndcate the computaton costs n order to attan P{CS} =99% for dfferent procedures n Fgure 1. Whle ordnal optmzaton s effcent, our OCBA can further reduce the smulaton tme by 75% for P{CS} =99%. It s worth notng that Rnott s procedure does not perform much better than the smple equal allocaton. Ths s because Rnott s procedure determnes the number of smulaton samples for all desgns usng only the nformaton of sample varances. On the hand, Rnott s procedure s much slower than the greedy allocaton, CCY and OCBA. Ths s because when determnng budget allocaton, the latter three procedures explot the nformaton of both sample means and varances, whle Rnott s procedure does not utlze the nformaton of sample means. The sample means can provde the valuable nformaton of relatve dfferences across the desgn space. CCY s more effcent than the greedy allocaton snce CCY ntends to optmze the smulaton effcency. Fnally, our OCBA s even faster than CCY; the computaton costs for attanng P{CS} =99% are 1,100 vs. 1,400 samples. Experment. Unform Dstrbuton We consder a non-normal dstrbuton for the performance measure: L(θ,ξ)) Unform ( 10.5, + 10.5), = 0, 1,..., 9. The endponts of the unform dstrbuton are chosen such that the correspondng varance s close to that n experment 1. Agan, we want to fnd a desgn wth the mnmum mean; desgn 0 s therefore the actual best desgn. All other settngs are dentcal to experment 1. Fgure contans the smulaton results for the fve allocaton procedures. We can see that the relatve performances of the dfferent procedures are very smlar to what we saw n experment 1. OCBA s the fastest and s more than three tmes faster than Rnott and equal allocaton. Experment 3. Normal Dstrbuton wth Larger Varance Ths s a varant of experment 1. All settngs are preserved except that the varance of each desgn s doubled. Namely, L(θ,ξ)) N(, 6 ), = 0, 1,...,9. Fgure 3 contans the smulaton results for the fve allocaton procedures. We can see that the relatve performances of dfferent procedures are very smlar wth what we see n prevous experments, except that bgger computng budgets are needed n order to obtan the same P{CS}, due to larger varance. Also, OCBA s more than four tmes faster than equal allocaton.
64 CHEN ET AL. Fgure. P{CS} vs. T usng fve dfferent allocaton procedures for experment. Unform dstrbutons wth 10 desgns. The computaton costs n order to attan P{CS} =99% are ndcated. Fgure 3. P{CS} vs. T usng three dfferent allocaton procedures for experment. Normal dstrbutons wth 10 desgns. The computaton costs n order to attan P{CS} =99% are ndcated. Experment 4. Flat & Steep Case Ths s another varant of experment 1. We consder three generc cases llustrated n Fgure 4.1 (also shown n Ho et al., 199): neutral, flat, and steep. The neutral case s already presented n experment 1. In the flat case, L(θ,ξ)) N(9 3 9, 6 ), = 0, 1,...,9;
THE EFFICIENCY OF ORDINAL OPTIMIZATION 65 Fgure 4.1. Illustraton of three generc cases: neutral, flat, steep. ( and n the steep case L(θ,ξ)) N 9 ( 9 3 ), 6 ), = 0, 1,...,9. In the flat case, good desgns are closer; a larger computng budget s therefore needed to dentfy the best desgn gven the same smulaton estmaton nose. On the other hand, t s easer to correctly select the best desgn n the steep case snce the good desgns are further spread out. The numercal results n Fgures 4. and 4.3 support ths conjecture. In ether case, OCBA s the fastest and s more than three tmes faster than equal allocaton. Experment 5. Bgger Desgn Space Ths s also a varant of experment 1. To see the performance of the OCBA algorthm wthn a bgger desgn space, we ncrease the number of desgns to 100. L(θ,ξ)) N(/10, 1 ), = 0, 1,,...,98, 99. Note that we have the range of the means for these 100 desgns the same as those n earler 10-desgn experments, whch s from 0 to 10. Snce the performances of Rnott s procedure and equal allocaton are very close, and the performances of CCY and the greedy allocaton are also close, we wll test and compare only OCBA, greedy and the equal allocaton n the remanng experments. Fgure 5 depcts the smulaton results. The speedup factor of usng OCBA s ncreased to n these experments. Ths s because a larger desgn space gves the OCBA algorthm more flexblty n allocatng the computng budget.
66 CHEN ET AL. Fgure 4.. P{CS} vs. T usng three dfferent allocaton procedures for the flat case n experment 4. Normal dstrbutons wth 10 desgns. Fgure 4.3. P{CS} vs. T usng three dfferent allocaton procedures for the steep case n experment 4. Normal dstrbutons wth 10 desgns. The computaton costs n order to attan P{CS} =99% are ndcated. Experment 6. A Buffer Resource Allocaton Problem A 10-node network shown n Fgure 6.1 s used to test dfferent buffer allocaton procedures ncludng our OCBA. Detals about the network can be found n Chen and Ho (1995). There are 10 servers and 10 buffers that are connected as a swtchng network. There are two classes
THE EFFICIENCY OF ORDINAL OPTIMIZATION 67 Fgure 5. P{CS} vs. T usng three dfferent allocaton procedures for experment 5. Normal dstrbutons wth 100 desgns. The computaton costs n order to attan P{CS} =99% are ndcated. Fgure 6.1. A 10-node network. of customers wth dfferent arrval dstrbutons, but wth the same servce requrements. We consder both exponental and non-exponental dstrbutons (unform) n the network. Both classes arrve at any of nodes 0 through 3, and leave the network after havng gone through three dfferent stages of servce. The routng s class dependent. Such a network could be the model for a large number of real-world systems, such as a manufacturng system, a communcaton or a traffc network. Fnte buffer szes at all nodes are assumed. In ths desgn problem, we are nterested n dstrbutng optmally lmted buffer spaces to dfferent nodes. Specfcally, we consder the problem of allocatng 1 buffer unts, among the 10 dfferent nodes numbered from 0
68 CHEN ET AL. Fgure 6.. P{CS} vs. T usng three dfferent allocaton procedures for experment 6. Note the x-axs s n log scale. There are 10 desgns. The computaton costs n order to attan P{CS} =99% are ndcated. to 9. We denote the buffer sze of node by B. Thus, B 0 + B 1 + B + + B 9 = 1. There are three constrants for symmetry reasons: B 0 = B 1 = B = B 3, B 4 = B 6, and B 5 = B 7. Totally there are 10 desgn alternatves for consderaton. The objectve s to select a desgn wth mnmum expected tme to process the frst 100 jobs from a same ntal state (that s [B 0, B 1, B,...,B 9 ] = [1, 1, 1, 1,, 1,, 1, 1, 1]). Fgure 6. depcts the smulaton results for the three allocaton procedures. Once agan, we can see that the relatve performances of dfferent procedures are very smlar to what we saw n the prevous experments, except that bgger computng budgets are needed n order to obtan the same P{CS}, due to the larger desgn space. The speedup factor of usng OCBA s about 3, whch s even bgger than that n experment 5. Note that n Fgure 6., the x-axs s n log scale snce the dfferences of the computaton costs usng dfferent approaches are very large. In addton, we can see that OCBA s much more effcent even when T s small, despte that our algorthm s developed based on asymptotc allocaton. Ths s also true n earler numercal experments. 5. Conclusons We present a hghly effcent procedure to dentfy the best desgn out of k (smulated) competng desgns. The purpose of ths technque s to further enhance the effcency of or-
THE EFFICIENCY OF ORDINAL OPTIMIZATION 69 dnal optmzaton n smulaton experments. The objectve s to maxmze the smulaton effcency, expressed as the probablty of correct selecton wthn a gven computng budget. Our procedure allocates replcatons n a way that optmally mproves an asymptotc approxmaton to the probablty of correct selecton. We also compare several dfferent allocaton procedures, ncludng a popular two-stage procedure n smulaton lterature. Numercal testng shows that our approach s much more effcent than all compared methods. Comparsons wth the crude ordnal optmzaton show that our approach can acheve a speedup factor of 3 4 for a 10-desgn example. The speedup factor s even hgher wth the problems havng a larger number of desgns. For a buffer resource allocaton problem, n whch there are 10 desgns, our approach s more than 0 tmes faster than crude ordnal optmzaton. Although our procedure allocates the avalable computng budget based on an asymptotc dervaton, all of our numercal results ndcate that our procedure s hghly effectve when the computng budget s small. Whle ordnal optmzaton can converge exponentally fast, our smulaton budget allocaton procedure provdes a way to further sgnfcantly mprove overall smulaton effcency. Acknowledgements Ths work has been supported n part by NSF under Grant DMI-973173, by the U.S. Department of Transportaton uner a grant from the Unversty Transportaton Centers Program through the Md-Atlantc Transportaton Consortum, by Sanda Natonal Laboratores under Contract BD-0618, and by the Unversty of Pennsylvana Research Foundaton. References Banks, J. 1998. Handbook of Smulaton. John Wley. Bechhofer R. E., Santner, T. J., and Goldsman, D. M. 1995. Desgn and Analyss of Experments for Statstcal Selecton, Screenng, and Multple Comparsons. John Wley & Sons, Inc. Bernardo, J. M., and Smth, A. F. M. 1994. Bayesan Theory. Wley. Bratley, P., Fox, B. L., and Schrage, L. E. 1987. A Gude to Smulaton, nd ed. Sprnger-Verlag. Cassandras, C. G., Da, L., and Panayotou, C. G. 1998. Ordnal optmzaton for determnstc and stochastc dscrete resource allocaton. IEEE Trans. on Automatc Control 43(7): 881 900. Chen, C. H. 1995. An effectve approach to smartly allocate computng budget for dscrete event smulaton. Proceedngs of the 34th IEEE Conference on Decson and Control, pp. 598 605. Chen, C. H. 1996. A lower bound for the correct subset-selecton probablty and ts applcaton to dscrete event system smulatons. IEEE Transactons on Automatc Control 41(8): 17 131. Chen, C. H., and Ho, Y. C. 1995. An approxmaton approach of the standard clock method for general dscrete event smulaton. IEEE Transactons on Control Systems Technology 3(3): 309 317. Chen, H. C., Chen, C. H., Da, L., and Yücesan, E. 1997. New development of optmal computng budget allocaton for dscrete event smulaton. Proceedngs of the 1997 Wnter Smulaton Conference, pp. 334 341. Chen, C. H., Wu, S. D., and Da, L. 1999. Ordnal comparson of heurstc algorthms usng stochastc optmzaton. IEEE Transactons on Robotcs and Automaton 15(1): 44 56. Chen, H. C., Chen, C. H., and Yücesan, E. 000. Computng efforts allocaton for ordnal optmzaton and dscrete event smulaton. To appear n IEEE Transactons on Automatc Control. Chck, S. E. 1997. Bayesan analyss for smulaton nput and output. Proceedngs of the 1997 Wnter Smulaton Conference, pp. 53 60. Da, L. 1996. Convergence propertes of ordnal comparson n the smulaton of dscrete event dynamc systems. Journal of Optmzaton Theory and Applcatons 91(): 363 388.
70 CHEN ET AL. DeGroot, M. H. 1970. Optmal Statstcal Decsons. McGraw-Hll, Inc. Faban, V. 1971. Stochastc Approxmaton, Optmzaton Methods n Statstcs. J. S. Rustag (ed.), Academc Press, New York. Goldsman, G., and Nelson, B. L. 1994. Rankng, selecton, and multple comparson n computer smulaton. Proceedngs of the 1994 Wnter Smulaton Conference, pp. 19 199. Gong, W. B., Ho, Y. C., and Zha, W. 1999. Stochastc comparson algorthm for dscrete optmzaton wth estmaton. SIAM Journal on Optmzaton. Ho, Y. C., Sreenvas, R. S., and Vakl, P. 199. Ordnal optmzaton of DEDS. Journal of Dscrete Event Dynamc Systems (): 61 88. Inoue, K., and Chck, S. 1998. Comparson of Bayesan and Frequentst assessments of uncertanty for selectng the best system. Proceedngs of the 1998 Wnter Smulaton Conference, pp. 77 734. Kushner, H. J., and Clark, D. S. 1978. Stochastc Approxmaton for Constraned and Unconstraned Systems. Sprnger-Verlag. Law, A. M., and Kelton, W. D. 1991. Smulaton Modelng & Analyss. McGraw-Hll, Inc. Patss, N. T., Chen, C. H., and Larson, M. E. 1997. SIMD parallel dscrete event dynamc system smulaton. IEEE Transactons on Control Systems Technology 5(3): 30 41. Rnott, Y. 1978. On two-stage selecton procedures and related probablty nequaltes. Communcatons n Statstcs A7: 799 811. Walker, R. C. 1999. Introducton to Mathematcal Programmng. Prentce Hall. Wlcox, R. R. 1984. A table for Rnott s selecton procedure. Journal of Qualty Technology 16(): 97 100. Yan, D., and Muka, H. 1993. Optmzaton algorthm wth probablstc estmaton. Journal of Optmzaton Theory and Applcatons 79: 345 371.