A Case Study for Optmal Dynamc Smulaton Allocaton n Ordnal Optmzaton Chun-Hung Chen, Dongha He, and Mchael Fu 4 Abstract Ordnal Optmzaton has emerged as an effcent technque for smulaton and optmzaton. Exponental convergence rates can be acheved n many cases. A good allocaton of smulaton samples across desgns can further dramatcally mprove the effcency of ordnal optmzaton by orders of magntude. However, the allocaton problem tself s a bg challenge. Most exstng methods offer approxmatons. Assumng the avalablty of perfect nformaton, we nvestgate theoretcally optmal allocaton schemes for some specal cases. We compare our theoretcally optmal solutons wth exstng approxmaton methods usng a seres of numercal examples. Whle perfect nformaton s not avalable n real lfe, such an optmal soluton provdes an upper bound for the smulaton effcency we can acheve. The results ndcate that the smulaton effcency can stll be further mproved beyond the exstng methods. Also the numercal testng shows that dynamc allocaton s much more effcent than statc allocaton. D I. ITRODUCTIO ISCRETE-EVET systems (DES) smulaton s a popular tool for analyzng systems and evaluatng decson problems, snce real stuatons rarely satsfy the assumptons of analytcal models. Whle smulaton has many advantages for modelng complex systems, effcency s stll a sgnfcant concern when conductng smulaton experments. To obtan a good statstcal estmate for a desgn decson, a large number of smulaton samples or replcatons s usually requred for each desgn alternatve. If the accuracy requrement s hgh and the total number of desgns n a decson problem s large, then the total smulaton cost can easly become prohbtvely hgh. Ordnal Optmzaton has emerged as an effcent. Ths work has been supported n part by SF under Grants DMI-9988867, DMI-000900, DMI-004906, DMI-00, and IIS-05074, by ASA Ames Research Center under Grants AG--565 and AG--64, by FAA under Grant 00-G-06, by AFOSR under Grant F4960006, and by George Mason Unversty Research Foundaton.. Dr. Chun-Hung Chen s wth Department of Systems Engneerng & Operatons Research, George Mason Unversty, 4400 Unversty Drve, MS 4A6, Farfax, VA 00, USA (phone: 70-99-57; emal: cchen9@gmu.edu).. Mr. Dongha He s wth Department of Systems Engneerng & Operatons Research, George Mason Unversty, 4400 Unversty Drve, MS 4A6, Farfax, VA 00, USA (emal: dhe@gmu.edu). 4. Dr. Mchael Fu s wth the Robert H. Smth School of Busness and Insttute for Systems Research, Unversty of Maryland College Park, MD, 074-85,USA (emal: mfu@rhsmth.umd.edu) technque for smulaton and optmzaton. The underlyng phlosophy s to obtan good estmates through ordnal comparson whle the value of an estmate s stll very poor (Ho et al. 99). If our goal s to fnd the good desgns rather than to fnd an accurate estmate of the best performance value, whch s true n many practcal stuatons, t s advantageous to use ordnal comparson for selectng a good desgn. Further Da (996) shows that the convergence rate for ordnal optmzaton can be exponental. Ths dea has been successfully appled to several problems (e.g., Hseh et al. 00, Patss et al. 997). Whle ordnal optmzaton could sgnfcantly reduce the computatonal cost for DES smulaton, t has been shown that the effcency can be further dramatcally mproved by ntellgently controllng the smulaton experments, or by determnng the effcent number of smulaton samples among dfferent desgns as smulaton proceeds (Chen et al. 997). Intutvely, to ensure a hgh probablty of correctly selectng a good desgn or a hgh algnment probablty n ordnal optmzaton, a larger porton of the computng budget should be allocated to those desgns that are crtcal n the process of dentfyng the best desgn. In other words, a larger number of smulatons must be conducted wth those crtcal alternatves n order to reduce these crtcal estmators' varances. On the other hand, lmted computatonal effort should be expended on non-crtcal desgns that have lttle effect on dentfyng the good desgns even f they have large varances. Overall smulaton effcency s mproved as less computatonal effort s spent on smulatng non-crtcal alternatves and more s spent on crtcal alternatves. Ideally, one would lke to allocate smulaton samples to desgns n a way that maxmzes the probablty of selectng the best desgn wthn a gven computng budget. Chen et al. (000) formalze ths dea and develop a new approach called optmal computng budget allocaton (OCBA) algorthm. They demonstrate that the speedup factor can be another order of magntude above and beyond the exponental convergence of ordnal optmzaton. In addton, several smulaton budget allocaton schemes have been developed for varous applcatons or from dfferent perspectves (Chck and Inoue 00, Lee 00, Tralovc and Pao 004). They also show that the effcency can be sgnfcantly enhanced.
In ths paper we extensvely study the effcency ssue of smulaton budget allocaton for ordnal optmzaton. In applyng the aforementoned smulaton budget allocaton methods, one has to determne a good allocaton of smulaton replcatons (budget) usng some nformaton and then perform smulaton accordngly. One challenge s that the objectve functon we ntend to estmate and then optmze s a crtcal component n determnng the smulaton budget allocaton. Unfortunately a good estmate of the objectve functon s usually not avalable untl after the smulaton s carred out. Wthout a good estmate, the budget allocaton may not be good, whch has an mpact on the smulaton effcency. There are two possble approaches for handlng ths ssue. One s to perform some prelmnary smulaton to obtan nformaton for determnng budget allocaton, and then allocate the remanng smulaton budget at once. We call ths the statc allocaton. On the other hand, one may allocate only a small porton of the smulaton budget each tme and utlze the most updated smulated nformaton to determne the new budget allocaton teratvely. We refer to the second approach as a dynamc allocaton. Intutvely, a dynamc allocaton should work better than a statc allocaton. In ths paper we consder a small but general problem. We present theoretcal optmal allocaton schemes for both statc and dynamc allocaton. We also test some exstng budget allocaton methods aganst the presented theoretcally optmal allocaton usng three numercal examples. We fnd that dynamc allocaton s ndeed much more effcent than statc allocaton. It s nterestng to see that OCBA performs better than the optmal statc allocaton. However, there s stll room for OCBA to mprove ts performance, because we observe that the optmal dynamc allocaton performs much better than OCBA. The paper s organzed as follows: In the next secton, we defne the notaton and the smulaton run allocaton problem for ordnal optmzaton. Secton gves a soluton to a theoretcally optmal allocaton problem. umercal experments are gven n Secton 4. Secton 5 concludes the paper. II. EFFICIET SIMULATIO ALLOCATIO FOR ORDIAL OPTIMIZATIO Suppose we have a complex dscrete-event system. A general smulaton and optmzaton problem wth fnte number of desgns can be defned as mn µ E ξ [L(θ, ξ)] () where θ Θ the search space s an arbtrary, huge, structureless but fnte set; θ s the system desgn parameter vector for desgn, =,,..., k; µ, the performance crteron whch s the expectaton of L, the sample performance, as a functonal of θ, and ξ, a random vector that represents uncertan factors n the systems. ote that for the complex systems consdered n ths paper, L(θ, ξ) s avalable only n the form of a complex calculaton va smulaton. The system constrants are mplctly nvolved n the smulaton process, and so are not shown n (). In smulaton approach, multple smulaton samples/replcatons are taken and then E[L(θ, ξ)] s estmated by the sample mean performance measure: j= L( θ, ξ ), where ξ j represents the j-th sample of ξ and represents the number of smulaton samples for desgn. For notatonal smplcty, defne j L(θ, ξ j ) whch s the j-th sample of the performance measure from desgn. In ths paper, we assume that the smulaton output s ndependent from replcaton to replcaton. The samplng across desgns s also ndependent. Also we assume j s normally dstrbuted. The normalty assumpton s usually not a problem, because typcal smulaton output s obtaned from an average performance or batch mean. Our goal s to select a desgn assocated wth the smallest mean performance measure among k alternatve desgns. Denote by : the sample average of the smulaton output for desgn ; = j= j, S : the sample varance of the smulaton output for desgn, : the varance for desgn,.e., = Var( j ). In practce, s unknown beforehand and so s approxmated by sample varance. b: the desgn wth the smallest sample mean performance; b = arg mn { }. δ b, b -. Wth the above notatons and assumpton, j ~ (µ, ). As ncreases, becomes a better approxmaton to µ n the sense that ts correspondng confdence nterval becomes narrower. The ultmate accuracy of ths estmate cannot mprove faster than /. ote that each sample of j requres one smulaton run. A large number of requred samples of j for all desgns may become very tme consumng. On the other hand, Da (996) shows that an algnment probablty of ordnal comparson can converge to.0 exponentally fast n most cases. Such an algnment probablty s also called the probablty of correct selecton or P{CS}. One example of P{CS} s the probablty that j
desgn b s actually the best desgn. Wth the advantage of such an exponental convergence, nstead of equally smulatng all desgns, Chen et al. (000) further mprove the performance of ordnal optmzaton by determnng the best numbers of smulaton samples for each desgn. Assume that the computaton cost for each run s roughly the same across dfferent desgns. The computaton cost can then be approxmated by + + + k, the total number of samples. We wsh to choose,,, k such that P{CS} s maxmzed, subject to a lmted computng budget T,.e., max P{CS}, L, k s.t. + + + k = T. () However, solvng such an optmal sample allocaton problem s a bg challenge because ) there s no closed-form expresson for P{CS} n general; ) P{CS} s a functon of the means and varances of all desgns whch are unknown; and ) a soluton should be found effcently. Otherwse the beneft of effcent run allocaton wll be lost. III. THEORETICALLY OPTIMAL COMPUTIG BUDGET ALLOCATIO - THREE DESIGS It s dffcult to solve problem (). However, for smaller problems wth some assumptons, t s possble to fnd the optmal soluton. In ths paper, frst we lmt our scope wthn problems havng only three desgns competng for smulaton budget allocaton. Second, we assume we have perfect nformaton untl the pont of determnng smulaton allocaton. amely, we assume we know the means, varances, and all the samples we have obtaned up to the pont. We consder two possble approaches of allocatng computng budget to desgns: statc vs. dynamc. In the statc approach, we solve problem () and determne ts optmal soluton * [ *, *,..., k *]. Then we perform * smulaton samples for desgn for all. For the dynamc approach, nstead of allocatng all of the T smulaton samples at the begnnng, we dynamcally allocate only a small number of smulaton samples at each teraton. In the dynamc approach, the computng budget allocaton s determned teratvely usng the most updated smulaton nformaton. Detals of these two approaches are presented n the followng subsectons. A. Theoretcally Optmal Statc Allocaton (TOSA) Procedure ote that P{CS} s a functon of means, varances, and (,,, k ). When the means and varances for all desgns are known, P{CS} can be calculated (or estmated through Monte Carlo smulaton) f the values of (,,, k ) are gven. Snce the total computng budget, T, consdered n ths paper s not bg, we can evaluate P{CS} for all possble combnatons of (,,, k ) wth a constrant that + + + k = T. Then the maxmum P{CS} and the correspondng (,,, k ) can be determned. For example, the three desgns n the second example gven n Secton 4 are: j ~ (0, 0 ), j ~ (,.5 ), and j ~ (, ). Suppose that the total computng budget T = + + = 0. In ths case, desgn s the best desgn, and P{CS} = Pr{ ( ) < ( ) and ( ) < ( ) } = Pr{ ( ) > 0 } Pr{ ( ) > 0 } = (.5 ) (.0 where ( ) s the cumulatve dstrbuton functon of the standard normal random varable. Snce the varance of desgn s zero, we know that n the theoretcally optmal allocaton, we should not allocate any sample to desgn as t wll not further reduce ts estmaton varance. We should allocate the lmted computng budget to Desgns and only. Thus we can easly evaluate P{CS} for all combnatons wth the constrant + = 0 and fnd the best allocaton. Once the optmal soluton ( *, *, *) s found, the TOSA allocates all the computng budget at a tme and performs * smulaton samples for each desgn accordngly. B. Theoretcally Optmal Dynamc Allocaton (TODA) Procedure In the dynamc approach, we allocate only one addtonal smulaton sample at each teraton. By utlzng the sampled nformaton, we determne the new allocaton dynamcally. Agan, we assume we have perfect nformaton of knowng the means, varances, and all the samples we have taken for all desgns up to the decson pont. For notatonal smplcty, we assume µ < µ < µ n the followng dscusson. Suppose we have conducted,, smulaton samples for the three desgns and obtaned the sample means:,, and. The decson problem s whch desgn we should choose to have one more smulaton sample such that the P{CS} can be maxmzed gven the nformaton we have. Denote a as the decson varable that gves the ndex of desgn whch we wll smulate at the next teraton and y a as the new sample obtaned after we perform ths addtonal smulaton on desgn a. Thus a* = arg max P{CS,,, a} a where a {,, } () When a =, P P{CS,,, a=} n + y = P{ < mn(, )} n + )
( n = +) mn{, } n Smlarly, f we decde to smulate desgn,.e., a =, then P P{CS,,, a=} n + y = P{ < < } n + = I( < ) - ( n + ) + µ where I( ) s an dentty functon. When a =, P P{CS,,, a=} n + y = P{ < < } n + = I( < ) - ( n + ) Thus, problem () s equvalent to a* = arg max P a (4) a To solve the optmal decson problem n (4), we need to consder dfferent cases based on the order of,, and as follows. Case. When s the smallest, ( n P = +) mn{, } n P = - ( n + ) + µ P = - ( n + ) Case. When < <, ( n P = + ) n P = 0 P = - ( n + ). Case. When < <, ( n P = + ) n P = - ( n + ) + µ P = 0 Case 4. When s the largest, ( n +) mn{ P =, P = 0; P = 0. } n ote that s a monotoncally ncreasng functon and (- ) = 0. To maxmze P a, t s equvalent to maxmze Q a, whch s defned as the parameter value n the functon as follows: Case. When s the smallest, Q = Q = ( n n +) mn{, } n - ( n + ) + µ n - ( n + ) Q = Case. When < <, Q = ( n + ) n Q = (- ) n - ( n + ) Q =. Case. When < <, ( n + ) n Q = Q = n - ( n + ) + µ Q = (- ) Case 4. When s the largest, Q = ( n +) mn{, } n Q = (- ); Q = (- ). ow the acton a can be easly determned by maxmzng Q. amely, a* = arg max Q a (5) a In summary, at each teraton, we calculate the most updated sample means,,, and. Then Q a s calculated and the acton a can be determned. We allocate the addtonal computng budget to desgn a. After desgn a s smulated for one more sample, a s updated, and the whole procedure s repeated untl the computng budget T s exhausted. IV. UMERICAL TESTIG AD EVALUATIO O PRACTICAL ALLOCATIO PROCEDURES In ths secton, we test and compare the theoretcally optmal allocaton schemes n Secton wth some practcal allocaton procedures usng three numercal examples. A. Practcal Allocaton Procedures In practce, the means and varances of all desgns are
unknown pror to performng smulatons. Instead of usng the real means and varances, practcal allocaton procedures apply sample means and sample varances obtaned from smulaton samples to determne addtonal smulaton allocaton. Three representatve allocaton procedures n ordnal optmzaton are consdered n ths paper. They are brefly summarzed as follows. ) Equal Allocaton (Equal) Ths has been wdely appled. The smulaton budget s equally allocated to all desgns. ) Proportonal To Varance (PTV) Ths s based on well-known two-stage Rnott procedure (Rnott 978). The dea s to allocate computng budget n a way that s proportonal to the estmated sample varances, S. ) OCBA by Chen el al. (000) Under a Bayesan model, OCBA approxmates P{CS} usng the Bonferron nequalty and offers an asymptotc soluton to ths approxmaton. Whle the run allocaton gven by OCBA s not an optmal allocaton when the smulaton budget s fnte, the numercal testng demonstrates that OCBA s a very effcent approach and can dramatcally reduce smulaton tme. In partcular, OCBA allocates smulaton runs accordng to:, = δ b,, j {,,..., k}, and j b, j j δ b, j (6) k b = b. (7) =, b ote that n OCBA, the allocaton s a functon of the dfferences n sample means and the varances, whch are approxmated by sample varances. B. umercal Testng We also consder both dynamc and two-stage allocaton for both PTV and OCBA. Intally, n 0 smulaton runs for each of k desgns are performed to get some nformaton such as sample mean and varance of each desgn durng the frst stage. Then dfferent allocaton procedures are appled to determne how to allocate the remanng smulaton budget. In the two-stage allocaton, all the remanng budget s allocated at once after the frst-stage smulaton. They are called PTV- and OCBA-. On the other hand, for dynamc allocaton, only an ncremental computng budget,, s allocated at each teraton after the frst stage. As smulaton proceeds, the sample means and sample varances of all desgns are computed from the data already collected up to that stage. The smulaton budget allocaton s determned dynamcally usng the most updated samplng nformaton. The procedure s contnued untl the total budget T s exhausted. We denote them as PTV-D and OCBA-D. The algorthm for PTV-D and OCBA-D s summarzed as follows. A Sequental Algorthm for OCBA or PTV Step 0. Perform n 0 smulaton replcatons for all desgns; l l l l 0; = = L = k = n 0. k l Step. If T, stop. = Step. Increase the computng budget (.e., number of addtonal smulatons) by and compute the l+ l+ l+ new budget allocaton,,, L, k, usng (6) and (7) for OCBA. Step. Perform addtonal max(0, - ) l+ smulatons for desgn, =,..., k. l l +. Go to Step. In the above algorthm, l s the teraton number. As smulaton evolves, desgn b, whch s the desgn wth the smallest sample mean, may change from teraton to teraton, although t wll converge to the optmal desgn as the l goes to nfnty. When b changes, Theorem s drectly appled n step. However, the older desgn b may not be smulated at all n ths teraton n step due to extra allocaton to ths desgn n earler teratons. The P{CS} for each procedure s estmated by countng the number of tmes the procedure successfully fnds the true best desgn out of,000,000 ndependent applcatons, and then dvdng ths number by,000,000. The choce of,000,000 macro replcatons leads to a standard error for the P{CS} estmate of under 0.00 or 0.%. The P{CS} estmate for each procedure wll serve as a measurement of ts effectveness for comparson purposes. In all of the examples, there are three desgn alternatves, the total computng budget s T = + + = 0, and we have set n 0 = 0 and = 5. ) Example. Ths s a specal case where the best desgn has zero varance and the two nferor desgns have the same performance. The three desgn alternatves are: j ~ (0, 0 ), j ~ (, ), and j ~ (, ). Practcally speakng, ths case mples that desgn has no estmaton uncertanty, whle desgn and desgn are extremely close but wth uncertanty. It s obvous that for TOSA, we should not allocate any smulaton budget to desgn, but should equally dvde the budget to desgns and. Thus, = 0 and = = 60. Table I shows the test results usng dfferent allocaton procedures. We see that TODA performs much better than TOSA. Ths shows the beneft of dynamc allocaton. We also see that OCBA performs better than Equal and PTV. However, t s nterestng to observe that OCBA-D l
outperforms TOSA by a very bg margn. Ths once agan demonstrates the beneft of dynamc allocaton. TABLE I PERFORMACE COMPARISO OF DIFFERET SIMULATIO RU ALLOCATIO PROCEDURES I EAMPLE. THE TOTAL COMPUTIG BUDGET IS 0 Procedures TOSA TODA Equal PTV- PTV-D OCBA- OCBA-D P{CS} 0.70 0.964 0.640 0.69 0.7 0.764 0.86 ) Example. Ths s the same as Example, except the varances of the two nferor desgns dffer: j ~ (0, 0 ), j ~ (,.5 ), and j ~ (, ). In ths case, P{CS} = Pr{ ( ) < ( ) and ( ) < ( ) } = Pr{ ( ) > 0 } Pr{ ( ) > 0 } = (.5 ) (.0 Snce the varance of desgn s zero, we should allocate the lmted computng budget to Desgns and only. Thus we can easly evaluate P{CS} for all combnatons wth the constrant + = 0 to fnd the optmal statc allocaton. Table II shows the test results usng dfferent allocaton procedures. TABLE II PERFORMACE COMPARISO OF DIFFERET SIMULATIO RU ALLOCATIO PROCEDURES I EAMPLE. THE TOTAL COMPUTIG BUDGET IS 0 Procedures TOSA TODA Equal PTV- PTV-D OCBA- OCBA-D P{CS} 0.84 0.98 0.764 0.79 0.799 0.85 0.899 ) Example. Ths s a more general case where all three desgns have non-zero varances and dfferent means: j ~ (0, ), j ~ (,.5 ), and j ~ (.5, ). In ths case, we evaluate P{CS} usng Monte Carlo smulaton for all combnatons. Table III shows the test results usng dfferent allocaton procedures. ) optmzaton. By ntellgently allocatng smulaton samples across desgns, we can further dramatcally mprove the effcency of ordnal optmzaton. Under some assumptons, we develop theoretcally optmal smulaton budget allocaton schemes for both verson of statc and dynamc samplng. We show that dynamc smulaton allocaton s more effcent than statc allocaton notwthstandng that n general problems a theoretcally optmal allocaton may not be at hand. Instead some work should be done on the dynamc OCBA algorthm to reduce the smulaton tme. REFERECES [] Chen, H. C., C. H. Chen, L. Da, E. Yücesan, "ew Development of Optmal Computng Budget Allocaton for Dscrete Event Smulaton," Proceedngs of the 997 Wnter Smulaton Conference, 4-4, 997. [] Chen, C. H., J. Ln, E. Yücesan, and S. E. Chck, "Smulaton Budget Allocaton for Further Enhancng the Effcency of Ordnal Optmzaton," Journal of Dscrete Event Dynamc Systems: Theory and Applcatons, Vol. 0, pp. 5-70, 000. [] Chen, C. H., K. Donohue, E. Yücesan, and J. Ln, "Optmal Computng Budget Allocaton for Monte Carlo Smulaton wth Applcaton to Product Desgn," Journal of Smulaton Practce and Theory, Vol., o., pp. 57-74, March 00. [4] Chck, S.E. and K. Inoue, "ew Two-Stage and Sequental Procedures for Selectng the Best Smulated System," Operatons Research, Vol. 49, pp. 7-74, 00. [5] Da, L., "Convergence Propertes of Ordnal Comparson n the Smulaton of Dscrete Event Dynamc Systems," Journal of Optmzaton Theory and Applcatons, Vol. 9, o., pp. 6-88, 996. [6] Ho, Y. C., R. S. Sreenvas, and P. Vakl, "Ordnal Optmzaton of DEDS," Journal of Dscrete Event Dynamc Systems,, #, 6-88, 99. [7] Hseh, B. W., C. H. Chen, and S. C. Chang, "Schedulng Semconductor Wafer Fabrcaton by Usng Ordnal Optmzaton-Based Smulaton," IEEE Transactons on Robotcs and Automaton, Vol. 7, o. 5, pp. 599-608, October 00. [8] Lee, L. H., " A smulaton Study on Samplng and Selectng under Fxed Computng Budget," Proceedngs of 00 Wnter Smulaton Conference, forthcomng, December 00. [9] Patss,. T., C. H. Chen, and M. E. Larson, "SIMD Parallel Dscrete Event Dynamc System Smulaton," IEEE Transactons on Control Systems Technology, Vol. 5, o., pp. 0-4, January, 997. [0] Rnott, Y., "On Two-stage Selecton Procedures and Related Probablty Inequaltes," Communcatons n Statstcs A7, 799-8, 978. [] Tralovc, L. and L. Y. Pao, "Computng Budget Allocaton for Effcent Rankng and Selecton of Varances wth Applcaton to Target Trackng Algorthms," to appear n IEEE Transactons on Automatc Control, 004. TABLE III PERFORMACE COMPARISO OF DIFFERET SIMULATIO RU ALLOCATIO PROCEDURES I EAMPLE. THE TOTAL COMPUTIG BUDGET IS 0 Procedures TOSA TODA Equal PTV- PTV-D OCBA- OCBA-D P{CS} 0.800 0.96 0.77 0.750 0.787 0.778 0.808 V. COCLUSIO Ths paper examnes the effcency ssue of smulaton budget allocaton for ordnal optmzaton, whch has emerged as an effcent technque for smulaton and