Continuous Optimal Timing

Transcription

1 Srlnd University Computer Science, Srbrücken, Germny My 6, 205

2 Outline Motivtion Preliminries Existing Algorithms Our Algorithm Empiricl Evlution Conclusion

3 Motivtion Probbilistic models unrelible/unpredictble system behviour: rndomized lgorithms: messge loss, component filure,... the probbility of reching consensus in leder election lgorithms is lmost

4 Motivtion Models we work with: run in continuous time comprise non-deterministic nd probbilistic behviour re good for: optimiztion over multiple vilble choices finding worst cse results properties: Is the mximl probbility of reching filure stte within n hour < 0.0?

5 Motivtion Model checking boils down to time-bounded rechbility problem: Wht is the mximl/miniml probbility to rech given set of sttes within given time bound? Severl lgorithms to tckle this problem re known they re polynomil, but still slow on industril size benchmrks there is no proper comprison between ll of them no one hs clue which lgorithm will be fster on specific benchmrk

6 Outline Motivtion Preliminries Existing Algorithms Our Algorithm Empiricl Evlution Conclusion

7 CTMDPs hve some money wste 0 risky broke relible gmble 99 do PhD rich 0.000

8 CTMDPs Continuous Time Mrkov Decision Process (CTMDP) is tuple C = (S, Act, R), where S - set of sttes Act - set of ctions R : S Act S R 0 rte function hve some money risky broke wste 0 relible gmble 99 do PhD rich 0.000

9 CTMDPs Continuous Time Mrkov Decision Process (CTMDP) is tuple C = (S, Act, R), where S - set of sttes Act - set of ctions R : S Act S R 0 rte function hve some money risky broke wste 0 relible Exit Rte E(s, α) = R(s, α, s ) s S CTMDP is Uniform if exit rtes over ll sttes nd ll vilble ctions re the sme gmble 99 rich do PhD 0.000

10 Resolution of Non-Determinism. Schedulers. Wht is the probbility of becoming rech before I die? hve some money wste 0 risky broke relible gmble 99 do PhD rich 0.000

11 Resolution of Non-Determinism. Schedulers. Wht is the probbility of becoming rech before I die? hve some money wste The nswer depends on chosen ctions risky 0 broke relible gmble 99 do PhD rich 0.000

12 Resolution of Non-Determinism. Schedulers. Wht is the probbility of becoming rech before I die? hve some money wste The nswer depends on chosen ctions A Scheduler σ (or controller, policy): σ : History Act Clsses of schedulers: Timed/Untimed - knowledge of time pssed (Tim/Unt) Erly/Lte - decision is fixed on entering stte/mybe chnged t ny time lter risky gmble 99 broke rich 0 relible do PhD 0.000

13 Rechbility Problem hve some money wste Wht is the mximl/miniml probbility to rech given set of sttes within given time? vl (s) := {l, e} sup Pr s [ σ T G ] σ Tim risky gmble 99 0 broke relible do PhD rich 0.000

14 Outline Motivtion Preliminries Existing Algorithms Our Algorithm Empiricl Evlution Conclusion

15 Existing Algorithms Erly Lte Exponentil Approximtion ExpStep- (by M. Neuheussr, L. Zhng) Improved Exponentil Approximtion ExpStep-k (by H. Htefi, H. Hermnns) Polynomil Approximtion PolyStep-k (by J. Fernley, M. Rbe, et l.) Adptive Step Approximtion AdptStep (by P. Buchholz, I. Schulz) All existing pproches use discretiztion

16 Outline Motivtion Preliminries Existing Algorithms Our Algorithm Empiricl Evlution Conclusion

17 Our Approch Fetures: Does NOT discretize the time horizon, insted pproximte vi different clss of schedulers: Less powerfull Untimed - for lower bound More powerfull Prophetic - for upper bound

18 Our Algorithm (Unif + ) input : CTMDP C = (S, Act, R), gol sttes G S, horizon T R >0, scheduler clss {l, e}, nd pproximtion error ε > 0 prms: trunction error rtio κ (0, ) output : vector v such tht v vl ε λ mximl exit rte E mx in C 2 repet 3 Cλ -uniformistion of C to the rte λ 4 v pproximtion of the lower bound vl for Cλ up to error ε κ 5 v pproximtion of the upper bound vl for Cλ up to error ε κ 6 λ 2 λ 7 until v v ε ( κ) 8 return v

19 Our Algorithm (Unif + ) input : CTMDP C = (S, Act, R), gol sttes G S, horizon T R >0, scheduler clss {l, e}, nd pproximtion error ε > 0 prms: trunction error rtio κ (0, ) output : vector v such tht v vl ε λ mximl exit rte E mx in C 2 repet 3 Cλ -uniformistion of C to the rte λ 4 v pproximtion of the lower bound vl for Cλ up to error ε κ 5 v pproximtion of the upper bound vl for Cλ up to error ε κ 6 λ 2 λ 7 until v v ε ( κ) 8 return v

20 Unif +. Uniformiztion Uniformize to the rte 4.5: originl lte erly s.5 s.5 s s 0 s 0 s 0.5 s 0, 2 b 2 s 2 b 2 s 2 b 2 s 2

21 Our Algorithm (Unif + ) input : CTMDP C = (S, Act, R), gol sttes G S, horizon T R >0, scheduler clss {l, e}, nd pproximtion error ε > 0 prms: trunction error rtio κ (0, ) output : vector v such tht v vl ε λ mximl exit rte E mx in C 2 repet 3 Cλ -uniformistion of C to the rte λ 4 v pproximtion of the lower bound for Cλ up to error ε κ 5 v pproximtion of the upper bound for Cλ up to error ε κ 6 λ 2 λ 7 until v v ε ( κ) 8 return v

22 Unif +. Bounds Lower Bound vl(s) := sup σ Unt i=0 Pr C λ,s σ [ ] T =i G Optiml rechbility probbility over untimed schedulers Upper Bound vl(s) := i=0 sup σ Unt Pr C λ,s σ [ ] T =i G Optiml rechbility probbility over prophetic schedulers

23 Our Algorithm (Unif + ) input : CTMDP C = (S, Act, R), gol sttes G S, horizon T R >0, scheduler clss {l, e}, nd pproximtion error ε > 0 prms: trunction error rtio κ (0, ) output : vector v such tht v vl ε λ mximl exit rte E mx in C 2 repet 3 Cλ -uniformistion of C to the rte λ 4 v pproximtion of the lower bound vl for Cλ up to error ε κ 5 v pproximtion of the upper bound vl for Cλ up to error ε κ 6 λ 2 λ 7 until v v ε ( κ) 8 return v

24 Outline Motivtion Preliminries Existing Algorithms Our Algorithm Empiricl Evlution Conclusion

25 Empiricl Evlution nd Comprison mx. S mx. rnge of mx. exit rtes best in erly (# of cses) best in lte (# of cses) PS: ,6 29,6 u + (32) u + (47) QS: ,5 44,9 u + (32) ps-3(8), u + (7), s (5) DPMS: , 9, u + (3), es-2(3), n/() s (24), u + (4), ps-3(6) GFS: u + (40) s (23), u + () FTWC: ,02 u + (25) u + (32) SJS: u + (57), es-2(2) u + (70), s (29) ES: u + (23), es-2(4), n/() u + (28), ps-3(2) Tble: Overview of experiments summrizing which lgorithm performed best how mny times; n/ indictes tht no lgorithm completed within 5 minutes.

26 Outline Motivtion Preliminries Existing Algorithms Our Algorithm Empiricl Evlution Conclusion

27 Conclusion Unif + performs very well for erly scheduling problems Unif + is competitive on lte scheduling problems Results on lte scheduling re inconclusive. Further insight into the problem is required The benefits of Unif + : it is esily switchble between erly/lte schedulers simplified version of Unif + with only itertion is very fst nd my give good posteriori error bounds

28 The End