Admissibility in Quantitative Graph Games

Transcription

1 Admissibility in Quantitative Graph Games Guillermo A. Pérez joint work with R. Brenguier, J.-F. Raskin, & O. Sankur (slides by O. Sankur) CFV 22/04/16 ULB

2 Reactive Synthesis: real-world example R2D2 s goal: Reach the gate G without collisions uncontrollable G controllable 1 Why is C3P0 uncontrollable? See Star Wars Episode II.

3 Reactive Synthesis: real-world example R2D2 s goal: Reach the gate G without collisions uncontrollable G controllable 1 Why is C3P0 uncontrollable? See Star Wars Episode II.

4 Reactive Synthesis: real-world example R2D2 s goal: Reach the gate G without collisions uncontrollable G controllable R2D2 can, regardless of what C3P0 does, reach G while avoiding collisions. 1 Why is C3P0 uncontrollable? See Star Wars Episode II.

5 Reactive Synthesis: two-player games on graphs Player ve controls squares and dam controls circles. 1 We are rooting for ve. C (collision) 1 Our results actually concern n-players. Brenguier, Pérez, Raskin, Sankur (ULB) Admissibility in Quantitative Graph Games ULB 3 / 23

6 Reactive Synthesis: two-player games on graphs Player ve controls squares and dam controls circles. 1 We are rooting for ve. C (collision) Is ve able to perpetually avoid C? 1 Our results actually concern n-players. Brenguier, Pérez, Raskin, Sankur (ULB) Admissibility in Quantitative Graph Games ULB 3 / 23

7 Reactive Synthesis: two-player games on graphs Player ve controls squares and dam controls circles. 1 We are rooting for ve. C (collision) Is ve able to perpetually avoid C? 1 Our results actually concern n-players. Brenguier, Pérez, Raskin, Sankur (ULB) Admissibility in Quantitative Graph Games ULB 3 / 23

8 Formalizing a bit The set of square vertices is denoted by V. Strategies A strategy σ for ve is a function V V V ; a strategy τ for dam, a function V (V \ V ) V. Brenguier, Pérez, Raskin, Sankur (ULB) Admissibility in Quantitative Graph Games ULB 4 / 23

9 Formalizing a bit The set of square vertices is denoted by V. Strategies A strategy σ for ve is a function V V V ; a strategy τ for dam, a function V (V \ V ) V. Winning strategies A strategy σ for ve is winning for her, w.r.t. to some objective φ, if for all strategies τ for dam, the resulting play π στ satisfies the objective φ. Winning strategies are a very robust solution concept. Brenguier, Pérez, Raskin, Sankur (ULB) Admissibility in Quantitative Graph Games ULB 4 / 23

10 Enter admissible strategies What if ve has no winning strategies? Brenguier, Pérez, Raskin, Sankur (ULB) Admissibility in Quantitative Graph Games ULB 5 / 23

11 Enter admissible strategies What if ve has no winning strategies? We would still like to avoid choosing bad strategies. Brenguier, Pérez, Raskin, Sankur (ULB) Admissibility in Quantitative Graph Games ULB 5 / 23

12 Enter admissible strategies What if ve has no winning strategies? We would still like to avoid choosing bad strategies. Admissible strategies For a player with objective φ, strategy σ is dominated by σ iff: for all strategies τ of the other player, if π στ = φ = π σ τ = φ; and for some strategy τ of the other player, π στ = φ π σ τ = φ. Non-dominated strategies are said to be admissible. Brenguier, Pérez, Raskin, Sankur (ULB) Admissibility in Quantitative Graph Games ULB 5 / 23

13 Enter admissible strategies What if ve has no winning strategies? We would still like to avoid choosing bad strategies. Admissible strategies For a player with objective φ, strategy σ is dominated by σ iff: for all strategies τ of the other player, if π στ = φ = π σ τ = φ; and for some strategy τ of the other player, π στ = φ π σ τ = φ. Non-dominated strategies are said to be admissible. Essentially, dominated strategies are bad since there is a better strategy. Brenguier, Pérez, Raskin, Sankur (ULB) Admissibility in Quantitative Graph Games ULB 5 / 23

14 Admissibility: some examples Even if not a winning strategy, ve should play something which allows her to win. Brenguier, Pérez, Raskin, Sankur (ULB) Admissibility in Quantitative Graph Games ULB 6 / 23

15 Admissibility: some examples Even if not a winning strategy, ve should play something which allows her to win. G dom. by G Brenguier, Pérez, Raskin, Sankur (ULB) Admissibility in Quantitative Graph Games ULB 6 / 23

16 Admissibility: some examples If ve does have a winning strategy σ, then σ is admissible. Brenguier, Pérez, Raskin, Sankur (ULB) Admissibility in Quantitative Graph Games ULB 7 / 23

17 Admissibility: some examples If ve does have a winning strategy σ, then σ is admissible. C Brenguier, Pérez, Raskin, Sankur (ULB) Admissibility in Quantitative Graph Games ULB 7 / 23

18 Admissibility: more motivation Motivation useful to compare strategies Brenguier, Pérez, Raskin, Sankur (ULB) Admissibility in Quantitative Graph Games ULB 8 / 23

19 Admissibility: more motivation Motivation useful to compare strategies (even if the goal of the other player is not known, unlike NE) Brenguier, Pérez, Raskin, Sankur (ULB) Admissibility in Quantitative Graph Games ULB 8 / 23

20 Admissibility: more motivation Motivation useful to compare strategies (even if the goal of the other player is not known, unlike NE) if goals and rationality of the players is common knowledge, we can even iterate Brenguier, Pérez, Raskin, Sankur (ULB) Admissibility in Quantitative Graph Games ULB 8 / 23

21 Admissibility: more motivation Motivation useful to compare strategies (even if the goal of the other player is not known, unlike NE) if goals and rationality of the players is common knowledge, we can even iterate Synthesis specific motivation simplifying the synthesis task: dominated strategies will not be winning, so we can remove them from the start Brenguier, Pérez, Raskin, Sankur (ULB) Admissibility in Quantitative Graph Games ULB 8 / 23

22 Admissibility: more motivation Motivation useful to compare strategies (even if the goal of the other player is not known, unlike NE) if goals and rationality of the players is common knowledge, we can even iterate Synthesis specific motivation simplifying the synthesis task: dominated strategies will not be winning, so we can remove them from the start good candidate for assume-guarantee synthesis Brenguier, Pérez, Raskin, Sankur (ULB) Admissibility in Quantitative Graph Games ULB 8 / 23

23 Admissibility: state-of-the-art For Boolean objectives... Brenguier, Pérez, Raskin, Sankur (ULB) Admissibility in Quantitative Graph Games ULB 9 / 23

24 Admissibility: state-of-the-art For Boolean objectives... Admissible strategies [Berwanger 2007] We might have infinite dominance chains, yet Brenguier, Pérez, Raskin, Sankur (ULB) Admissibility in Quantitative Graph Games ULB 9 / 23

25 Admissibility: state-of-the-art For Boolean objectives... Admissible strategies [Berwanger 2007] We might have infinite dominance chains, yet every non-admissible strategy is dominated by some admissible one Brenguier, Pérez, Raskin, Sankur (ULB) Admissibility in Quantitative Graph Games ULB 9 / 23

26 Admissibility: state-of-the-art For Boolean objectives... Admissible strategies [Berwanger 2007] We might have infinite dominance chains, yet every non-admissible strategy is dominated by some admissible one and iteration does terminate. Brenguier, Pérez, Raskin, Sankur (ULB) Admissibility in Quantitative Graph Games ULB 9 / 23

27 Admissibility: state-of-the-art For Boolean objectives... Admissible strategies [Berwanger 2007] We might have infinite dominance chains, yet every non-admissible strategy is dominated by some admissible one and iteration does terminate. The set of admissible strategies is regular. Brenguier, Pérez, Raskin, Sankur (ULB) Admissibility in Quantitative Graph Games ULB 9 / 23

28 Admissibility: state-of-the-art For Boolean objectives... Admissible strategies [Berwanger 2007] We might have infinite dominance chains, yet every non-admissible strategy is dominated by some admissible one and iteration does terminate. The set of admissible strategies is regular. The complexity of related decision problems has been studied [B,R, & Sassolas 2014]. Brenguier, Pérez, Raskin, Sankur (ULB) Admissibility in Quantitative Graph Games ULB 9 / 23

29 Admissibility: state-of-the-art For Boolean objectives... Admissible strategies [Berwanger 2007] We might have infinite dominance chains, yet every non-admissible strategy is dominated by some admissible one and iteration does terminate. The set of admissible strategies is regular. The complexity of related decision problems has been studied [B,R, & Sassolas 2014]. Assume-admissible synthesis has also been considered [B,R, & Sankur 2015]. Brenguier, Pérez, Raskin, Sankur (ULB) Admissibility in Quantitative Graph Games ULB 9 / 23

30 Admissibility: capturing all admissible strats C 2 G 4 5 ve: Reach G & dam: Avoid vertex C Brenguier, Pérez, Raskin, Sankur (ULB) Admissibility in Quantitative Graph Games ULB 10 / 23

31 Admissibility: capturing all admissible strats C 2 G 4 5 dam s admissible strategies: do not take any dotted edges Brenguier, Pérez, Raskin, Sankur (ULB) Admissibility in Quantitative Graph Games ULB 10 / 23

32 Admissibility: capturing all admissible strats C 2 G 4 5 ve s admissible strategies: do not take dotted edges + infinitely often go to 1 so that dam can help Brenguier, Pérez, Raskin, Sankur (ULB) Admissibility in Quantitative Graph Games ULB 10 / 23

33 Admissibility: capturing all admissible strats C 2 G 4 5 Any pair of admissible strategies conforms to this graph + ve eventually reaching 1. Note: a play resulting from any such pair of strategies will satisfy both objectives. Brenguier, Pérez, Raskin, Sankur (ULB) Admissibility in Quantitative Graph Games ULB 10 / 23

34 Outline Moving to Quantitative Objectives Brenguier, Pérez, Raskin, Sankur (ULB) Admissibility in Quantitative Graph Games ULB 11 / 23

35 Outline 1 Preliminaries Moving to Quantitative Objectives Brenguier, Pérez, Raskin, Sankur (ULB) Admissibility in Quantitative Graph Games ULB 11 / 23

36 Outline 1 Preliminaries Moving to Quantitative Objectives 2 Value-based characterization of adm. strategies Brenguier, Pérez, Raskin, Sankur (ULB) Admissibility in Quantitative Graph Games ULB 11 / 23

37 Outline Moving to Quantitative Objectives 1 Preliminaries 2 Value-based characterization of adm. strategies 3 Existence of adm. strategies Brenguier, Pérez, Raskin, Sankur (ULB) Admissibility in Quantitative Graph Games ULB 11 / 23

38 Outline Moving to Quantitative Objectives 1 Preliminaries 2 Value-based characterization of adm. strategies 3 Existence of adm. strategies 4 LTL(-ish) characterization of outcomes Brenguier, Pérez, Raskin, Sankur (ULB) Admissibility in Quantitative Graph Games ULB 11 / 23

39 Admissibility in Quantitative Games Let us assume now a weighted graph and a payoff function Val( ). We will focus on admissible strategies for ve. Brenguier, Pérez, Raskin, Sankur (ULB) Admissibility in Quantitative Graph Games ULB 12 / 23

40 Admissibility in Quantitative Games Let us assume now a weighted graph and a payoff function Val( ). We will focus on admissible strategies for ve. Given strategies σ, τ, let Val(σ, τ) denote the payoff obtained by ve. Dominance Strategy σ for ve is dominated by σ if for all strategies τ for dam, Val(σ, τ) Val(σ, τ), there is some strategy τ for dam, Val(σ, τ) < Val(σ, τ). 10 s s 5 s s 1 s 2 s 4 Brenguier, Pérez, Raskin, Sankur (ULB) Admissibility in Quantitative Graph Games ULB 12 / 23

41 Admissibility in Quantitative Games Let us assume now a weighted graph and a payoff function Val( ). We will focus on admissible strategies for ve. Given strategies σ, τ, let Val(σ, τ) denote the payoff obtained by ve. Dominance Strategy σ for ve is dominated by σ if for all strategies τ for dam, Val(σ, τ) Val(σ, τ), there is some strategy τ for dam, Val(σ, τ) < Val(σ, τ). 10 s s 5 s s 1 s 2 s 4 Brenguier, Pérez, Raskin, Sankur (ULB) Admissibility in Quantitative Graph Games ULB 12 / 23

42 Admissibility in Quantitative Games Let us assume now a weighted graph and a payoff function Val( ). We will focus on admissible strategies for ve. Given strategies σ, τ, let Val(σ, τ) denote the payoff obtained by ve. Dominance Strategy σ for ve is dominated by σ if for all strategies τ for dam, Val(σ, τ) Val(σ, τ), there is some strategy τ for dam, Val(σ, τ) < Val(σ, τ). 10 s s 5 s s 1 s 2 s 4 Brenguier, Pérez, Raskin, Sankur (ULB) Admissibility in Quantitative Graph Games ULB 12 / 23

43 Admissibility in Quantitative Games Let us assume now a weighted graph and a payoff function Val( ). We will focus on admissible strategies for ve. Given strategies σ, τ, let Val(σ, τ) denote the payoff obtained by ve. Dominance Strategy σ for ve is dominated by σ if for all strategies τ for dam, Val(σ, τ) Val(σ, τ), there is some strategy τ for dam, Val(σ, τ) < Val(σ, τ). 10 s s 5 s s 1 s 2 s 4 Brenguier, Pérez, Raskin, Sankur (ULB) Admissibility in Quantitative Graph Games ULB 12 / 23

44 Admissibility in Quantitative Games Let us assume now a weighted graph and a payoff function Val( ). We will focus on admissible strategies for ve. Given strategies σ, τ, let Val(σ, τ) denote the payoff obtained by ve. Dominance Strategy σ for ve is dominated by σ if for all strategies τ for dam, Val(σ, τ) Val(σ, τ), there is some strategy τ for dam, Val(σ, τ) < Val(σ, τ). 10 s s 5 s s 1 s 2 s 4 Brenguier, Pérez, Raskin, Sankur (ULB) Admissibility in Quantitative Graph Games ULB 12 / 23

45 Values of a Game Values For play prefix h, strategy σ for ve, let aval(h) denote the antagonistic value: aval(h, σ) = inf τ Val(h π h στ ), aval(h) = sup σ inf τ Val(h π h στ ). and cooperative value: cval(h, σ) = sup τ Val(h π h στ ), cval(h) = sup σ,τ Val(h π h στ ). Brenguier, Pérez, Raskin, Sankur (ULB) Admissibility in Quantitative Graph Games ULB 13 / 23

46 Values of a Game Values For play prefix h, strategy σ for ve, let aval(h) denote the antagonistic value: aval(h, σ) = inf τ Val(h π h στ ), aval(h) = sup σ inf τ Val(h π h στ ). and cooperative value: Tool to study dominance. For instance, σ is dominated: cval(h, σ) = sup τ Val(h π h στ ), cval(h) = sup σ,τ Val(h π h στ ). (by σ ) if cval(s 1, σ) < aval(s 1, σ ), or, if σ takes an edge s s with cval(s ) < aval(s) Brenguier, Pérez, Raskin, Sankur (ULB) Admissibility in Quantitative Graph Games ULB 13 / 23

47 Characterization of Admissible Strategies Brenguier, Pérez, Raskin, Sankur (ULB) Admissibility in Quantitative Graph Games ULB 14 / 23

48 Examples Can we represent admissible strategies by removing edges based on aval and cval? as in the Boolean case

49 Examples Can we represent admissible strategies by removing edges based on aval and cval? as in the Boolean case Edges to be removed depend on the prefix: 10 s s 5 s s 1 s 2 s 4? s 7 Idea: after s 1 s 2, remember that the strategy has committed to get more than 5.

50 Examples Are worst-case optimal strategies always admissible? σ worst-case optimal if aval(s, σ) = aval(s) Worst-case optimal strategies are admissible if obtaining more than the antagonistic value implies a risk 10-1 s 2 s 1 1 Both strategies are admissible

51 Examples s s 2 s 1 1 aval(s 1 ) = aval(s 2 ) = aval(s 3 ) = 1.

52 Examples s s 2 s 1 1 aval(s 1 ) = aval(s 2 ) = aval(s 3 ) = 1.

53 Examples s s 2 s 1 1 aval(s 1 ) = aval(s 2 ) = aval(s 3 ) = 1. Not all worst-case optimal strategies are admissible

54 Examples s s 2 s 1 1 aval(s 1 ) = aval(s 2 ) = aval(s 3 ) = 1. Not all worst-case optimal strategies are admissible Here going left safely maximizes the cooperative value : The antagonistic value is still optimal aval(s 1, σ) = 1 The cooperative value is good: cval(s 1, σ) = 10 σ dominated by σ if aval(h, σ) = cval(h, σ) = aval(h, σ ) < cval(h, σ ).

55 One last value of the game Let us define acval(h) at prefix h: acval(h) = sup{cval(h, σ) σ strategy s.t. aval(h, σ) = aval(h)} If acval(h) = aval(h), then all WCO strategies are admissible. Brenguier, Pérez, Raskin, Sankur (ULB) Admissibility in Quantitative Graph Games ULB 16 / 23

56 One last value of the game Let us define acval(h) at prefix h: acval(h) = sup{cval(h, σ) σ strategy s.t. aval(h, σ) = aval(h)} If acval(h) = aval(h), then all WCO strategies are admissible. If acval(h) > aval(h), then a WCO strategy is admissible iff cval(h, σ) > aval(h). Brenguier, Pérez, Raskin, Sankur (ULB) Admissibility in Quantitative Graph Games ULB 16 / 23

57 Characterization of Admissible Strategies Assumption: WCO strategies exist: h, σ, aval(h, σ) = aval(h).

58 Characterization of Admissible Strategies Assumption: WCO strategies exist: h, σ, aval(h, σ) = aval(h). Theorem A strategy σ for ve is admissible iff at all prefixes h ending in V : cval(h, σ) > aval(h) or aval(h, σ) = aval(h) = acval(h)

59 Characterization of Admissible Strategies Assumption: WCO strategies exist: h, σ, aval(h, σ) = aval(h). Theorem A strategy σ for ve is admissible iff at all prefixes h ending in V : cval(h, σ) > aval(h) or aval(h, σ) = aval(h) = acval(h) 10 s s 5 s s 1 s 2 s 4 9 = cval(s 1, σ) > aval(s 1 ) = 5, 9 = cval(s 1 s 2 s 4, σ) > aval(s 1 s 2 s 4 ) = 3.

60 Characterization of Admissible Strategies Assumption: WCO strategies exist: h, σ, aval(h, σ) = aval(h). Theorem A strategy σ for ve is admissible iff at all prefixes h ending in V : cval(h, σ) > aval(h) or aval(h, σ) = aval(h) = acval(h) s s 2 s = cval(s 1, σ) > aval(s 1 ) = 1

61 Characterization of Admissible Strategies Assumption: WCO strategies exist: h, σ, aval(h, σ) = aval(h). Theorem A strategy σ for ve is admissible iff at all prefixes h ending in V : 10 cval(h, σ) > aval(h) or aval(h, σ) = aval(h) = acval(h) -1 s 2 s 1 1 aval(s 1, σ) = acval(s 1 ) = 1.

62 Existence of Admissible Strategies Brenguier, Pérez, Raskin, Sankur (ULB) Admissibility in Quantitative Graph Games ULB 18 / 23

63 Existence of Admissible Strategies (or why the existence of WCO strategies is a valid assumption) Brenguier, Pérez, Raskin, Sankur (ULB) Admissibility in Quantitative Graph Games ULB 18 / 23

64 Existence In general, admissible strategies do not always exist. Brenguier, Pérez, Raskin, Sankur (ULB) Admissibility in Quantitative Graph Games ULB 19 / 23

65 Existence In general, admissible strategies do not always exist. Theorem In a given class of games, if WCO and Co-Op strategies exist, then admissible strategies always exist. Otherwise, some games have no admissible strategies. Cooperatively-optimal (Co-Op) strategy (cval(s 1, σ) = cval(s 1 )) Brenguier, Pérez, Raskin, Sankur (ULB) Admissibility in Quantitative Graph Games ULB 19 / 23

66 Outcomes Compatible with Admissible Strategies Brenguier, Pérez, Raskin, Sankur (ULB) Admissibility in Quantitative Graph Games ULB 20 / 23

67 Admissible Outcomes: Temporal Logic Assumption: Prefix-independent objectives (e.g. mean payoff) Logic Consider LTL, and in addition consider predicates: Val > v which holds at infinite play ρ if Val(ρ) > v. Predicate aval v meaning that aval at current vertex is v Predicate acval v

68 Admissible Outcomes: Temporal Logic Assumption: Prefix-independent objectives (e.g. mean payoff) Logic Consider LTL, and in addition consider predicates: Val > v which holds at infinite play ρ if Val(ρ) > v. Predicate aval v meaning that aval at current vertex is v Predicate acval v Witness vertices: adversary vertices q where some successor q satisfies cval(q ) > v Witness v (q) q, q q, cval(q ) > v. Witnesses that our strategy gave the adversary the opportunity to help us obtain more than v

69 Set of Outcomes under Admissible Strategies Define formulae φ 1 = v avalues (aval v (Val > v F(Witness v ))), φ 2 = v avalues (aval v acval v Val = v G (aval v )). Recall the characterization of admissible strategies: h, last(h) V : cval(h, σ) > aval(h) or aval(h, σ) = aval(h) = acval(h) Brenguier, Pérez, Raskin, Sankur (ULB) Admissibility in Quantitative Graph Games ULB 22 / 23

70 Set of Outcomes under Admissible Strategies Define formulae φ 1 = v avalues (aval v (Val > v F(Witness v ))), φ 2 = v avalues (aval v acval v Val = v G (aval v )). And we set Φ adm = G ( V φ 1 φ 2 ). Theorem A play ρ is compatible with some admissible strategy of ve, iff ρ = Φ adm. Recall the characterization of admissible strategies: h, last(h) V : cval(h, σ) > aval(h) or aval(h, σ) = aval(h) = acval(h) Brenguier, Pérez, Raskin, Sankur (ULB) Admissibility in Quantitative Graph Games ULB 22 / 23

71 Conclusion Value-based characterization of adm. strategies. Brenguier, Pérez, Raskin, Sankur (ULB) Admissibility in Quantitative Graph Games ULB 23 / 23

72 Conclusion Value-based characterization of adm. strategies. Admissible actions depend on prefix (unlike for Boolean objectives). Admissibility is decidable (given a strategy). Brenguier, Pérez, Raskin, Sankur (ULB) Admissibility in Quantitative Graph Games ULB 23 / 23

73 Conclusion Value-based characterization of adm. strategies. Admissible actions depend on prefix (unlike for Boolean objectives). Admissibility is decidable (given a strategy). Existence of adm. strategies (under hypotheses: WCO and Co-Op strategies must exist). Brenguier, Pérez, Raskin, Sankur (ULB) Admissibility in Quantitative Graph Games ULB 23 / 23

74 Conclusion Value-based characterization of adm. strategies. Admissible actions depend on prefix (unlike for Boolean objectives). Admissibility is decidable (given a strategy). Existence of adm. strategies (under hypotheses: WCO and Co-Op strategies must exist). LTL+payoff characterization of the outcomes under adm. strats. Brenguier, Pérez, Raskin, Sankur (ULB) Admissibility in Quantitative Graph Games ULB 23 / 23

75 Conclusion Value-based characterization of adm. strategies. Admissible actions depend on prefix (unlike for Boolean objectives). Admissibility is decidable (given a strategy). Existence of adm. strategies (under hypotheses: WCO and Co-Op strategies must exist). LTL+payoff characterization of the outcomes under adm. strats. allows us to do model checking under admissibility if we have a prefix-indep. objective, and if aval, cval, and acval are computable, then we can model-check the game against an LTL+payoff spec (decidable even for MP). Brenguier, Pérez, Raskin, Sankur (ULB) Admissibility in Quantitative Graph Games ULB 23 / 23

76 Conclusion Value-based characterization of adm. strategies. Admissible actions depend on prefix (unlike for Boolean objectives). Admissibility is decidable (given a strategy). Existence of adm. strategies (under hypotheses: WCO and Co-Op strategies must exist). LTL+payoff characterization of the outcomes under adm. strats. allows us to do model checking under admissibility if we have a prefix-indep. objective, and if aval, cval, and acval are computable, then we can model-check the game against an LTL+payoff spec (decidable even for MP). Thank you for your attention. Brenguier, Pérez, Raskin, Sankur (ULB) Admissibility in Quantitative Graph Games ULB 23 / 23