Game Theory Analyzing Games: From Optimality to Equilibrium Manar Mohaisen Department of EEC Engineering Korea University of Technology and Education (KUT)
Content Optimality Best Response Domination Nash Equilibrium Multiple Nash Equilibria Mixed-strategy Nash Equilibrium Pareto Optimality Social Optimality 2
Optimality Optimal Strategy In single-agent environment, it is a strategy that maximizes the agent s expected payoffs o Environment might be stochastic leading to uncertainty o In case of multi-agent games, where each agent tries to maximize her payoffs, situation gets more complex o In multi-agent settings, the optimal strategy for a given agent is meaningless Solution Concepts o Interesting subsets of the strategy profiles o Examples: Pareto Optimality and Nash Equilibrium 3
Best Response Definition Let s 1 and s 1 be pure strategies for player 1 Let s 2 be a pure strategy for player 2, then o s 1 is best response to s 2 if o s 1 is strict best response to s 2 if Example o A is strict best response of P 1 to P 2 s A o A is strict best response of P 1 to P 2 s B o B is strict best response of P 2 to P 1 s A o A is strict best response of P 2 to P 1 s B u s' s 1 1 2 1 1 2 u ( s, s ) (, ) u s' s 1 1 2 1 1 2 u ( s, s ) (, ) P2: A P2: B P1: A.48,.12.60,.40 P1: B.40,.60.32,.08 What shall we call P1 s A strategy? 4
Domination Domination Let s i and s i be two strategies for player i and S -i be the set of all possible strategy profiles for the other players, then o s i strictly dominates s i if for every s -i S -i, u i (s i, s -i ) > u i (s i, s -i ) s -i = (s 1,, s i-1, s i+1,, s n ) s = (s i, s -i ) Example o For P1: B is a dominant strategy P1 can choose B regardless of P2 s choice o For P2: B is a dominant strategy P2: A P2: B P1: A -1, -1-4, 0 P1: B 0, -4-3, -3 5
Domination Domination Example 2 o A is a dominant strategy for P1, P2 does not have any dominant strategy P2: A P2: B P1: A 5, 5 1, 2 P1: B 2, 0 0, 4 Example 3 o No dominant strategy P2: A P2: B P1: A 5, 5 1, 2 P1: B 2, 0 2, 4 6
Nash Equilibrium In Absence of Strictly Dominant Strategy We need a way to determine what will be likely to happen Example (two firms attracting clients) o Doing business with A, B and C are worth 8, 2, and 2, respectively. o If F1 and F2 approach the same client, it will equally divide its business between the firms. o Client A is large, it will do business with the firms only if both approach A. o F1, a small firm, gets a zero payoff if it approaches a client while F2 approaches a different one. F2: A F2: B F2: C F1: A 4, 4 0, 2 0, 2 F1: B 0, 0 1, 1 0, 2 F1: C 0, 0 0, 2 1, 1 7
Nash Equilibrium F2: A F2: B F2: C F1: A 4, 4 0, 2 0, 2 F1: B 0, 0 1, 1 0, 2 F1: C 0, 0 0, 2 1, 1 Example contd. o Any strictly dominant strategy? NO o Any best responses? YES A is strict best response of F1 to F2 s A A is strict best response of F2 to F1 s A C is strict best response of F2 to F1 s B B is strict best response of F2 to F1 s C B is strict best response of F1 to F2 s B C is strict best response of F1 to F2 s C 8
Nash Equilibrium F2: A F2: B F2: C F1: A 4, 4 0, 2 0, 2 F1: B 0, 0 1, 1 0, 2 F1: C 0, 0 0, 2 1, 1 Example contd. o Any best responses? YES A is best response of F1 to F2 s A A is best response of F2 to F1 s A o The pair of strategies (A, A) is the only best responses to each other Even if there is no dominant strategy, we expect firms to play strategies that are best responses to each other Nash Equilibrium 9
Nash Equilibrium Definition A strategy profile s = (s 1, s 2,, s n ) is a Nash equilibrium if, for all agents i, s i is a best response to s -i. o No player has any incentive to deviate from the best response strategy, leading the concept of equilibrium o Strategies that are not best responses to each other don t result in an equilibrium state: why should a player keep her strategy when she can do better by deviating? o Single Nash equilibrium Players will play strategies in this equilibrium o Moe than one Nash equilibrium Factors, other than payoffs, come into play 10
Nash Equilibrium Theorem Every game with a finite number of players and strategies has at least one Nash equilibrium Strict Nash Equilibrium o strategy profile s = (s 1, s 2,, s n ) is a strict Nash equilibrium, if, for all agents i and all strategies s i s i, u i (s i, s -i ) > u i (s i, s -i ) Stable since each player chooses a strict best response Pure strategy (no probability assigned to actions) Nash equilibrium can be either strict or weak 11
Nash Equilibrium Weak Nash Equilibrium o strategy profile s = (s 1, s 2,, s n ) is a strict Nash equilibrium, if, for all agents i and all strategies s i s i, u i (s i, s -i ) >= u i (s i, s -i ) Not stable because at least one player has a best response that is not in the equilibrium Mixed-strategy Nash equilibriums (to be addressed shortly) are necessarily weak Multiple Nash Equilibria? o Case 1: Coordination Games o Case 2: pure-strategy and mixed-strategy equilibriums 12
Multiple Nash Equilibria Coordination Game Shared goal is to coordinate on same strategy Example o You and your partner are preparing slides for a common project o You can t reach her and you need to start and decide whether to use PowerPoint or Apple s Keynote software to prepare your half P2: P P2: K P1: P 1, 1 0, 0 P1: K 0, 0 1, 1 13
Multiple Nash Equilibria Which equilibrium is more likely to be chosen? o Thomas Schelling proposed the focal point to resolve this difficulty o Focal Point: Natural reasons, possibly outside the payoff structure, cause players to focus on one of the Nash equilibria Social Conventions o They can help people coordinate among multiple equilibria. o Drivers meeting at an intersection, to avoid collision, They both need to turn right in Korea They both need to turn left in Japan 14
Multiple Nash Equilibria Variants of Coordination Games Unbalanced Coordination Game: Battle of the Sexes Game o (P, P) and (K, K) are still Nash equilibria. o It is hard to predict which equilibrium will be played based on the payoff structure and the social conventions. P2: P P2: K P1: P 2, 1 0, 0 P1: K 0, 0 1, 2 15
Multiple Nash Equilibria Variants of Coordination Games Unbalanced Coordination Game: Stag Hunt Game o Two people are out hunting If they work together, they can hunt a stag and share it (high payoff) If each works on his own, each can catch a hare (still good payoff) If one hunter tries to catch a stag on his own, he gets nothing (high penalty) Which equilibrium is more likely to be chosen? P2: Stag P2: Hare P1: Stag 4, 4 0, 3 P1: Hare 3, 0 3, 3 16
Multiple Nash Equilibria Variants of Coordination Games Hawk-Dove Game o Two animals compete to decide how a piece of food to be divided between them o If they behave passively, each gets a payoff of 3 o If one behaves aggressively and the other behaves passively, the aggressive gets 5 and the passive one gets 1 o If both behave aggressively, they destroy the food and they get nothing P2: Dove P2: Hawk P1: Dove 3, 3 1, 5 P1: Hawk 5, 1 0, 0 17
Multiple Nash Equilibria Variants of Coordination Games International Relations o Each country either behave friendly or aggressively o Which equilibrium is more likely to be chosen? P1 Friendly P1 Aggressively P2 Friendly P2 Aggressively 50, 50 0, 40 40, 0 20, 20 18
Mixed-strategy Equilibrium Best Response (again) Player i s best response to the strategy profile s -i is a mixed strategy s* i S i such that u i (s* i, s -i ) > u i (s i, s -i ) for all strategies s i S i In Absence of Nash Equilibrium Agents are competing in an attack-defense game How to predict agents behavior? o Once players are allowed to behave randomly (mixed strategies) there will exist an equilibrium or more. 19
Mixed-strategy Equilibrium Example Matching Pennies Game (zero-sum game) o Direct conflict of interests o No equilibrium: Agents have incentive to chose other strategies that increase their payoffs Therefore, each agent tries to make it difficult for opponent to predict what she will play. This is done via randomization P2: H P2: T P1: H -1, +1 +1, -1 P1: T +1, -1-1, +1 20
Mixed-strategy Equilibrium Example contd. o P1 plays H with probability p ]0, 1[ and T with probability (1-p) such that P2 becomes indifferent about playing H or T. It means that P2 can t take advantage by knowing P1 s choice o Let s compute the payoffs of P2 If P1 chooses a probability p and P2 chooses H, P2 s payoff is» (-1)(p) + (1)(1-p) = 1 2p If P1 chooses a probability p and P2 chooses T, P2 s payoff is» (1)(p) + (-1)(1-p) = -1 + 2p P2: H P2: T P1: H -1, +1 +1, -1 P1: T +1, -1-1, +1 21
Mixed-strategy Equilibrium Example contd. o If 1 2p 2p 1, P2 has a best response to the mixed strategy o Then, 1 2p should be equal to 2p 1 so that P2 does not have any best response to the mixed strategy 1-2p = 2p -1 p = ½ 1 0.5 P2: H P2: T P1: H -1, +1 +1, -1 P1: T +1, -1-1, +1 payoff of P2 0-0.5 P2:H & 1-2p P2:T & 2p - 1-1 0 0.2 0.4 0.6 0.8 1 p 22
Mixed-strategy Equilibrium Example contd. o Let s compute the payoffs of P1 If P2 chooses a probability q and P1 chooses H, P1 s payoff is» (-1)(q) + (1)(1-q) = 1 2q If P2 chooses a probability q and P1 chooses T, P1 s payoff is» (1)(q) + (-1)(1-q) = -1 + 2q As in the case of P2,» 1-2q = 2q-1 q = ½ o The only mixed-strategy Nash equilibrium is achieved when p = q = ½ P2: H P2: T P1: H -1, +1 +1, -1 P1: T +1, -1-1, +1 23
Mixed-strategy Equilibrium Example contd. o The game is symmetric; same holds for P1 1 0.5 P2:H & 1-2p P2:T & 2p - 1 1 0.5 P2:H & 1-2p P2:T & 2p-1 P2's expected payoff payoff of P2 0 P2:H is best response P2:T is best response payoff of P2 0-0.5-0.5-1 0 0.2 0.4 0.6 0.8 1 p If P2 plays pure strategy then -For p > 0.5, P2 s best response is T -For p < 0.5, P2 s best response is H -Why shall P1 offer her opponent to enjoy a best response strategy? -1 0 0.2 0.4 0.6 0.8 1 p At equilibrium (q = p = ½ ) -P2 has a fixed expected payoff regardless of the strategy chosen by P1 and vice-versa equilibrium - If any agent deviates from the equilibrium probability, the other agent will deviate in response to maximize her payoff 24
Mixed-strategy Equilibrium Example: American football Game P2: DP P2: DR o If the defense matches the offense strategy P1: OP 0, 0 10, -10 The offense gains 0 yards. o If the offense runs while the defense defends against the pass the offense gains 5 yards. o If the offense passes while the defense defends against the run The offense gains 10 yards. P1: OR 5, -5 0, 0 o Analysis from P1 s point of view If P2 chooses a probability q and P1 chooses OP, P1 s payoff is» (0)(q) + (10)(1-q) = 10 10q If P2 chooses a probability q and P1 chooses OR, P1 s payoff is» (5)(q) + (0)(1-q) = 5q 25
Mixed-strategy Equilibrium Example: American football Game o Analysis from P2 s point of view If P1 chooses a probability p and P2 chooses DP, P2 s payoff is» (0)(p) + (-5)(1-p) = 5p - 5 If P1 chooses a probability p and P1 chooses DR, P2 s payoff is» (-10)(p) + (0)(1-p) = -10p o Apparently, there is no pure-strategy Nash equilibrium o Mixed-strategy Nash equilibrium 10 10q = 5q q = 2/3 5p 5 = -10p p = 1/3 P2: DP P2: DR P1: OP 0, 0 10, -10 P1: OR 5, -5 0, 0 26
Pareto Optimality global optimization Pareto Domination Strategy profile s Pareto dominates profile s if for all i N, u i (s) u i (s ), and there exists some j N such that u j (s) > u j (s ) o In other words: An outcome s is at least as good as s, and there is an agent who still prefers s to s o In other words: can you make an agent better off without harming the other agent? Pareto Optimality Strategy profile s is Pareto optimal, or strictly Pareto efficient, if there does not exist another strategy profile s S that Pareto dominates s o A game may have one or more Pareto optimality 27
Pareto Optimality Examples o In zero-sum games, all strategy profiles are Pareto efficient P2: A P2: B P1: A 2, 1 0, 0 P1: B 0, 0 1, 2 P2: L P2: R P1: L 1, 1 0, 0 P1: R 0, 0 1, 1 P2: H P2: T P1: H 1, -1-1, 1 P1: T -1, 1 1, -1 P2: H P2: T P1: H -1, -1-4, 0 P1: T 0, -4-3, -3 28
Social Optimality - global optimization Definition A choice of strategies one by each player is a social welfare maximizer (or socially optimal) if it maximizes the sum of players payoffs. o We should be careful that sometimes we can t combine the payoffs to obtain a meaningful social optimality. P2: A P2: B P1: A 11, 6 10, 8 P1: B 7, 7 30, 1 29
Summary Optimality Best Response Domination Nash Equilibrium Multiple Nash Equilibria Mixed-strategy Nash Equilibrium Pareto Optimality Social Optimality 30