LECTURE 4: MIXED STRATEGIES (CONT D), BIMATRIX GAMES

Transcription

1 LECTURE 4: MIXED STRATEGIES (CONT D), BIMATRIX GAMES

2 Mixed Strategies in Matrix Gaes (revision) 2 ixed strategy: the player decides about the probabilities of the alternative strategies (su of the probabilities = 1); when the decisive oent coes, he/she akes a rando selection of the strategy with the stated probabilities notation: ixed strategies = colun vectors x and y, ith eleent is the probability of ith row/colun of atrix A being picked: x x1 y1 x2 y2, y. x yn payoffs becoe rando variables; decisions use expected payoffs: n EZ x a y x Ay EZ 1 i ij j 2 i1j1

3 Mixed-Strategy NE (revision) 3 atheatical definition: NE is a cobination of (ixed) strategies x* and y* with the property that x Ay* x* Ay* x* Ay for all ixed strategies x and y. value of the gae (v): player 1 s expected payoff at NE ( x* Ay* ) Basic Theore on Matrix Gaes: for any atrix A there exists a ixed-strategy NE. finding ixed-strategy NE s: graphical solution (2 n and 2 atrices only) linear prograing (general n case)

4 Finding NE Linear Prograing (revision) 4 Step 1: If there is a negative eleent in the payoff atrix, ake all eleents of the atrix positive by adding the sae positive nuber to all eleents of the atrix. (This does changes the gae, but only into a strategically equivalent one.) Step 2: Solve the linear prograing proble axiize p 1 + p p n subject to a 11 p 1 + a 12 p a 1n p n 1, a 21 p 1 + a 22 p a 2n p n 1, a 1 p 1 + a 2 p a n p n 1, p i 0, i = 1,,n. Step 3: Divide the prial and dual solutions by the optial value of the objective function: the prial solution deterines the strategy of player 2. the dual solution deterines the strategy of player 1.

5 Finding NE Linear Prograing (cont d) 5 note: if we use the sybol 1 n to denote vector 1 n 1 1 n 1 eleents we can siplify the LP proble fro step 2 as axiize subject to z 1 n Ap 1 p 0. p,

6 Proof of the Basic Theore on Matrix Gaes 6 Step 1: show that without loss of generality, we can assue that the eleents of atrix A are all positive. Step 2: show that the following conditions for NE existence are equal (i.e., find sipler, but equal versions of NE conditions): (C1 ) There exist x* and y* such that for all ixed strategies x and y. x Ay* x* Ay* x* Ay (C2 ) There exist x*, y* and v such that for all ixed strategies x and y. x Ay* v x* Ay (C3 ) There exist x*, y* and v such that for all pure strategies x and y. x Ay* v x* Ay Steps 3 and 4: prove the existence of x*, y* and v satisfying (C3 ) using the linear prograing Duality Theore.

7 Proof of the Basic Theore (cont d) 7 Step 1: WLOG, all eleents of the payoff atrix A can be assued to be positive. if there s a negative eleent, we can turn the gae into a strategically equivalent one with positive eleents by adding a sufficiently large constant c to all eleents of A (thus obtaining atrix A) atheatically: A A ce, where E it s easy to see that x A y x A y c (no atter what the strategies are, player 1 gets an extra payoff of c), hence x Ay* x* Ay* x* Ay x Ay* + c x* Ay* + c x* Ay c, so if we find a ixed-strategy NE for for A A, it is also a ixed-strategy NE

8 Proof of the Basic Theore (cont d) 8 Step 2: the following conditions of NE existence are equal (x* and y* denote ixed strategies of the two players, v is a real nuber): (C1 ) There exist x* and y* such that for all ixed strategies x and y. (C2 ) There exist x*, y* and v such that for all ixed strategies x and y. (C3 ) There exist x*, y* and v such that for all pure strategies x and y. first, we prove ( C1 ) ( C2) : if (C1) holds, then (C2 ) holds as well with ( C1 ) ( C2) : because the inequality in (C2 ) holds for all ixed strategies x and y, it has to hold for x = x*, y = y* as an instance, which yields: x* Ay* v x* Ay* v x* Ay* next, we prove ( C2) ( C3 ). Here, ( C2) ( C3 ) is obvious, as pure strategies are only a special case of ixed strategies. ( C1 ) ( C2) x Ay* x* Ay* x* Ay x Ay* v x* Ay x Ay* v x* Ay v x* Ay*

9 Proof of the Basic Theore (cont d) 9 the ost difficult part in step 2 is proving ( C2) ( C3) the arguent goes as follows: Mixed strategies are convex cobinations of pure strategies; therefore, the expected payoff for a ixed strategy is a convex cobination of the expected payoff for pure strategies. Thus, if x Ay* v holds for all pure strategies x, it has to hold for all ixed strategies as well. Convex cobination of n vectors: Let v 1,v 2,,v n be n vectors of equal size. A convex cobination of these vectors is a vector c1v 1 c2v2 c n v n, where c 1,c 2,,c n are real nubers between 0 and 1, the su of which is 1. iagine player 1 has 3 alternative actions (Top, Middle, and Botto row) ixed strategies are in the for x = (x 1, x 2, x 3 ) T, which can be expressed as a convex cobination of pure strategies: x = x 1 (1,0,0) T + x 2 (0,1,0) T + x 3 (0,0,1) T

10 Proof of the Basic Theore (cont d) Graphical illustration of a ixed-strategy space of player 1: x 3 Strategy space of player 1: x = (x 1,x 2,x 3 ) T such that x 1 + x 2 + x 3 = 1. 1 B: (0,0,1) Convex cobination of T, M and B 1 M: (0,1,0) 0 (x 1, x 2, x 3 ) 1 T: (1,0,0) x 2 x 1

11 Proof of the Basic Theore (cont d) including expected payoff in the plot: x 3 = 1 x 1 x 2, so x 3 needn t be plotted; we plot expected payoff instead: EZ 1 = (x 1, x 2, x 3 )A y* linear function of (x 1, x 2, x 3 ) EZ 1 Expected payoffs: height = EZ 1 EZ 1 (B ) Ay* 3 1 EZ 1 (M ) 1 M 0 B T 1 EZ 1 (T ) x x 1 2 Strategy space

12 Proof of the Basic Theore (cont d) the expected payoff for a ixed strategy (x 1, x 2, x 3 ) T is a convex cobination of the expected payoff for the pure strategies T, M, and B: EZ 1 EZ 1 (B ) EZ 1 (x 1, x 2, x 3 ) = x 1 EZ 1 (T ) + x 2 EZ 1 (M ) + x 3 EZ 1 (B ) EZ 1 (T ) EZ 1 (M ) 1 M 0 B T 1 (x 1, x 2, x 3 ) T = x 1 (1,0,0) T + x 2 (0,1,0) T + x 3 (0,0,1) T x 2 x 1

13 Proof of the Basic Theore (cont d) x Ay* v therefore, in order to show (the height of the whole of the upper triangle is below the level v), it s enough to show it for the three pure strategies (vertices of the triangle) height = v v EZ 1 (B ) v v EZ 1 (T ) EZ 1 (M ) 1 M 0 B T 1 x 2 x 1

14 Proof of the Basic Theore (cont d) 14 trying to find NE strategy for player 2 we re looking for y* = (y 1,,y n ) T such that x A y* v for all ixed strategies x fro the previous discussion, it suffices for the inequality to hold for all pure strategies x

15 Proof of the Basic Theore (cont d) 15 trying to find NE strategy for player 2 we re looking for y* = (y 1,,y n ) T such that x A y* v for all ixed strategies x fro the previous discussion, it suffices for the inequality to hold for all pure strategies x algebraically: we re looking for y* = (y 1,,y n ) T such that (1,0,0,,0) Ay* (0,1,0,,0) Ay* (0,0,0,,1) Ay* v v v a11y1 a12y2 a1 nyn v, v a 21y1 a22y2 a2nyn v, * v Ay v 1 a1 y1 a2 y2 an yn v v and, of course, y* is a ixed strategy: y 1 + y y n = 1 and 0 y i 1.

16 Proof of the Basic Theore (cont d) 16 trying to find NE strategy for player 2 we re looking for y* = (y 1,,y n ) T such that x A y* v for all ixed strategies x fro the previous discussion, it suffices for the inequality to hold for all pure strategies x algebraically: we re looking for y* = (y 1,,y n ) T such that (1,0,0,,0) Ay* (0,1,0,,0) Ay* (0,0,0,,1) Ay* v v v a11y1 a12y2 a1 nyn v, v a 21y1 a22y2 a2nyn v, * v Ay v 1 a1 y1 a2 y2 an yn v v and, of course, y* is a ixed strategy: y 1 + y y n = 1 and 0 y i 1. a siilar approach can be used while looking for NE strategy of player 1 for player 1, we use the inequality: v x* Ay y A x* x and y swapped, A transposed, instead of (see the next step)

17 Proof of the Basic Theore (cont d) 17 Step 3: if a cobination of v, x* = (x 1,,x ) T, and y*= (y 1,,y n ) T satisfies a y a y a y v, a x a x a x v, n n a y a y a y v, a x a x a x v, n n a y a y a y v, a x a x a x v, n n 1n 1 2n 2 n y, y, y 0, x, x, x 0, 1 2 n 1 2 y y y 1, x x x 1, 1 2 n 1 2 or, in brief, A y* v1, A x* v1, y* 0, x* 0, 1 y* 1, 1 x* 1, n n then x* and y* are the NE strategies.

18 Proof of the Basic Theore (cont d) 18 Step 4: there exist x* and y* that satisfy the conditions fro step 3 (and, therefore, are the NE strategies). this is the crucial part of the proof; it uses the linear prograing Duality Theore Prial and Dual LP probles: Prial proble : axiize z c x Dual proble: iniize f b y subject to subject to Ax b, A y c, x 0. y 0. Duality Theore: If both the prial and the dual proble have feasible solutions (i.e., solutions that satisfy the constraints), both have optial solutions as well, and the optial objective values are equal (f * = z*)

19 Proof of the Basic Theore (cont d) 19 we gradually turn the conditions fro step 4 into a prial and dual LP proble; first, divide all the inequalities and equations by v and substitute p i = y i / v and q j = x j / v : a p a p a p 1, a q a q a q 1, n n a p a p a p 1, a q a q a q 1, n n a p a p a p 1 a q a q a q 1, n n 1n 1 2n 2 n p, p, p 0, q, q, q 0, 1 2 n 1 2 p p p 1 / v, q q q 1 / v, 1 2 n 1 2 or, in brief, A p 1, A q 1, p0, q 0, 1 p 1 / v, 1 q 1 / v. n n

20 Proof of the Basic Theore (cont d) 20 now we split the conditions for p and q into two linear prograing probles, taking the LHS of the last row as the objectives: axiize z subject to 1 p iniize f 1 n subject to A p 1, A q 1, p 0. q 0. q n

21 Proof of the Basic Theore (cont d) 21 now we split the conditions for p and q into two linear prograing probles, taking the LHS of the last row as the objectives: axiize z subject to 1 p iniize f 1 n subject to A p 1, A q 1, p 0. q 0. q n the two LP probles are in the prial-dual relationship (with b = 1 an c = 1 n ) both have feasible solutions (take p = 0 and q with sufficiently large eleents)

22 Proof of the Basic Theore (cont d) 22 now we split the conditions for p and q into two linear prograing probles, taking the LHS of the last row as the objectives: axiize z subject to 1 p iniize f 1 n subject to A p 1, A q 1, p 0. q 0. q n the two LP probles are in the prial-dual relationship (with b = 1 an c = 1 n ) both have feasible solutions (take p = 0 and q with sufficiently large eleents) therefore, according to the Duality Theore, both have optial solutions ( p* and q* ) with equal objective values ( f * = z* = 1 / v ). it s easy to check that then x* = v q* and y* = v p* satisfy the conditions fro step 4, and thus are the NE strategies

23 Proof of the Basic Theore (cont d) 23 Step 1: WLOG, all eleents of the payoff atrix A can be assued to be positive. Step 2: the following conditions of NE existence are equal : (C1) x Ay* x* Ay* x* Ay for all ixed strategies x and y. (C2 ) x Ay* v x* Ay for all ixed strategies x and y. (C3 ) x Ay* v x* Ay for all pure strategies x and y. Step 3: if a cobination of v, x = (x 1,,x ) T, and y = (y 1,,y n ) T satisfies then x and y are the NE strategies. Step 4: there exist x and y that satisfy the conditions fro step 4 (and, therefore, are the NE strategies). A y v1, A x v1, 1 y 1, 1 x 1, n y 0, x 0, n

24 Finding NE Linear Prograing (revision) 24 Step 1: If there is a negative eleent in the payoff atrix, ake all eleents of the atrix positive by adding the sae positive nuber to all eleents of the atrix. (This does changes the gae, but only into a strategically equivalent one.) Step 2: Solve the linear prograing proble axiize p 1 + p p n subject to a 11 p 1 + a 12 p a 1n p n 1, a 21 p 1 + a 22 p a 2n p n 1, a 1 p 1 + a 2 p a n p n 1, p i 0, i = 1,,n. Step 3: Divide the prial and dual solutions by the optial value of the objective function: the prial solution deterines the strategy of player 2. the dual solution deterines the strategy of player 1.

25 25 Biatrix Gaes = non-constant-su gaes in noral for: a finite set of agents: {1,2} strategy spaces (finite): { X,Y} strategy profile: (x,y) payoff functions: Z 1 (x,y), Z 2 (x,y) payoffs written in two atrices, typically denoted by A = (a ij ) and B = (b ij ) a ij = the payoff of player 1 for strategy profile (i,j) (i.e., player 1 picks i th row and player 2 picks j th colun) b ij = the payoff of player 2 for strategy profile (i,j) typically, A and B written down in a single atrix with double entries: A ;5 2;6,, ;. 3 4 B 7 8 A B 3;7 4;8

26 26 Prisoner s Dilea (again ) PD is a biatrix gae with atrices A , B Player 2 1 \ 2 Stay silent Betray Player 1 Stay silent -1 ; ; 0 Betray 0 ; ; -5

27 27 NE s in Non-Constant-Su Gaes the sae Nash-Equilibriu concept as in case of atrix gaes (one can t be better off when he/she alone deviates fro NE) atheatical definition (for pure strategies): A strategy profile (x*, y*) with the property that Z ( x, y*) Z ( x*, y*), 1 1 Z ( x*, y) Z ( x*, y*) 2 2 for all x X and y Y is a NE.

28 28 NE s in Non-Constant-Su Gaes the sae Nash-Equilibriu concept as in case of atrix gaes (one can t be better off when he/she alone deviates fro NE) atheatical definition (for pure strategies): A strategy profile (x*, y*) with the property that Z ( x, y*) Z ( x*, y*), 1 1 Z ( x*, y) Z ( x*, y*) 2 2 for all x X and y Y is a NE. finding a NE using the best-response approach: player 1 plays her best response to the colun selected by player 2 NE has to be the axiu in the colun in atrix A player 2 plays her best response to the row selected by player 1 NE has to be the axiu in the row in atrix B

29 Prisoner s Dilea (yet again ) 29 player 1 s best response: if player 2 stays silent, player 1 s best response is to betray. Circle (B,S). if player 2 betrays, player 1 s best response is to betray as well. Circle (B,B). Player 2 1 \ 2 Stay silent Betray Player 1 Stay silent -1 ; ; 0 Betray 0 ; ; -5

30 Prisoner s Dilea (yet again ) 30 player 2 s best response: if player 1 stays silent, player 2 s best response is to betray. Square (S,B). if player 1 betrays, player 2 s best response is to betray as well. Square (B,B). Player 2 1 \ 2 Stay silent Betray Player 1 Stay silent -1 ; ; 0 Betray 0 ; ; -5

31 Prisoner s Dilea (yet again ) 31 (B,B) is the unique NE not Pareto efficient ((S,S) better for both players) for both players, strategy S is strictly doinated by strategy B Player 2 1 \ 2 Stay silent Betray Player 1 Stay silent -1 ; ; 0 Betray 0 ; ; -5

32 Mixed-Strategy NE s in Biatrix Gaes 32 atheatical definition: NE is a cobination of (ixed) strategies x* and y* with the property that x A y* x* A y*, x* B y x* B y* for all ixed strategies x and y. Inequalities explained: x Ay* x* Ay* x* By x* By* can be written as: E Z 1 (x, y*) E Z 1 (x*, y*), which eans: If player 1 deviates fro NE his/her expected payoff will not increase can be written as: E Z 2 (x*, y) E Z 2 (x*, y*), which eans: If player 2 deviates fro NE his/her expected payoff will not increase

33 33 Exercise 1: Battle of the Sexes (again) find pure-strategy NE s in the Battle of the Sexes gae: Boy Girl \ Boy Football Shopping Girl Football 2 ; 3 0 ; 0 Shopping 1 ; 1 3 ; 2

34 34 Exercise 1: Battle of the Sexes (again) find pure-strategy NE s in the Battle of the Sexes gae: Boy Girl \ Boy Football Shopping Girl Football 2 ; 3 0 ; 0 Shopping 1 ; 1 3 ; 2 pure-strategy NE s are: (F,F) and (S,S) (note: different payoffs!) in addition, there s one ixed strategy equilibriu: 1 / 4 3 / 4 x*, y* 3 / 4 1 / 4

35 35 Exercise 1: Battle of the Sexes (cont d) assue there s a ixed solution with all eleents positive (i.e., x 1, x 2, y 1, y 2 > 0) if the girl best-responds with a ixed strategy, the boy ust ake her indifferent between F and S with his ixed strategy (why?) therefore: EZ 1 (F,y) = 2 y (1 y 1 ) = 1 y (1 y 1 ) = EZ 1 (S,y), and y 1 = 3/4 siilarly, EZ 2 (x,f ) = 3 x (1 x 1 ) = 0 x (1 x 1 ) = EZ 2 (x,s ), and x 1 = 1/4 Girl \ Boy Football Shopping Football 2 ; 3 0 ; 0 Shopping 1 ; 1 3 ; 2 x 1 1 x 1 y 1 1 y 1

36 Exercise 2: Gae of Chicken 36 find NE s in the gae of Chicken: two drivers drive towards each other on a collision course either at least one swerves, or both ay die in the crash whoever swerves is called a chicken (a coward) Player 2 1 \ 2 Swerve Straight Player 1 Swerve 0 ; 0-1 ; 1 Straight 1 ; ; -10

37 Exercise 3: Doinated NE s 37 find pure-strategy NE s in the following gae which of the two NE s would you choose if you were player 1? which of the two NE s would you choose if you were player 2? Player 2 1 \ 2 L R Player 1 T 7 ; 9-2 ; -1 B -2 ; 0 6 ; 4 the NE (B,R) is doinated by NE (T,L) (T,L) is strategically ore credible

38 Exercise 4: Only Mixed NE s 38 find out if there are any pure-strategy NE s in the following biatrix gae if not, find a ixed-strategy NE the way we used for the Battle of Sexes Player 2 1 \ 2 L R Player 1 T 3 ; 5 2 ; -1 B 4 ; 2-2 ; 5

39 NE Existence in Biatrix Gaes 39 Nash Existence Theore (John Nash, 1950): Every noral-for gae with finite strategy spaces has a ixed-strategy NE. possible scenarios for biatrix gaes: unique NE in pure strategies (prisoner s dilea) ultiple NE s (pure and ixed), no doination (BoS) ultiple NE s (pure and ixed) with doination (Ex. 3) no pure NE s, (ixed NE s only) (Ex. 3) note: apart fro the 2 2 case, ixed NE s are generally difficult to find (non-linear prograing techniques)

40 Doinated strategies in Biatrix Gaes 40 as in atrix gaes, doinated strategies can be eliinated to siplify the proble however, it s only safe to eliinate strictly doinated strategies (as opposed to only weakly doinated ones) Exaple 1: prisoner s dilea (yes, indeed, yet again ) strategy Stay silent is strictly doinated for both players it doesn t atter whether we start eliinating rows or coluns, we always end up with the unique NE: Player 2 1 \ 2 Stay silent Betray Player 1 Stay silent -1 ; ; 0 Betray 0 ; ; -5

41 Doinated strategies in Biatrix Gaes (cont d) 41 Exaple 2: M weakly doinates T and B two different eliination processes: 1 eliinates T, 2 eliinates L (2;1) 1 eliinates B, 2 eliinates R (1;1) Player 2 1 \ 2 L R T 1 ; 1 0 ; 0 Player 1 M 1 ; 1 2 ; 1 B 0 ; 0 2 ; 1

42 Cooperation in Biatrix Gaes 42 so far, we assued the players do not cooperate note: in atrix gaes, no cooperation is possible (why?) with cooperation, NE is not the relevant principle anyore; still, it can be used in the decision-aking process as a certain bargaining tool (or as a benchark describing the case the players fail to agree on cooperation, see below) two different cooperation settings: cooperation with transferable payoffs cooperation with non-transferable payoffs in both cases, players cooperate only of it pays for both; i.e., both earn ore than in the non-cooperative setting what is the non-cooperative payoff? 1. the NE payoff (if this can be decided) 2. the guaranteed payoff (bully-proof)

43 Guaranteed Payoff: An Exaple 43 Player 2 1 \ 2 L M R T 3 ; 1 9 ; ; 2 Player 1 C -9 ; 9-5 ; ; -2 B -10 ; 9 13 ; 4 5 ; 4 the guaranteed payoff for a strategy is the worst possible result: for player 1, the worst-case scenarios for the individual strategies are T: 3, C: 9, B: 10 guaranteed payoff of player 1 = 3 for player 2, we have L: 1, M: 10, R: 2 guaranteed payoff = 1 guaranteed payoff of player 1/2 is the largest row/colun iniu note that there s no NE in pure strategies here

44 Guaranteed Payoff: Another Exaple 44 Player 2 1 \ 2 L M R T 3 ; 1 9 ; ; 2 Player 1 C -9 ; 9-5 ; 35 2 ; -2 B -10 ; 9 13 ; 4 5 ; 4 when deciding about the guaranteed payoffs, one can leave out strictly doinated strategies of both players (iplausible bullying) leaving out strategy C of player 1 increases player 2 s guaranteed profit to 2 notation: non-cooperative (i.e., NE or guaranteed) profits will be denoted as v(1) for player 1 and v(2) for player 2

45 Cooperation with Transferable Payoffs 45 switch of players focus: fro individual payoffs to the total payoff (which can be redistributed afterwards): 1 \ 2 L M R T 3 ; 1 9 ; ; 2 C -9 ; 9-5 ; 35 2 ; -2 B -10 ; 9 13 ; 3 5 ; 4 1 \ 2 L M R T C B the axiu attainable total payoff = v(1,2) = 30 crucial question: how to divide the total payoff?

46 46 Distribution of Payoffs iputation: a potential final distribution of payoffs to both players (a 1 for player 1, a 2 for player 2) core of the gae: the set of all iputations (a 1,a 2 ) such that: e.g., for the gae fro the previous slides, superadditive effect: v(1,2) v(1) v(2) = = 25 a a v(1,2), a 1 2 a a a a a 1 2 v(1), v(2), 30, 3, a fair division: each player gets her guaranteed payoff + half of the superadditive effect: a* 3 25 / , a * 2 25 /

47 LECTURE 4: MIXED STRATEGIES (CONT D), BIMATRIX GAMES