TUFTS UNIVERSITY DEPARTMENT OF CIVIL AND ENVIRONMENTAL ENGINEERING ES 152 ENGINEERING SYSTEMS Spring Lesson 16 Introduction to Game Theory

Transcription

1 TUFTS UNIVERSITY DEPARTMENT OF CIVIL AND ENVIRONMENTAL ENGINEERING ES 52 ENGINEERING SYSTEMS Spring 20 Introduction: Lesson 6 Introduction to Game Theory We will look at the basic ideas of game theory. I believe there is an entire course on it in the economics department. Game theory involves optimization in a competitive environment.. The outcome is due to the choice of strategies, not only by the player but also by his or her opponent. 2. We will look at two person zero sum games; the simplest type. 3. We assume that both players are totally rational. In economics that means each acts only to benefit themselves. 4. We also assume that they have a common information base. Whatever one knows, the other knows. 5. Finally we assume that they pursue a minimax (or maximin) strategy. This means they strive to minimize their maximum loss (maximize their minimum gain). A conservative strategy, but possibly consistent with the common information base. 6. The players may have different sets of strategies. Comparing three strategies each with two equally likely outcomes Outcome Strategy 2 Choose if you wish to maximize maximum gain maximize expected gain minimize maximum loss Evaluating Payoff Tables (k) Let p ij be the payoff to player k if player plays his i th strategy and player j plays his j th () (2) strategy. Note that p ij = - p ij ; that is what we mean by a zero sum game. Here is a Payoff Table from the point of view of Player. es52 Lesson 6

2 Example : 2 3 min 2 4 Player Strategies max 2 4 Note that the minimax and the maximin are the same table entry. According to the minimax principle each player will use his strategy. Note that neither player stands to gain by switching strategies and in fact each stands to lose. Also note that this is not a fair game since if each plays his optimum strategy Player wins and Player 2 loses. In this example strategy completely dominates the other strategies for Player and the same is true for Player 2. When a strategy is dominated it is never used. Example 2: 2 3 min Player Strategies max There is no dominance here but again the minimax and the maximin are the same table entry, so Player plays his strategy 2 and Player 2 plays his strategy. Again, neither can improve by changing so the solution is stable. This is a fair game. Example 3: 2 3 min Player Strategies max There is no dominance and the minimax and maximin are different entries in the table. Player 2 would benefit by changing strategies from 3 to 2 but only if Player didn t change. The problem is that since they both share all information Player can then improve his situation by switching to strategy 2 and so on.. There seems to be no stable optimal solution. So what is the solution? The answer lies in the recognition that since your opponent knows everything you know no one can pursue an unanticipated strategy unless your strategy is chosen at random (i.e., you don t know for sure what you are going to do). The strategy is to pursue each of the pure strategies -, 2, or 3 with a certain probability. This is known as a mixed strategy. (Actually, pure strategies are just special cases of mixed strategies). There exists a stable mixed strategy meaning that neither player can increase his expected payoff by changing his probabilities. We can determine that optimum mixed strategy by linear es52 Lesson 6 2

3 programming. (Note the use of expected payoff as the performance measure in the mixed strategy case.) Linear Programming in a Nutshell: When a constrained optimization problem has both a linear performance measure (objective function) and linear constraints it is known as a linear programming problem. This is the class of optimization problems for which the most powerful solution techniques exist. Here is a typical problem in what is referred to as the standard form. Maximize Z = 2x + 3x2 Subject to (the constraints) 4x + 3x2 2 2x + 4x2 8 5x + 2x2 4 x 0 x2 0 This is a two dimensional problem as it has two decision variables, x and x2, but linear problems can be solved with thousands of decision variables. Here is the solution to the problem using LINGO. max = 2*x + 3*x2;!subject to; 4*x + 3*x2 <= 2; 2*x + 4*x2 <= 8; 5*x + 2*x2 <= 4; Objective value: Infeasibilities: Total solver iterations: 2 X X It assumes the non-negativity constraints for the decision variables. Every linear programming problem has a dual. In this case it is: es52 Lesson 6 3

4 min = 2*y + 8*y2 +4*y3;!subject to; 4*y + 2*y2 + 5*y3 >= 2; 3*y + 4*y2 + 2*y3 >= 3; Objective value: Infeasibilities: Total solver iterations: 2 Y Y Y Let s compare the two problems (primal and dual). There are lots of relationships. We will note only a few.. Maximum of objective function in the primal equals the minimum of the objective function in the dual. 2. The interchange of the coefficients in the objective function with the constraint values. 3. Transposing of constraint coefficients. 4. Interchange of slack or surplus values with the reduced cost values. 5. Interchange of value with dual price. If you were to take a course on linear programming, why this (and much more) is so would all in due course be explained to you. Here we will take it all on faith that it is always so. (Note: I just saw the play Two Men of Florence so pardon my excursion into the relative roles of faith and science.) OK, now let apply this to our game theory problem. We will let x T = [ x x2.. xm] be the mixed strategy for Player, where xi is the probability of playing pure strategy i. Likewise y T = [ y y2..yn] is the mixed strategy for Player 2. (So x and y are probability vectors their elements are non-negative and sum to.0.) Each player wishes to maximize his expected payoff. For Player that expected payoff is m n () = pij xiy j i= j= E. es52 Lesson 6 4

5 Let v be the expected payoff that Player gets if both players pursue their optimum strategy. Of course, if Player pursues his optimum strategy and Player 2 does not then Player s expected payoff is greater than v. Therefore, m n = () E pij xiy j i= j= v, with equality holding when Player 2 plays his optimum strategy. Since this holds for all mixed strategies, and pure strategies are just special cases of mixed strategies, it is also true for pure strategies. A pure strategy k for Player 2 is a vector y with a.0 in the k th entry and all the rest of the entries are 0. Therefore all the following hold as well, m i= This, along with xi 0, i =,m, and m i= () p x v for j =, n. x ij i = i gives us all the constraints we need for a linear programming formulation. All we need is an objective function. Since Player wishes to maximize v we will let xm+ = v and our linear programming problem becomes maximize Z = xm+ subject to: m i= m i= p x () ij x i = i x m+ for j = xi 0 for i =,m+., n Example 4: We all know the game Rock, Paper, Scissor.. Determine the optimum strategy for each player. Is the result consistent with your experience? 2. What if one of the players decides to use a different mixed strategy but his opponent does not know and continues playing the optimum strategy? 3. What if his opponent does know? Can he adjust his own strategy to take advantage of this? If so, what s his optimum strategy. es52 Lesson 6 5

6 . We will do this using LINGO. R P S min R Player Strategies P S max Clearly there is no dominance and no stable solution exists. Our own experience with this game tells us that if you become predictable you will begin to lose. It is the poster boy for the mixed strategy approach. We will do this using LINGO but because LINGO does not like negative numbers much we will add one () to every payoff (and then we can subtract the one at the end). R P S min R Player Strategies P S max Here is the LINGO solution for Player.!es52!Rock, Paper, Scissor as a Game;!Lesson 6-; max = x4;!subject to; *x + 2*x2 + 0*x3 >= x4; 0*x + *x2 + 2*x3 >= x4; 2*x + 0*x2 + *x3 >= x4; x + x2 + x3 = ; x >= 0; x2 >= 0; x3 >= 0; x4 >= 0; v = x4 - ; Objective value: Total solver iterations: 3 X X es52 Lesson 6 6

7 X X V We see from the result v = x4 = 0 that Rock, Paper, Scissors is a fair game and, as you all knew, the optimum mixed strategy is to play each pure strategy with equal probability. The results under Dual Price tell us that Player 2 should also use the equal probability approach as well. 2. We will look at this using a simulation. First we will check it out. es52 Lesson 6 Rock, Paper, Scissor Part 2 Player ` Player 2 Rock Paper Scissor Number of Rounds Player Wins Player 2 Wins Ties OK, that looks reasonable. Now assume Player 2 goes to a different strategy. es52 Lesson 6 Rock, Paper, Scissor Part 2 Player ` Player 2 Rock Paper Scissor Number of Rounds Player Wins Player 2 Wins Ties es52 Lesson 6 7

8 No statistically significant difference. Does that make sense? Let s see. Player wins if either he plays Rock and Player 2 plays Scissor (/3 *0.), he plays Paper and Player 2 plays Rock (/3*0.6), or he plays Scissor and Player 2 plays Paper (/3*0.3), which totals to /3. So, even if his opponent plays a non-optimal strategy he only benefits if he adjusts his strategy to take advantage of it. 3. We can set this up in LINGO as well. The expected payoff for Player is E where the yj s are fixed. m n () = pij xiy j i= j=!es52 Lesson 6 Part 3!Rock, Paper, Scissor; max = x*(*y+0*y2+2*y3)+x2*(2*y+*y2+0*y3)+x3*(0*y+2*y2+*y3)-; y= 0.6; y2=0.3; y3 = 0.;!This is Player 2's known strategy; x + x2 + x3 =.0; Objective value: Infeasibilities: Total solver iterations: 0 X Y Y Y X X It is no longer a fair game. We see that Player can expect to win. His strategy is to play Paper every time. The simulation confirms the obvious. es52 Lesson 6 8

9 es52 Lesson 6 Rock, Paper, Scissor Part 3 Player ` Player 2 Rock Paper 0.3 Scissor 0 0. Number of Rounds Player Wins 5997 Player 2 Wins Ties 3007 Return to Example 3: 2 3 min Player Strategies max Here s the LINGO analysis.!es52 Lesson 6;!Game from the perspective of Player!Add 4 to payoffs to assure x4 >= 0; max = x4;!subject to; 4*x + 9*x2 + 6*x3 >= x4; 2*x + 8*x2 + 7*x3 >= x4; 6*x + *x2 + 0*x3 >= x4; x + x2 + x3 = ; x >= 0; x2 >= 0; x3 >= 0; x4 > = 0; v = x4-4;!v is the payoff to Player I; Objective value: Total solver iterations: 3 X es52 Lesson 6 9

10 X X X V x = [ ] y = [ ] Aside from the dual, it is also possible to formulate this game from the perspective of Player max Player Strategies min !es52!Lesson 6;!Game from the perspective of Player 2!Add 5 to payoffs to assure y4 >= 0; max = y4;!subject to; 5*y + 7*y2 + 3*y3 >= y4; 0*y + *y2 + 8*y3 >= y4; 3*y + 2*y2 + 9*y3 >= y4; y + y2 + y3 = ; y >= 0; y2 >= 0; y3 >= 0; y4 >= 0;!w = y4-5;!w is the payoff to Player II; es52 Lesson 6 0

11 Objective value: Total solver iterations: 3 Y Y Y Y w = = y = [ ] x = [ ] es52 Lesson 6