Extensive-Form Games with Imperfect Information

Transcription

1 May 6, 2015

2 Example 2, 2 A 3, 3 C Player 1 Player 1 Up B Player 2 D 0, 0 1 0, 0 Down C Player 1 D 3, 3

3 Extensive-Form Games With Imperfect Information Finite No simultaneous moves: each node belongs to a single player (possibly chance. ) Information Sets The nodes h at which i moves are partitioned into sets H. The interpretation is that player i knows that it is his move and some node in the set has been reached, he does not know which one. (Behavioral) Strategies Subgames Always begin with singleton information sets Can t break information sets Nash and Subgame Perfect Nash Equilibria

4 The Weakness of SPNE 1 Stay out Enter, Red Enter, Green 2 (0, 0) Fight Accom. Fight Accom. ( 1, 2) (1, 1) ( 1, 2) (1, 1) This example has no proper subgames so SPNE is just NE. So we need to extend the concept of sequential rationality

5 Beliefs Does the player s continuation strategy maximize his continuation payoff? We need a way to pose the same question at a non-singleton information set. But what is the continuation payoff at a non-singleton information set? We need to specify the player s beliefs over the nodes in the information set. Definition A system of beliefs is a collection µ = (µ 1, µ 2,..., µ N ) where µ i ( H) H specifies a probability distribution over the nodes in any information set H belonging to player i. We interpret µ( H) H as i s belief about which node in H has been reached.

6 Continuation Payoffs Given a behavioral strategy profile σ At any node h belonging to i we can compute the probability distribution over terminal nodes. And thus we can compute the expected payoff to i conditional on having arrived at h, u i (σ h) Now to compute the continuation payoff at a non-singleton information set H, we take expectations with respect to the belief µ( H): u i (σ H) = µ(h H)u i (σ h) h H

7 Sequential Rationality Definition A behavioral strategy profile σ is sequentially rational relative to a system of beliefs µ if for each i and for each information set H belonging to i, for every alternative strategy σ i. u i (σ H) u i (σ i, σ i H)

8 Assessments In extensive form games with imperfect information a solution is described by two objects The strategy profile The system of beliefs A pair (σ, µ) consisting of the strategy profile σ and the system of beliefs µ is called an assessment. Solution concepts will impose requirements on the assessment: Sequential rationality Some condition on the beliefs.

9 Sequential Rationality 1 Stay out Enter, Red Enter, Green 2 (0, 0) Fight Accom. Fight Accom. ( 1, 2) (1, 1) ( 1, 2) (1, 1) Sequential rationality alone restores the intuitive solution of this game.

10 Beliefs 1, 1 H Player 1 H Player 2 T 1, 1 1 1, 1 T H Player 1 T 1, 1

11 Sequential Rationality Alone Is Not Enough Sequential rationality alone is not enough for this game. Consider the assessment in which both players play T but 2 s belief assigns probability greater than 1/2 to the top node. It is sequentially rational. But it isn t even a Nash equilibrium. (This is Matching Pennies)

12 Bayes Rule on the Path Given a behavioral strategy profile, we say that a system of beliefs satisfies Bayes rule on the path if at every information set H which is arrived at with positive probability, the belief µ i ( H) is the conditional probability distribution.

13 (Weak) Perfect Bayesian Equilibrium Definition An assessment (σ, µ) is a (weak) Perfect Bayesian Equilibrium if it is sequentially rational and satisfies Bayes rule on the path. So in an extensive form game with imperfect information, to solve for a Weak PBE is to State the strategy profile σ. State the system of beliefs µ. Verify that the system of beliefs is derived from σ on the path. Verify that the strategy profile is sequentially rational relative to µ.

14 Problem with Weak PBE 1/2, 0, 0 A 1, 1, 1 H Player 1 Player 3 H B Player 2 T 1, 1, 1 1 1, 1, 1 T H Player 3 T 1, 1, 1

15 Problems with Weak PBE Consider the following assessment: 1 Plays A 2 Plays T 3 plays T 3 s belief assigns probability greater than 1/2 to his top node. This is a weak PBE Sequentially rational Satisfies Bayes rule on the path. But it is not a subgame perfect Nash equilibrium. (The subgame is matching pennies.) The unique SPE has 1 playing B.

16 Related Problem 2, 2 A 3, 0 C Player 1 Player 1 Up B Player 2 D 0, 3 1 0, 3 Down C Player 1 D 3, 0

17 Failure of OSDP for Weak PBE Consider the following assessment. 1 Plays (A, C ) 2 Plays Down 1 assigns probability 1 to the top node in his information set. This is not sequentially rational for player 1 he can improve his payoff at the initial node by changing his strategy to B, D. But there is no profitable one stage deviation at any information set.

18 A Partial Fix Bayes rule where possible. i.e. suppose All of the nodes in an information set have a common immediate predecessor And that predecessor is a singleton information set Then the mixed action at that predecessor should define the belief at the information set. There are other examples where it is possible to apply Bayes rule even at off-path information sets. Applying the where possible principle is a bit of a case-by-case matter.

19 Application: Take it or leave it offers Single object for sale. Player 1 is the seller, Player 2 is a buyer. The buyer has a value v > 0 for the object. The seller has a prior belief µ about v. The seller demands a price p for the good. The buyer accepts or rejects. Payoffs from sale: v p for the buyer p for the seller Payoffs are zero if no sale.

20 Extensive Form Game begins with a move by Nature. (By the way this is how we put incomplete information into an extensive-form game.) Nature selects v with an exogenous probability distribution given by the prior µ. Player 1 does not observe Nature s move (A single information set includes them all.) Player 2 observes the offer made as well as Nature s move (singleton information set.)

21 Sequential Rationality For the Buyer Because all of the buyer s moves are at singleton information sets, sequential rationality pins down his continuation strategy, except when indifferent. The buyer accepts any offer strictly below his value. The buyer rejects any offer strictly above his value. Sequential rationality alone does not restrict his response to prices equaling his value. (Think of the complete information ultimatum game.)

22 Binary Values Suppose v can take on two possible values {v l, v h } with v h > v l. With probabilities µ h and µ l. Bayes rule now pins down the seller s belief at his information set.

23 Sequential Rationality for the Seller We can immediately rule out certain price offers No price offer less than v l can be sequentially rational. No price offer between v l and v h can be sequentially rational. No price offer greater than v h can be sequentially rational. So the only possible sequentially rational price offers are p = v l and p = v h.

24 Marginal Revenue Think back to textbook undergrad micro. A monopoly who lowers price faces a tradeoff Lower sales price More sales Our seller faces the same tradeoff, but more sales means higher probability of sale.

25 Marginal Revenue At price v h, a sale occurs with probability µ h < 1. At price v l : Sale occurs with probability 1. The marginal sale earns revenue µ l v l There is a loss of revenue on inframarginal sales equal to µ h (v h v l ) Marginal revenue is the net: µ l v l µ h (v h v l )

26 Marginal Revenue The seller will cut the price to v l if and only if marginal revenue is positive: v l µ h µ l (v h v l ) > 0

27 Perfect Bayesian Equilibrium The seller offers v l if marginal revenue is positive, v h if negative. (Either if zero.) The buyer accepts the seller s offer and any offer strictly greater than his value. The seller s beliefs are given by Bayes rule

28 Marginal Revenue Is Cleaner In The Continuum Case Suppose now that v is distributed according to a continuous CDF given by F with density f. f (v)v [1 F (v)] is the marginal revenue from reducing price below v (marginally). The seller wants to sell to type v if and only if v 1 F (v) f (v) > 0. and so the optimal take-it-or-leave-it offer is the v at which marginal revenue is zero.

29 Kuhn s Theorem There is a sense in which we can interchangeably use mixed and behavioral strategies. For example, in a game of perfect information: Fix any mixed strategy. There is a behavioral strategy which, regardless of the opponent s strategy, induces the same distribution over terminal nodes. Likewise for any behavioral strategy there is an equivalent mixed strategy. What about games with imperfect information?

30 Imperfect Recall Wear Wear -1 Wash Wash 0 1 Consider the laundry game. Consider the behavioral strategy which randomizes with equal probability. This is the optimal strategy. It has no equivalent mixed strategy.

31 Kuhn s Theorem Theorem In a finite extensive-form game with perfect recall, for every mixed strategy there is at least one behavioral strategy which induces the same distribution over terminal nodes against all strategy profiles of the opponents. Likewise, for any behavioral strategy there is at least one mixed strategy which induces the same distribution over terminal nodes against any strategy profile of the opponents.

32 Consistency If the behavioral strategy profile is fully mixed then all information sets are on the path. Beliefs are defined everywhere by Bayes rule. Thus, for any strategy profile, there are nearby strategy profiles which would give uniquely defined beliefs. Definition An assessment (σ, µ) is consistent if there is a sequence of fully mixed behavioral strategy profiles which converges to σ such that the corresponding sequence of belief systems (derived using Bayes rule) converges to µ.

33 Sequential Equilibrium Definition An assessment (σ, µ) is a sequential equilibrium if it is sequentially rational and consistent. Every finite game has a sequential equilibrium. A sequential equilbirium is subgame-perfect. A sequential equilibrium has beliefs defined by Bayes rule wherever possible. When beliefs are consistent, the one-stage deviation principle holds. Additional restrictions are implied.

34 Forward Induction 1 Out In 2, 0 L 2 R 1 l r l r 1, 3 0, 0 0, 0 3, 1