Decision Theory. Course details. course notes 2008/2009. Studymanual (online) c L.C. van der Gaag, S. Renooij, P.

Transcription

1 Decision Theory course notes 2008/2009 c L.C. van der Gaag, S. Renooij, P. de Waal Master Applied Computing Science, UU ICS Lecturer: Prerequisite: Literature: Examination: Course details dr. S. Renooij silja@cs.uu.nl Probability theory Textbook Studymanual (online) written exam (closed book) practical assignment Deadlines: date: March 2 describing choice of decision problem date: April 6 written report (on paper) 1 / / 401 Decision Theory: Course overview A precise and systematic study of the formal and abstract properties of decision making scenarios. This course: Models and decision analysis techniques for optimal decision making with and without uncertainty or risk Subjects: Decision criteria Decision trees Utility theory Multi-attribute decision making Analyses of uncertainties Overview of treated techniques for rational decision making 1 attribute n 2 attributes certainty - Analytic Hierarchy Process (AHP) uncertainty - maximin - maximax - minimax regret - dominance (det.) risk - Bayes criterion - multi-attribute UT - dominance (prob.) (MAUT) - utility theory (UT) The risk versus uncertainty distinction is often used to distinguish between theories which: use the (subjective or objective) assignment of mathematical probabilities (risk) make probability assignments (uncertainty) 3 / / 401

2 The Straatnieuws problem Introduction Arie is a Straatnieuws seller: each morning, he purchases a number of copies at 1.00 euro each; throughout the day, he sells as many copies as he can, typically between 16 and 20, at 1.50 euro each; at the end of the day, he discards the remaining copies. How many copies should he purchase tomorrow morning? 5 / / 401 The aneurysm problem Maria presents with an intracranial aneurysm. The attending physician can decide to perform a preventive surgical procedure: if left untreated, the aneurysm may rupture in time Maria may die or may be seriously disabled; the surgical procedure has associated immediate risks Maria may die during the surgery or may be seriously disabled. What should the physician do? The Amsterdam Airport problem The Dutch government has to decide upon a site for future airport facilities. The objectives are: to minimise the costs to the government; to raise the capacity of airport facilities in the Netherlands; to improve the safety of airport traffic; to minimise environmental damage; to minimise noise levels; to minimise displacement of inhabitants;... 7 / / 401

3 The decision-analysis cycle identification Hard decision problems Many real-life decision problems are hard because they involve inherent uncertainty; they involve complex patterns of preference; they involve multiple objectives with trade-offs; they involve a large number of issues. Decision analysis provides effective methods and techniques for structuring and solving hard decision problems. structuring quantification solution sensitivity analysis further analysis needed? no implementation yes 9 / / 401 Elements of decision problems: variables Identification What is the problem to solve (decision context), and what are the alternatives? What are the objectives, and how are they measured (attributes)? Which uncertain events are relevant, and what are their outcomes? What are the possible consequences from decisions and uncertain events, and which do you prefer? Note that the relevancy of uncertain events and the consequences depend on your objectives. A simple decision problem includes two types of variable: a decision variable D with values, or decisions, d 1,...,d n, n 2; a chance variable C with values, or states, c 1,...,c m, m 2. Example Reconsider the decision context of the Straatnieuws problem: the decision variable is the number of copies to buy: D = {0, 1,,...,16, 17, 18, 19, 20,...,50} the chance variable is tomorrow s demand of copies: C = {16, 17, 18, 19, 20} 11 / / 401

4 Elements of decision problems: probabilities A simple decision problem includes, for each decision d i, a probability distribution Pr di (C) Pr(C d i ) over all states of C given that the decision d i has been implemented. Example Reconsider the Straatnieuws problem. We assume that the likelihood of any state is independent of the decision taken; the likelihood of the various states is uniformly distributed. Pr(16) = 0.2 Pr(18) = 0.2 Pr(20) = 0.2 So: Pr(17) = 0.2 Pr(19) = / 401 Elements of decision problems: rewards A simple decision problem includes a reward function r : D C IR that assigns a reward to any decision d i and any state c j, i = 1,...,n, j = 1,...,m. Example Reconsider the Straatnieuws problem. The reward function is: { 1.5 cj d r(d i,c j ) = i if c j d i 0.5 d i otherwise 14 / 401 Elements of decision problems: rewards Example Reconsider the Straatnieuws problem. The reward function is: { 1.5 cj d r(d i,c j ) = i if c j d i 0.5 d i otherwise The associated reward matrix is: Organising the elements The elements of a decision problem are often organised in a decision tree. Example Reconsider the aneurysm problem. The following decision tree organises the elements of the problem: no surgery surgery p(rupture) = 0.29 p(no rupture) = 0.71 p(deceased) = 0.02 p(disabled) = 0.06 p(cured) = 0.92 p(deceased) = 0.55 p(disabled) = 0.15 p(cured) = / / 401

5 The Mexico City Airport problem Case study: the Mexico City Airport problem It is The Mexican government has to formulate a strategy for developing major airport facilities for the Mexico City metropolitan area: expand the current Texcoco airport, built in 1928; develope a new airport at a different location (Zumpango); a (temporary(?)) combination of the previous two options. The decision problem phrased: Over the next 30 years, how should airport facilities be developed to assure adequate service for the region during the period from now to the year 2000? In particular: what kind of aircraft activity (International, Domestic, Military, or General) should be operating at which site and when? (1971 SOP study, 50 man-days consultancy; for details see Keeney & Raiffa 1993, Chapter 8) 17 / / 401 M.C. Airport: some background M.C. Airport: government objectives 10 miles Zumpango Expressway Mexico City airport Lake 1930 : population < : population ; passengers a year 2000 : population ; passengers 19 / 401 For the decision problem at hand the following objectives are identified: 1 minimise the total construction and maintenance costs; 2 provide adequate capacity to meet air traffic demands; 3 minimise access time to airports; 4 maximise safety of airport traffic; 5 minimise social disruption (displacement) caused by provision of new facilities; 6 minimise effects of noise pollution due to air traffic. Note that objectives concern different interest groups: 1 and 2 affect the government; 2, 3 and 4 affect the passengers; 4, 5 and 6 affect the residents. 20 / 401

6 M.C. Airport: design choices Analysing a large scale decision problem involving long-term planning is hard. The type of analysis depends on its purpose: Dynamic: designed to only indicate what action to take right now; subsequent actions will depend on the unfolding of future critical events; Static: designed to identify strategies, which currently seem effective, for development over a period of time. For different reasons the static approach is adopted. In addition, some simplifying assumptions w.r.t time are made: the 30 year period is divided into 3 decades; at any one time a category of activity can operate at only one site; changes are allowed in 1975, 1985 and / 401 M.C. Airport: decision variables The decision problem at hand includes one decision variable: variable D with alternatives d 75 d 85 d 95, d j {T, Z} {I, D, M, G}, for site and aircraft activity over the subsequent time periods. The total number of alternatives is (2 4 ) 3 = 4096, but only about 100 are sensible. It seems more insightful to model the problem using three subsequent decision variables: a decision variable D 75 with alternatives di 75 for site and aircraft activity in the period ; a decision variable D 85 with alternatives di 85 for site and aircraft activity in the period ; a decision variable D 95 with alternatives di 95 for site and aircraft activity in 1995 and after; Each d j i is again an element of the set {T, Z} {I, D, M, G} and has 2 4 = 16 alternatives. 22 / 401 M.C. Airport: chance variables Different uncertain events were identified as influencing the objectives. These are captured by the following six chance variables: C 1 = the net present value of the total costs in millions of pesos (discount rate of 12%); C 2 = the maximum number of aircraft operations possible (capacity); C 3 = the average two-way access time weighed by the expected number of travellers from each zone in M.C.; C 4 = the average number of people killed or seriously injured per aircraft accident over a 30 year period (safety); C 5 = the number of people displaced by airport development; C 6 = the average number of people that is annually subjected to a noise level 90 CNR. Assuming that all chance variables are statistically independent, the problem can be modelled with a single chance variable C = (C 1,C 2,C 3,C 4,C 5,C 6 ). 23 / 401 M.C. Airport: the decision tree D 75 Z-IDMG T-IDMG T-GM,Z-ID D 85 D 85 D 85 T-M,Z-IDG T-GM,Z-ID D 95 D 95 Z-IDMG T-M,Z-IDG C C Pr(c) (x 1, x 2, x 3, x 4, x 5, x 6 ) 24 / 401

7 Dominance Decision criteria (Not in textbook) Consider a simple decision problem with the decision variable D = {d 1,...,d n }, n 2, the chance variable C = {c 1,...,c m }, m 2, and the reward function r(d,c). A decision d i can dominate a decision d j ; this dominance is said to be deterministic, if the property can be established from the reward function only; probabilistic (or stochastic), if the property is established from the distribution of rewards. Note: the textbook (Clemen) uses different definitions! 25 / / 401 Deterministic dominance Consider a simple decision problem with the decision variable D = {d 1,...,d n }, n 2, the chance variable C = {c 1,...,c m }, m 2, and the reward function r(d,c). A decision d i is said to deterministically dominate d j if r(d i,c k ) r(d j,c k ) for all c k C, and there is a c l with r(d i,c l ) > r(d j,c l ); A decision is deterministically admissible if it is not dominated by any other decision. 27 / 401 Deterministic dominance an example (I) Reconsider the Straatnieuws problem with its reward matrix Now suppose that Arie purchases 12 copies: The decision to buy 12 copies is deterministically dominated by the decision to buy 16 copies and is deterministically inadmissible. 28 / 401

8 Deterministic dominance an example (II) Reconsider the Straatnieuws problem with its reward matrix Likewise, Arie does not consider purchasing 21 (or more) copies: The decision to buy 21 copies is deterministically dominated by the decision to buy 20 copies. Deterministic dominance an example (III) Suppose that in the Straatnieuws problem, we have the following reward matrix Now obviously all alternatives are equivalent and none of the decisions deterministically dominate any of the other. 29 / / 401 Three simple decision criteria Consider a simple decision problem with the decision variable D = {d 1,...,d n }, n 2, the chance variable C = {c 1,...,c m }, m 2, and the reward function r(d,c): the maximin criterion selects a decision d i that maximises the minimal reward over all outcomes; the maximax criterion selects a decision d i that maximises the maximal reward over all outcomes; the minimax regret criterion selects a d i that minimises the maximal regret over all outcomes, where the regret r(d i,c j ) for d i and c j is defined by ( r(d i,c j ) = max k ) r(d k,c j ) r(d i,c j ) 31 / 401 The maximin criterion The maximin criterion selects a decision d i that maximises min j r(d i,c j ) Example: Reconsider Arie s Straatnieuws problem with its reward matrix: min The maximum of the minimal reward is 8.00 euro. Arie decides to purchase 16 copies. 32 / 401

9 The maximax criterion The maximax criterion selects a decision d i that maximises max j r(d i,c j ) Example: Reconsider Arie s Straatnieuws problem with its reward matrix: max The maximum of the maximal reward is euro. Arie decides to purchase 20 copies. 33 / 401 The minimax regret criterion The minimax regret criterion selects a decision d i that minimises max j r(d i,c j ), where r(d i,c j ) is a regret matrix: ( ) r(d i,c j ) = maxr(d k,c j ) r(d i,c j ) k 34 / 401 Example: Reconsider Arie s Straatnieuws problem: max: The regret matrix is: The minimax regret criterion continued Example Reconsider the regret matrix of the Straatnieuws problem: max The minimum of the maximal regret is 1.50 euro. Arie decides to purchase 17 copies. 35 / / 401

10 The Bayes criterion A note on the three criteria Reconsider the Straatnieuws problem, this time with Pr(16) = 0.96 Pr(18) = 0.01 Pr(20) = 0.01 Pr(17) = 0.01 Pr(19) = 0.01 The maximax criterion still opts for purchasing 20 copies. Consider a simple decision problem with the decision variable D = {d 1,...,d n },n 2; the chance variable C = {c 1,...,c m },m 2; the probability distributions Pr(C d i ), d i D; the reward function r(d, C). The Bayes criterion selects a decision d i that maximises the expected reward IE(r(d i,c)) = j Pr(c j d i ) r(d i,c j ) 37 / / 401 The Bayes criterion an example Reconsider Arie s Straatnieuws problem with and Pr(16) = 0.1 Pr(18) = 0.3 Pr(20) = 0.2 Pr(17) = 0.2 Pr(19) = 0.2 The expected rewards are IE(r(16, C)) = 8.00 IE(r(18, C)) = 8.40 IE(r(20, C)) = 7.30 IE(r(17, C)) = 8.35 IE(r(19, C)) = 8.00 Arie decides to purchase 18 copies. 39 / 401 A distribution of rewards Consider a simple decision problem as before. Let R be a reward variable with values R = {r(d i,c j ) d i D,c j C} The distribution Pr(R d i ) over the reward variable R, induced by a decision d i, and defined by Pr(r d i ) = Pr(c j d i ) c j with r(d i, c j ) = r for each r R, is termed the distribution of rewards for d i (also: reward profile, or risk profile). 40 / 401

11 A distribution of rewards an example Reconsider the Straatnieuws problem with Pr(16) = 0.2 Pr(18) = 0.1 Pr(20) = 0.2 Pr(17) = 0.3 Pr(19) = 0.2 The decision to purchase 19 copies induces the following distribution of rewards: probability reward and the following cumulative distribution: cumulative probability reward 41 / 401 Probabilistic dominance For each decision d i of a simple decision problem, let Pr(R d i ) be the distribution of rewards for d i and let F(R d i ) be the associated cumulative distribution of rewards. A decision d i is said to probabilistically dominate d j if F(r k d i ) F(r k d j ) for all r k R, and there is an r l with F(r l d i ) < F(r l d j ); A decision is probabilistically admissible if it is not dominated by any other decision. cumulative probability d j reward d i 42 / 401 Probabilistic dominance an example Reconsider the, simplified, aneurysm problem with no surgery surgery p(deceased) = 0.05 p(disabled) = 0.25 p(cured) = 0.70 p(deceased) = 0.01 p(disabled) = 0.04 p(cured) = 0.95 The cumulative distributions of rewards are cumulative probability no surgery reward surgery The decision to abstain from surgery is probabilistically dominated by the decision to perform a surgical procedure. 43 / 401 Probabilistic dominance and expected reward Reconsider the, simplified, aneurysm problem with no surgery surgery p(deceased) = 0.05 p(disabled) = 0.25 p(cured) = 0.70 p(deceased) = 0.01 p(disabled) = 0.04 p(cured) = 0.95 The decision to abstain from surgery is probabilistically dominated by the decision to perform a surgical procedure, and therefore has a lower expected reward: vs / 401

12 Deterministic dominance (Clemen) Consider a simple decision problem with the decision variable D = {d 1,...,d n }, n 2, the chance variable C = {c 1,...,c m }, m 2, and the reward function r(d,c). For each decision d i, let Pr(R d i ) be the distribution of rewards for d i and let F(R d i ) be the associated cumulative distribution of rewards. A decision d i is said to deterministically dominate d j if there exists an r R such that F(r d j ) = 1, and F(r d i ) = Pr(r d i ); Note that the above amounts to min k r(d i,c k ) max k r(d j,c k ) 45 / 401 Deterministic dominance (Clemen) an example (I) Reconsider the Straatnieuws problem with its reward matrix Recall that if Arie purchases 12 copies, then his reward will be 6.00 euros, regardless of the demand. The decision to buy 12 copies is also deterministically dominated by the decision to buy 16 copies, according to the textbook: min k r(16,c k ) = 8.00 maxr(12,c k ) = 6.00 k 46 / 401 Deterministic dominance (Clemen) an example (II) Reconsider the Straatnieuws problem with its reward matrix Should Arie consider purchasing 21 (or more) copies? The decision to buy 21 copies is not deterministically dominated by the decision to buy 20 copies, according to the textbook: min k r(20,c k ) = 4.00 maxr(21,c k ) = 9.00 k 47 / 401 Deterministic dominance (Clemen) an example (III) Suppose that in the Straatnieuws problem, we have the following reward matrix Now we have for all pairs of alternatives d i and d j that min k r(d i,c k ) maxr(d j,c k ) k so all alternatives deterministically dominate all others, according to the textbook definition. 48 / 401

13 The ing problem Decision trees Colaco has some assets and considers putting a new soda on the national. It has a choice of ing the new soda; not ing the new soda. It can also decide to first perform a local survey to gain additional information. Then, the first decision is whether or not to perform a local survey; the second decision is whether or not to. What should Colaco do? 49 / / 401 Sequential decisions In general, a decision problem may involve taking a sequence of decisions over time: any decision may influence a subsequent one; inbetween the decisions, observations may become available that provide additional information. The decision maker then has to consider the entire planning horizon: first decision second decision planning horizon last decision Chance trees A chance tree is a tree-like structure that organises the chance variables of a decision problem, along with their probabilities: each (chance) node represents a chance variable; each edge emanating from a chance node has associated a value of the represented variable; the label of such an edge captures the represented value and associated probability. Example p(local success) = 0.60 before taking the first decision. 51 / 401 p(local failure) = / 401

14 Scenarios in a chance tree Inverting a chance tree A scenario is a path from the root of a tree to an endpoint. A chance tree describes one or more scenarios. The probability of a scenario equals the product of the probabilities associated with the values along the scenario. Example: Reconsider the ing problem with Consider a chance tree with the statistical variables B and C: Pr(b) Pr( b) Pr(c b) Pr( c b) Pr(c b) Pr( c b) Pr(bc) Pr(b c) Pr( bc) Pr( b c) Pr(l) = 0.60 Pr( l ) = 0.40 Pr(n l) = 0.85 Pr( n l) = 0.15 Pr(n l ) = 0.10 Pr( n l ) = 0.90 Pr(ln) = 0.51 Pr(l n) = 0.09 Pr( ln) = 0.04 Pr( l n) = 0.36 Inverting the tree is equivalent to applying Bayes Theorem: Pr(c) Pr( c) Pr(b c) Pr( b c) Pr(b c) Pr(bc) Pr( bc) Pr(b c) The scenario probability Pr(ln) equals Pr(ln) = Pr(l) Pr(n l) = / 401 Pr( b c) Pr( b c) 54 / 401 The ing problem revisited The ing problem continued Consider the chance tree with the chance variables L and N: Pr(l) = 0.60 Pr( l ) = 0.40 Pr(n l) = 0.85 Pr( n l) = 0.15 Pr(n l ) = 0.10 Pr( n l ) = 0.90 Pr(ln) = 0.51 Pr(l n) = 0.09 Pr( ln) = 0.04 Pr( l n) = 0.36 Now, consider the incomplete chance tree: Pr(n) =? Pr( n) =? Pr(l n) =? Pr( l n) =? Pr(l n) =? Pr(ln) = 0.51 Pr( ln) = 0.04 Pr(l n) = 0.09 To invert the tree, first its structure is inverted. Then, the scenario probabilities are associated with the appropriate endpoints: Pr(n) =? Pr( n) =? Pr(l n) =? Pr( l n) =? Pr(l n) =? Pr( l n) =? Pr(ln) = 0.51 Pr( ln) = 0.04 Pr(l n) = 0.09 Pr( l n) = 0.36 Pr( l n) =? Pr( l n) = 0.36 The prior probabilities of the values of N are computed from the scenario probabilities using the basic rule of marginalisation: Pr(n) = Pr(ln) + Pr( ln) = 0.55 Pr( n) = Pr(l n) + Pr( l n) = / / 401

15 The ing problem continued Consider the incomplete chance tree: The ing problem continued Pr(n) = 0.55 Pr(l n) =? Pr(ln) = 0.51 The inverted chance tree thus is: Pr( n) = 0.45 Pr( l n) =? Pr(l n) =? Pr( l n) =? Pr( ln) = 0.04 Pr(l n) = 0.09 Pr( l n) = 0.36 Pr(n) = 0.55 Pr(l n) = 0.93 Pr( l n) = 0.07 Pr(ln) = 0.51 Pr( ln) = 0.04 The conditional probability Pr(l n) is computed from So, Pr(n) Pr(l n) = Pr(ln) Pr( n) = 0.45 Pr(l n) = 0.2 Pr( l n) = 0.8 Pr(l n) = 0.09 Pr( l n) = 0.36 Pr(l n) = Pr(ln) Pr(n) = = / / 401 Decision trees A decision tree is a tree-like structure that organises the elements of a decision problem: each decision node represents a decision variable; each edge emanating from a decision node has associated a value of the represented variable; the label of such an edge captures the represented decision. Scenarios in a decision tree (Recall: a scenario is a path from the root of a tree to an endpoint) A decision tree describes various scenarios: each endpoint represents a consequence of the preceding scenario; the label of an endpoint captures the reward of the preceding scenario. Example Example p() = euro p() = euro 59 / / 401

16 The ing problem revisited Time A decision tree reflects, from left to right, the course of time within its scenarios: the value of a chance node before a decision node should be known with certainty before the decision is taken; the value of a chance node after a decision node need not be known before the decision is taken. Reconsider Colaco s ing problem and its elements: the decision variables are the decision S whether (s) or not ( s) to perform a survey; the decision M whether (m) or not ( m) to the new soda; the chance variables are the variable L capturing local success (l) or failure ( l ) of the soda; the variable N capturing (n) or failure ( n) of the soda; the probability distribution Pr(LN) has Pr(n l) = 0.85 Pr(l) = 0.60 Pr( n l) = 0.15 Pr( l ) = 0.40 Pr(n l ) = 0.10 Pr( n l ) = / / 401 The ing problem revisited local success p = 0.60 p = 0.85 given assets of euro; the costs of survey of euro; the gain from of euro; the loss from of euro; the rewards r(sm, LN) associated with the consequences of different scenarios are: r(sm,ln) = r( sm,ln) = r(sm,l n) = r( sm,l n) = r(s m,ln) = r( s m,ln) = survey survey local failure p = 0.40 p = 0.55 p = 0.15 p = 0.10 p = / 401 p = / 401

17 Strategies Reduced decision trees A strategy is a prescription for choosing decisions upon traversing a decision tree from the root to an endpoint. Example: The decision tree for Colaco s ing problem organises six strategies: 1 perform a local survey if successful, then else do not ; 2 perform a local survey if successful, then else either; 3 perform a local survey if successful, then else also; 4 perform a local survey if successful, then else ; 5 perform a local survey and rightaway; 6 perform a local survey and. 65 / 401 A decision tree is in reduced form if the root of the tree is the only decision node Any decision tree has an equivalent tree in reduced form. Example: The reduced decision tree for Colaco s ing problem organises the six strategies explicitly: strategy 1 strategy 5 local success p = 0.60 local failure p = 0.40 p = 0.55 p = 0.45 p = 0.85 p = / 401 Evaluating decision trees Solving the ing problem Reconsider the fifth strategy for the ing problem: 5. perform a local survey and immediately. strategy 5 p = p = Bayes criterion computes the expected reward to be = euro. 67 / / 401

18 Solving the ing problem cntd. Now reconsider the first strategy for the ing problem: p = 0.85 local success p = 0.60 strategy 1 p = 0.15 local failure p = 0.40 Consider the following part of a decision tree: Folding back part 1 s Pr(c 1 ) Pr( c 1 ) Then, by the distributive law, we have that Pr(c 2 c 1 ) Pr( c 2 c 1 ) Pr(c 2 c 1 ) Pr( c 2 c 1 ) Pr(c 1 c 2 ) r(s,c 1 c 2 ) + Pr(c 1 c 2 ) r(s,c 1 c 2 ) + Pr( c 1 c 2 ) r(s, c 1 c 2 ) + Pr( c 1 c 2 ) r(s, c 1 c 2 ) r(s, c 1 c 2 ) r(s, c 1 c 2 ) r(s, c 1 c 2 ) r(s, c 1 c 2 ) Bayes criterion computes the expected reward of this strategy to be ( ) + ( ) = = euro. 69 / 401 = Pr(c 1 ) [Pr(c 2 c 1 ) r(s,c 1 c 2 ) + Pr( c 2 c 1 ) r(s,c 1 c 2 ) ] + Pr( c 1 ) [Pr(c 2 c 1 ) r(s, c 1 c 2 ) + Pr( c 2 c 1 ) r(s, c 1 c 2 ) ] The property generalises to a sequence of chance variables of arbitrary length. 70 / 401 Solving the ing problem cntd. Now reconsider the first strategy for the ing problem: p = 0.85 local success p = 0.60 strategy 1 p = 0.15 Solving the ing problem cntd. Reconsider the following strategies of the ing problem: local success p = 0.85 p = 0.60 strategy 1 p = 0.15 local failure p = 0.40 local failure p = 0.40 if the survey shows local failure, then the (expected) reward is euro; if the survey shows local success, then the expected reward is = ; the expected reward of the strategy is = / 401 strategy 2 local success p = 0.60 local failure p = 0.40 Strategy 1 has a higher expected reward than strategy 2 and is therefore preferred. 72 / 401

19 Folding back part 2 Consider the following part of a decision tree: IE(r(d,C 1 C 2 )) = Pr(c 1 ) d d Pr(c 2 c 1 d) Pr( c 2 c 1 d) Pr(c 2 c 1 d ) Pr( c 2 c 1 d ) r(d, c 1 c 2 ) r(d, c 1 c 2 ) r( d, c 1 c 2 ) r( d, c 1 c 2 ) = Pr(c 1 ) Pr(c 2 c 1 d) r(d,c 1 c 2 ) + + Pr( c 1 ) Pr( c 2 c 1 d) r(d, c 1 c 2 ) = = Pr(c 1 ) [Pr(c 2 c 1 d) r(d,c 1 c 2 ) + Pr( c 2 c 1 d) r(d,c 1 c 2 )] + Pr( c 1 ) [Pr(c 2 c 1 d) r(d, c 1 c 2 ) + Pr( c 2 c 1 d) r(d, c 1 c 2 )] IE(r( d,c 1 C 2 )) = = Pr(c 1 ) [Pr(c 2 c 1 d) r( d,c1 c 2 ) + Pr( c 2 c 1 d) r( d,c1 c 2 ) ] + Pr( c 1 ) [Pr(c 2 c 1 d) r( d, c1 c 2 ) + Pr( c 2 c 1 d) r( d, c1 c 2 ) ] 73 / 401 Solving the ing problem cntd. Reconsider part of Colaco s decision tree: survey local success p = 0.60 local failure p = 0.40 p = 0.85 p = 0.15 p = 0.10 p = 0.90 if the survey shows local success, then the best decision is to ; if the survey shows local failure, then the best decision is not to. 74 / 401 Fold-back analysis The following procedure implements Bayes criterion for computing an optimal strategy from a decision tree: From the endpoints to the root do for each chance node C with values c 1,...,c m, m 2, compute the expected reward IEr(C) = Pr(c i π) IEr(L i ) i=1,...,m where L 1,...,L m are the successors of C and π is the preceding scenario; for each decision node D with values d 1,...,d n, n 2, compute the maximum expected reward IEr(D) = max i=1,...,n IEr(C i) survey survey local success p = 0.60 local failure p = p = 0.55 p = 0.85 p = 0.15 p = 0.10 p = where C 1,...,C n are the successors of D. 75 / p = / 401