Maximum Expected U/lity

Transcription

1 Probabilis/c Graphical Models Ac/ng Decision Making Maximum Expected U/lity

2 Simple Decision Making A simple decision making situation D: A set of possible actions Val(A)={a 1,,a K } A set of states t Val(X) = {x 1,,x N } A distribution P(X A) A utility function U(X, A)

3 Expected Utility Want to choose action a that maximizes the expected utility

4 Simple Influence Diagram m 0 m 1 m Market Found U f 1 f 0 m m m

5 More Complex Influence Diagram Difficulty Intelligence Study Grade V G Letter Job V Q V S

6 Information Edges m 0 m 1 m Market Decision rule δ at action node A is a CPD: P(A Parents(A)) U Survey Found s 0 s 1 s 2 m m m f 0 f 1 m m m

7 Expected Utility with Information Want to choose the decision rule δ A that maximizes the expected utility

8 Finding MEU Decision Rules Market Survey Found U

9 Finding MEU Decision Rules Market s 0 s 1 s 2 f 1 m 0 m 1 m 2 m m m m m m f 0 f 1 s s s f 0 U Survey Found

10 More Generally

11 MEU Algorithm Summary To compute MEU & optimize decision at A: Treat A as random variable with arbitrary CPD Introduce utility factor with scope Pa U Eliminate all variables except A, Z (A s parents) to produce factor μ(a, Z) For each z, set:

12 Decision Making under Uncertainty MEU principle provides rigorous foundation PGMs provide structured t representation ti for probabilities, actions, and utilities PGM inference methods (VE) can be used for Finding the optimal strategy Determining overall value of the decision situation Efficient methods also exist for: Multiple utility components Multiple decisions

13 Probabilis"c Graphical Models Ac"ng Decision Making U"lity Func"ons

14 Utilities and Preferences 0.2 $4mil p 0.25 $3mil f 0.8 $ $0

15 Utility = Payoff? 0.8 $4mil p 1 $3mil f 0.2 $0 0 $0

16 St. Petersburg Paradox Fair coin is tossed repeatedly until it comes up heads, say on the n th toss Payoff = $2 n

17 U D U U($500) D= $0 with prob. 1-p $1000 with prob. p $ reward Certainty t equivalent insurance/risk premium

18 Typical Utility Curve U $

19 Multi-Attribute Utilities All attributes affecting gpreferences must be integrated into one utility function Human life Micromorts QALY (quality-adjusted life year)

20 Example: Prenatal diagnosis U 1 (T) + U 2 (K) + U 3 (D,L) + U 4 (L,F) Testing Knowledge Down s Loss of syndrome fetus Future pregnancy

21 Summary Our utility function determines our preferences about decisions s that t involve uncertainty t Utility generally depends on multiple factors Money, time, chances of death, Relationship is usually nonlinear Shape of utility curve determines attitude to risk Multi-attribute utilities can help decompose high-dimensional function into tractable pieces

22 Probabilis0c Graphical Models Ac0ng Decision Making Value of Perfect Informa0on

23 Value of Information VPI(A X) is the value of observing X before choosing an action at A D = original influence diagram D X A = influence diagram with edge X A

24 Finding MEU Decision Rules Market Survey Found U

25 Value of Information Theorem: VPI(A X) 0 VPI(A X) = 0 if and only if the optimal decision rule for D is still optimal for D rule for D is still optimal for D X A

26 Value of Information Example s 1 s 2 s 3 s State 1 State 1 s 2 s s 1 f 0 f Funding 1 Company Funding 2 s s V δ*(c S 2 ) = P(c 2 )=1 if S = 3 2 s P(c 1 )=1 otherwise EU(D[c 1 ]) = 0.72 EU(D[c 2 ]) = 0.33 MEU(D S 1 C) = 0.743

27 Value of Information Example s 1 s 2 s 3 s State 1 State 1 s 2 s s 1 f 0 f Funding 1 Company Funding 2 s s V δ*(c S 2 ) = P(c 2 )=1 if S = s,s P(c 1 )=1 otherwise EU(D[c 1 ]) = 0.35 EU(D[c 2 ]) = 0.33 MEU(D S 2 C) = 0.43

28 Value of Information Example s 1 s 2 s 3 s State 1 State 1 s 2 s s 1 f 0 f Funding 1 Company Funding 2 s s V δ*(c S 2 ) = P(c 2 )=1 if S = s,s P(c 1 )=1 otherwise EU(D[c 1 ]) = EU(D[c 2 ]) = MEU(D S 1 C) =

29 Summary Influence diagrams provide clear and coherent semantics for the value of making an observation Difference between values of two IDs Information is valuable if and only if it induces a change in action in at least one context t