Engineering Decisions

Transcription

1 GSOE9210 Decisions uner certainty an ignorance 1 Decision problem classes 2 Decisions uner certainty 3

2 Outline Decision problem classes 1 Decision problem classes 2 Decisions uner certainty 3 Decision problem classes Decision problem classes Decision problems can be classifie base on an agent s epistemic state: Decisions uner certainty: the agent knos the actual state Decisions uner uncertainty: (full uncertainty): the agent believes multiple states/outcomes are possible; likelihoos unknon Decisions uner risk: the agent believes multiple states/outcomes are possible; likelihoo information available

3 Outline Decisions uner certainty 1 Decision problem classes 2 Decisions uner certainty 3 Decisions uner certainty Decisions uner certainty Example (Project bugeting) You are a lea softare engineer in a major softare company. Your R & D team has propose three possible projects, A, B, an C, each ith a ifferent life-time. The net profits over the life of the projects are liste in the ajacent table. profit ($M) A 20 B 13 C 17 Which project oul you choose?

4 Complex outcomes Decisions uner certainty Project life-time cash-flos: A: three years, big initial set-up costs B: one year immeiate return C: three years, small initial set-up costs Ne perspective: Outcomes escribe by vectors: e.g., for A: ( 10, 5, 25). cashflo ($M) Year A B C What is more important: maximising total return, preserving cash, etc.? Which project oul you choose? Decisions uner certainty Composite outcomes: Net Present Value (NPV) The Net Present Value (NPV) of a project is the value of the project at present Cash flos in the future are orth less than in the present Moel this by a iscount rate for each perio; assume iscount rate of 20% NPV(A) = = 11.5 NPV(B) = 13.0 NPV(C) = = 11.7 More generally: v(x 1, x 2, x 3 ) = x 1 + x γ + x 3 (1 + γ) 2

5 Decisions uner certainty Decisions uner certainty Example (School fun-raising) A school committee is looking to hol a fun-raiser. It has a choice beteen holing a fête or a sports ay. S F s 0 s f S F s f In this example: A = {S, F} Ω = {s, f} S = {s 0 } Which action is preferre: F or S? Which outcome is preferre: f or s? Decisions uner certainty Decisions uner certainty: value functions Example (School fun-raising) A school committee is looking to hol a fun-raiser. It has a choice beteen holing a fête or a sports ay. It expects to make $150 for a fête but only $120 for a sports ay. s 0 V S $120 $120 F $150 $150 S F $120 $150 Value function over outcomes: v : Ω R In this example: v(s) = $120 v(f) = $150 Value function over actions; i.e., V : A R; here V (A) = v(ω), here ω = ω(a, s 0 )

6 Rational ecisions Decisions uner certainty Decision theory seeks to moel normative (i.e., rational) ecision-making: i.e., ecisions ieal rational agents ought to make Which principles govern rational ecision-making? Rationality Principle 1 (Elimination) Face ith to possible alternatives, rational agents shoul never choose the less preferre one. The principle of elimination says that rational agents shoul eliminate less preferre actions from consieration A rational ecision rule shoul satisfy this principle Decisions uner certainty Rational ecisions uner certainty Corollary Given a value function V : A R over actions, rational agents shoul prefer action A to B iff V (A) > V (B). s 0 V S $120 $120 F $150 $150 Since F is preferre to S (V (F) > V (S)), S is eliminate (by elimination), hence the rational choice is the remaining option: F S F $120 $150 Corollary A rational agent shoul not choose actions hich are not preference maximal; i.e., they shoul choose only actions that are preference maximal.

7 Outline 1 Decision problem classes 2 Decisions uner certainty 3 Decisions uner uncertainty Example (Uncertain school fun-raising) Procees of the school fun-raiser epen on the eather; on a ry ay () the school expects to make $150 for a fête (F) but only $120 for a sports ay (S). Hoever, on a et ay () the sports ay ill net $85 an the fête only $75. S F S ay is ry F ay is et 75

8 Decisions uner uncertainty Problem Ho to assign values to uncertain actions? S F Ho shoul V (S) an V (F) be efine? Ho shoul V (over actions) epen on v (over outcomes)? Lotteries Definition (Lottery) A lottery over a finite set of states S, an outcomes, or prizes, Ω, is a function l : S Ω. The lottery l is ritten: here for each s i S, ω i = l(s i ). l = [s 1 : ω 1 s 2 : ω 2... s n : ω n ] Example (Dry or et?) The uncertain situation in hich the eather on a given future ay is unknon represente by the lottery: l S = [ : $120 : $85] $120 $85

9 Decision problems an lotteries [s 1 : ω 1 s 2 : ω 2... s n : ω n ] Each uncertain action correspons to a lottery Choosing an action correspons to choosing among the lotteries on offer Problem The problem of evaluating actions amounts to the problem of etermining ho to compare an/or evaluate lotteries. s 1 s 2. s n. ω 1 ω 2 ω n Decisions uner uncertainty: ignorance Example (Raffle) There are four raffle tickets in a hat. Each ticket is either blue or re, but you on t kno ho many of each there are. Blue tickets in $3; re ones lose ($0). The cost of entering the raffle is $1. Exercises Dra the ecision tree an table for this problem Shoul you ra a ticket in the raffle? What if you kne there ere three blue tickets? Four? None? Ho many blue tickets oul there have to be to make it orth entering? If there ere n blue tickets (0 n 4), hat oul the prize have to be to make it orthhile entering?

10 Definition (Decision rule) A ecision rule is a ay of choosing, for each ecision problem, an action or set of actions. Rational ecision rules uner ignorance: Optimistic MaxiMax rule: nothing venture, nothing gaine Wal s pessimistic Maximin rule: its better to be safe than sorry Huricz s mixe optimistic-pessimistic rule: use an optimism inex α Savage s minimax Regret rule; least opportunity loss Laplace s principle of insufficient reason rule MaxiMax MaxiMax associates ith each action the state hich yiels the most preferre outcome (i.e., preference maximal) The MaxiMax ecision rule selects the action(s) hich yiel a preference-maximal outcome among these

11 Decisions uner uncertainty Example (Uncertain school fun-raising) Procees of the school fun-raiser epen on the eather; on a ry ay () the school expects to make $150 for a fête (F) but only $120 for a sports ay (S). Hoever, on a et ay () the sports ay ill net $85 an the fête only $75. S F S ay is ry F ay is et 75 MaxiMax (MM): aim for the best s 1 s 2 s 3 V A A A For each action fin the best possible outcome of all possible cases/states; i.e., for each ro fin the maximum value: V MM (A) = M(A) = max{v(ω(a, s)) s S} Choose the actions/ros ith maximal value: A 1 Equivalently: fin the maximum value of the entire table, choose the ro/action ith this value r MM (ω) = arg max{m(a) A A}

12 MaxiMax s 1 s 2 A 10 0 B 9 9 Which action is better? Ho coul ties be broken? s 1 s 2 s 3 V A A A Risk attitues MaxiMax is a ecision rules for extreme risk takers Some agents may prefer risks if the favourable outcomes are sufficiently esirable MaxiMax oul be a rational ecision rule ecision-makers ith risk-taking attitues/preferences In many cases it is ise to be risk averse (islike risk): i.e., avoi, reuce, or protect against risk What might a risk averse ecision rule look like?

13 Decisions uner uncertainty Example (Uncertain school fun-raising) Procees of the school fun-raiser epen on the eather; on a ry ay () the school expects to make $150 for a fête (F) but only $120 for a sports ay (S). Hoever, on a et ay () the sports ay ill net $85 an the fête only $75. S F S ay is ry F ay is et 75 Maximin (Mm): best in the orst case Assume the orst case/state ill occur for each action s 1 s 2 s 3 V A A A For each action fin the orst possible outcome uner all possible cases/states; i.e., for each ro fin the minimum value: V (A) = m(a) = min{v(ω(a, s)) s S} m(a) is sometimes calle the security level of action A Choose the action/ro ith the maximum of these: A 3 r Mm (ω) = arg max{m(a) A A}

14 Maximin s 1 s 2 A 10 0 B 1 1 Which action is better? Ho coul ties be broken? s 1 s 2 s 3 V A A A Huricz s optimism inex s 1 s 2 s 3 M m αm + (1 α)m A A A For each action/ro, fin the minimum (m) an maximum (M) values Calculate a eighte sum base on the optimism inex α (e.g., α = 3 4 ); i.e., V (A) = αm(a) + (1 α)m(a) = 3 4 M m. Choose the ro/action that maximises this value: A 1 Exercise What happens hen α = 1? α = 0?

15 MaxiMax an Maximin s 1 s 2 A B Compare problems above: s 1 s 2 A B 5 5 s 1 s 2 A a 1 a 2 B b 1 b 2 MaxiMax an Maximin choose the same action for any values of a 1, a 2, b 1, b 2, provie a 1 > b 1 b 2 > a 2 is preserve; since M(A) = a 1 > M(B), m(b) > a 2 = m(a) remain unchange; i.e., the actual numbers are irrelevant for the rules In this case the ifferences a 1 b 1 an b 2 a 2 are irrelevant provie M(A) > M(B) an m(b) > m(a) Decisions uner uncertainty Example (Uncertain school fun-raising) Procees of the school fun-raiser epen on the eather; on a ry ay () the school expects to make $150 for a fête (F) but only $120 for a sports ay (S). Hoever, on a et ay () the sports ay ill net $85 an the fête only $75. S F S ay is ry F ay is et 75

16 Regret Definition (Regret) The regret, or opportunity loss, of an outcome in a given state is the ifference beteen the outcome s value an that of the best possible outcome for that state. Consier the fun-raising problem iscusse earlier: S F M s M R S F The maximum regret for the sports ay is 30 but only 10 for the fête The action hich minimises the maximum regret is F minimax Regret s 1 s 2 s 3 A A A s 1 s 2 s 3 V A A A For each column/state s, fin its maximum value (M s ) Construct the regret table: R(ω) = M s v(ω) For each action/ro fin the maximum regret: V (A) = max{r(ω(a, s)) s S} Choose the ro/action that minimises the regret: A 3

17 Laplace s insufficient reason s 1 s 2 s 3 V A = A = A = 3 Assume each state is equally likely For each ro/action calculate the mean value: V (A) = 1 n v(ω(a, s 1)) n v(ω(a, s n)) Choose the ro/action ith maximum value: A 1 Exercise Ho coul you simplify this ecision rule? Decision rules s 1 s 2 s 3 s 4 V A A A A For a value function V on actions, a ecision rule r V is efine by: r V (p) = arg max{v (A) A A} On the ecision problem above, hich rules agree (i.e., choose the same actions)? V MM (A) = max{v(ω(a, s)) s S} V Mm (A) = min{v(ω(a, s)) s S} V mmr (A) = max{r(a, s) s S}