Decision Making Under Uncertainty

Transcription

1 CSC384: Intro to Artificil Intelligence Preferences Decision Mking Under Uncertinty Decision Trees DBN: 15.1 nd 15.5 Decision Network: 16.1,16.2,16.5,16.6 I give root plnning prolem: I wnt coffee ut coffee mker is roken: root reports No pln! 1 2 Preferences Preference Orderings We relly wnt more roust ehvior. Root to know wht to do if my primry gol cn t e stisfied I should provide it with some indiction of my preferences over lterntives e.g., coffee etter thn te, te etter thn wter, wter etter thn nothing, etc. But it s more complex: it could wit 45 minutes for coffee mker to e fixed wht s etter: te now? coffee in 45 minutes? could express preferences for <everge,time> pirs A preference ordering is rnking of ll possile sttes of ffirs (worlds) S these could e outcomes of ctions, truth ssts, sttes in serch prolem, etc. s t: mens tht stte s is t lest s good s t s t: mens tht stte s is strictly preferred to t We insist tht is reflexive: i.e., s s for ll sttes s trnsitive: i.e., if s t nd t w, then s w connected: for ll sttes s,t, either s t or t s 3 4

2 Why Impose These Conditions? Decision Prolems: Certinty Structure of preference ordering imposes certin rtionlity requirements (it is wek ordering) E.g., why trnsitivity? Suppose you (strictly) prefer coffee to te, te to OJ, OJ to coffee If you prefer X to Y, you ll trde me Y plus $1 for X I cn construct money pump nd extrct ritrry mounts of money from you Best Worst A decision prolem under certinty is: set of decisions D e.g., pths in serch grph, plns, ctions set of outcomes or sttes S e.g., sttes you could rech y executing pln n outcome function f : D S the outcome of ny decision preference ordering over S A solution to decision prolem is ny d* D such tht f(d*) f(d) for ll d D 5 6 Decision Prolems: Certinty Decision Mking under Uncertinty A decision prolem under certinty is: set of decisions D set of outcomes or sttes S n outcome function f : D S preference ordering over S A solution to decision prolem is ny d* D such tht f(d*) f(d) for ll d D e.g., in clssicl plnning we tht ny gol stte s is preferred/equl to every other stte. So d* is solution iff f(d*) is solution stte. I.e., d* is solution iff it is pln tht chieves the gol. More generlly, in clssicl plnning we might consider different gols with different vlues, nd we wnt d* to e pln tht optimizes our vlue. getcoffee.8.2 chc, mess chc, mess donothing chc, mess Suppose ctions don t hve deterministic outcomes e.g., when root pours coffee, it spills 20% of time, mking mess preferences: chc, mess chc, mess chc, mess Wht should root do? decision getcoffee leds to good outcome nd d outcome with some proility decision donothing leds to medium outcome for sure Should root e optimistic? pessimistic? Relly odds of success should influence decision ut how? 7 8

3 Utilities Expected Utility Rther thn just rnking outcomes, we must quntify our degree of preference e.g., how much more importnt is hving coffee thn hving te? A utility function U: S R ssocites relvlued utility with ech outcome (stte). U(s) quntifies our degree of preference for s Note: U induces preference ordering U over the sttes S defined s: s U t iff U(s) U(t) U is reflexive, trnsitive, connected With utilities we cn compute expected utilities! In decision mking under uncertinty, ech decision d induces distriution Pr d over possile outcomes Pr d (s) is proility of outcome s under decision d The expected utility of decision d is defined s S EU ( d) = Pr ( s) U ( s) d 9 10 Expected Utility The MEU Principle Sy U(chc, ms) = 10, U( chc, ms) = 5, U( chc,ms) = 0, Then EU(getcoffee) = 8 EU(donothing) = 5 If U(chc, ms) = 10, U( chc, ms) = 9, U( chc,ms) = 0, EU(getcoffee) = 8 EU(donothing) = 9 The principle of mximum expected utility (MEU) sttes tht the optiml decision under conditions of uncertinty is the decision tht hs gretest expected utility. In our exmple if my utility function is the first one, my root should get coffee if your utility function is the second one, your root should do nothing 11 12

4 Computtionl Issues Decision Prolems: Uncertinty At some level, the solution to decision prolem is trivil, however: complexity lies in the fct tht the decisions nd outcome function re rrely specified explicitly e.g., in plnning or serch prolem, you construct the set of decisions y constructing pths or exploring serch pths. Then we hve to evlute the expected utility of ech. Computtionlly hrd! e.g., we find pln chieving some expected utility e Cn we stop serching? Must convince ourselves no etter pln exists Generlly requires serching entire pln spce, unless we hve some clever tricks A decision prolem under uncertinty is: set of decisions D set of outcomes or sttes S n outcome function Pr : D (S) (S) is the set of distriutions over S (e.g., Pr d ) utility function U over S A solution to decision prolem under uncertinty is ny d* D such tht EU(d*) EU(d) for ll d D Expected Utility: Notes Expected Utility: Notes Note tht this viewpoint ccounts for oth: uncertinty in ction outcomes uncertinty in stte of knowledge ny comintion of the two s s1 s2 0.3 s3 0.7 s4 Stochstic ctions 0.7 s1 0.3 s2 0.7 t1 0.3 t2 0.7 w1 0.3 w2 Uncertin knowledge Why MEU? Where do utilities come from? underlying foundtions of utility theory tightly couple utility with ction/choice utility function cn e determined y sking someone out their preferences for ctions in specific scenrios (or lotteries over outcomes) Utility functions needn t e unique if I multiply U y positive constnt, ll decisions hve sme reltive utility if I dd constnt to U, sme thing U is unique up to positive ffine trnsformtion 15 16

5 So Wht re the Complictions? Outcome spce is lrge like ll of our prolems, sttes spces cn e huge don t wnt to spell out distriutions like Pr d explicitly Soln: Byes nets (or relted: influence digrms) So Wht re the Complictions? Decision spce is lrge usully our decisions re not one-shot ctions rther they involve sequentil choices (like plns) if we tret ech pln s distinct decision, decision spce is too lrge to hndle directly Soln: use dynmic progrmming methods to construct optiml plns (ctully generliztions of plns, clled policies like in gme trees) An Simple Exmple Distriutions for Action Sequences Suppose we hve two ctions:, We hve time to execute two ctions in sequence This mens we cn do either: [,], [,], [,], [,] Actions re stochstic: ction induces distriution Pr (s i s j ) over sttes e.g., Pr (s 2 s 1 ) =.9 mens pro. of moving to stte s 2 when is performed t s 1 is.9 similr distriution for ction How good is prticulr sequence of ctions? s s2 s3 s12 s s4 s5 s6 s7 s8 s9 s10 s11 s14 s15 s16 s17 s18 s19 s20 s

6 Distriutions for Action Sequences.5.5 s4 s5 s2 s6 Sequence [,] gives distriution over finl sttes Pr(s4) =.45, Pr(s5) =.45, Pr(s8) =.02, Pr(s9) =.08 Similrly: s3 s s7.2.8 s8 s s10 s11 s14 [,]: Pr(s6) =.54, Pr(s7) =.36, Pr(s10) =.07, Pr(s11) =.03 nd similr distriutions for sequences [,] nd [,] s15 s s16 s s18 s19 s s20 s21 21 How Good is Sequence? We ssocite utilities with the finl outcomes how good is it to end up t s4, s5, s6, Now we hve: EU() =.45u(s4) +.45u(s5) +.02u(s8) +.08u(s9) EU() =.54u(s6) +.36u(s7) +.07u(s10) +.03u(s11) etc 22 Utilities for Action Sequences Action Sequences re not sufficient s2 u(s4) u(s5) u(s6) s3 s etc. s12 s13 u(s21) Looks lot like gme tree, ut with chnce nodes insted of min nodes. (We verge insted of minimizing) 23 s2 s3 s s s4 s5 s6 s7 s8 s9 s10 s11 s14 s15 s16 s17 Suppose we do first; we could rech s2 or s3: At s2, ssume: EU() =.5u(s4) +.5u(s5) > EU() =.6u(s6) +.4u(s7) At s3: EU() =.2u(s8) +.8u(s9) < EU() =.7u(s10) +.3u(s11) After doing first, we wnt to do next if we rech s2, ut we wnt to do second if we rech s3 s s18 s19 s20 s21 24

7 Policies This suggests tht when deling with uncertinty we wnt to consider policies, not just sequences of ctions (plns) We hve eight policies for this decision tree: [; if s2, if s3 ] [; if s12, if s13 ] [; if s2, if s3 ] [; if s12, if s13 ] [; if s2, if s3 ] [; if s12, if s13 ] [; if s2, if s3 ] [; if s12, if s13 ] Contrst this with four plns [; ], [; ], [; ], [; ] note: ech pln corresponds to policy, so we cn only gin y llowing decision mker to use policies Evluting Policies Numer of plns (sequences) of length k exponentil in k: A k if A is our ction set Numer of policies is even lrger if we hve n= A ctions nd m= O outcomes per ction, then we hve (nm)k policies Fortuntely, dynmic progrmming cn e used e.g., suppose EU() > EU() t s2 never consider policy tht does nything else t s2 How to do this? ck vlues up the tree much like minimx serch Decision Trees Squres denote choice nodes these denote ction choices y decision mker (decision nodes) Circles denote chnce nodes these denote uncertinty regrding ction effects Nture will choose the child with specified proility Terminl nodes leled with utilities denote utility of finl stte (or it could denote the utility of trjectory (rnch) to decision mker s1 Evluting Decision Trees Procedure is exctly like gme trees, except key difference: the opponent is nture who simply chooses outcomes t chnce nodes with specified proility: so we tke expecttions insted of minimizing Bck vlues up the tree U(t) is defined for ll terminls (prt of input) U(n) = exp {U(c) : c child of n} if n is chnce node U(n) = mx {U(c) : c child of n} if n is choice node At ny choice node (stte), the decision mker chooses ction tht leds to highest utility child 27 28

8 Evluting Decision Tree U(n3) =.9*5 +.1*2 s1 U(n4) =.8*3 +.2*4 U(s2) = mx{u(n3), U(n4)} n1 n2 decision or (whichever is mx).3.7 U(n1) =.3U(s2) +.7U(s3) s2 s3 U(s1) = mx{u(n1), U(n2)} decision: mx of, n3 n Decision Tree Policies Note tht we don t just compute vlues, ut policies for the tree A policy ssigns decision to ech choice node in tree Some policies cn t e distinguished in terms of their expected vlues e.g., if policy chooses t node s1, choice t s4 doesn t mtter ecuse it won t e reched Two policies re implementtionlly indistinguishle if they disgree only t unrechle decision nodes rechility is determined y policy themselves n3 s2 n1.3.7 n4 s3 s1 s4 n2 30 Key Assumption: Oservility Full oservility: we must know the initil stte nd outcome of ech ction specificlly, to implement the policy, we must e le to resolve the uncertinty of ny chnce node tht is followed y decision node e.g., fter doing t s1, we must know which of the outcomes (s2 or s3) ws relized so we know wht ction to do next (note: s2 nd s3 my prescrie different tions) Computtionl Issues Svings compred to explicit policy evlution is sustntil Evlute only O((nm)d ) nodes in tree of depth d totl computtionl cost is thus O((nm)d ) Note tht this is how mny policies there re ut evluting single policy explicitly requires sustntil computtion: O(nmd ) totl computtion for explicity evluting ech policy would e O(ndm2d )!!! Tremendous vlue to dynmic progrmming solution 31 32

9 Computtionl Issues Tree size: grows exponentilly with depth Possile solutions: ounded lookhed with heuristics (like gme trees) heuristic serch procedures (like A*) Other Issues Specifiction: suppose ech stte is n ssignment to vriles; then representing ction proility distriutions is complex (nd rnching fctor could e immense) Possile solutions: represent distriution using Byes nets solve prolems using decision networks (or influence digrms) Lrge Stte Spces (Vriles) Exmple Action using Dynmic BN (15.1 nd 15.5) To represent outcomes of ctions or decisions, we need to specify distriutions Pr(s d) : proility of outcome s given decision d Pr(s,s ): pro. of stte s given tht ction performed in stte s But stte spce exponentil in # of vriles spelling out distriutions explicitly is intrctle Byes nets cn e used to represent ctions this is just joint distriution over vriles, conditioned on ction/decision nd previous stte Deliver Coffee ction Mt Mt+1 Tt Tt+1 Lt Lt+1 Ct Ct+1 Rt Rt+1 M mil witing C Crig hs coffee T l tidy R root hs coffee L root locted in Crig s office T T(t+1) T(t+1) T F L R C C(t+1) C(t+1) T T T F T T T F T F F T T T F F T F T F F F F F f J (Tt,Tt+1) f R (Lt,Rt,Ct,Ct+1) 35 36

10 Dynmic BN Action Representtion Dynmic BN Action Representtion Dynmic Byesin networks (DBNs): wy to use BNs to represent specific ctions list ll stte vriles for time t (pre-ction) list ll stte vriles for time t+1 (post-ction) indicte prents of ll t+1 vriles these cn include time t nd time t+1 vriles network must e cyclic specify CPT for ech time t+1 vrile Note: generlly no prior given for time t vriles we re (generlly) interested in conditionl distriution over post-ction sttes given prection stte so time t vrs re instntited s evidence when using DBN (generlly) Exmple of Dependence within Slice Use of BN Action Reprsnt n Throw rock t window ction DBNs: ctions concisely,nturlly specified These look it like STRIPS nd the situttion clculus, ut llow for proilistic effects Alrmt Alrmt+1 P(lt+1 lt, rt) = 1 P(lt+1 lt, rt+1) = 0 P(lt+1 lt,rt+1) =.95 Brokent Brokent+1 P(rokent+1 rokent) = 1 P(rokent+1 rokent) =.6 Throwing rock hs certin proility of reking window nd setting off lrm; ut whether lrm is triggered depends on whether rock ctully roke the window

11 Use of BN Action Representtion How to use: use to generte expectimx serch tree to solve decision prolems use directly in stochstic decision mking lgorithms First use doesn t uy us much computtionlly when solving decision prolems. But second use llows us to compute expected utilities without enumerting the outcome spce (tree) We will see something like this with decision networks Decision Networks Decision networks (more commonly known s influence digrms) provide wy of representing sequentil decision prolems sic ide: represent the vriles in the prolem s you would in BN dd decision vriles vriles tht you control dd utility vriles how good different sttes re Smple Decision Network Decision Networks: Chnce Nodes Chills TstResult Chnce nodes rndom vriles, denoted y circles s in BN, proilistic dependence on prents Disese Fever BloodTst optionl U Drug Disese Pr(flu) =.3 Pr(ml) =.1 Pr(none) =.6 Pr(f flu) =.5 Pr(f ml) =.3 Pr(f none) =.05 Fever BloodTst TstResult Pr(pos flu,t) =.2 Pr(neg flu,t) =.8 Pr(null flu,t) = 0 Pr(pos ml,t) =.9 Pr(neg ml,t) =.1 Pr(null ml,t) = 0 Pr(pos no,t) =.1 Pr(neg no,t) =.9 Pr(null no,t) = 0 Pr(pos D, t) = 0 Pr(neg D, t) = 0 Pr(null D, t) =

12 Decision Networks: Decision Nodes Decision Networks: Vlue Node Decision nodes vriles decision mker sets, denoted y squres prents reflect informtion ville t time decision is to e mde In exmple decision node: the ctul vlues of Ch nd Fev will e oserved efore the decision to tke test must e mde gent cn mke different decisions for ech instntition of prents Chills Fever BloodTst BT {t, t} 45 Vlue node specifies utility of stte, denoted y dimond utility depends only on stte of prents of vlue node generlly: only one vlue node in decision network Utility depends only on disese nd drug Disese BloodTst optionl U Drug U(fludrug, flu) = 20 U(fludrug, ml) = -300 U(fludrug, none) = -5 U(mldrug, flu) = -30 U(mldrug, ml) = 10 U(mldrug, none) = -20 U(no drug, flu) = -10 U(no drug, ml) = -285 U(no drug, none) = Decision Networks: Assumptions Decision nodes re totlly ordered decision vriles D 1, D 2,, D n decisions re mde in sequence e.g., BloodTst (yes,no) decided efore Drug (fd,md,no) Decision Networks: Assumptions No-forgetting property ny informtion ville when decision D i is mde is ville when decision D j is mde (for i < j) thus ll prents of D i re prents of D j Network does not show these implicit prents, ut the links re present, nd must e considered when specifying the network prmeters, nd when computing. Chills Fever BloodTst Drug Dshed rcs ensure the no-forgetting property 47 48

13 Policies Let Pr(D i ) e the prents of decision node D i Dom(Pr(D i )) is the set of ssignments to prents A policy δ is set of mppings δ i, one for ech decision node D i δ i :Dom(Pr(D i )) Dom(D i ) δ i ssocites decision with ech prent sst for D i For exmple, policy for BT might e: δ BT (c,f) = t δ BT (c, f) = t Chills BloodTst δ BT ( c,f) = t δ BT ( c, f) = t Fever Vlue of Policy Vlue of policy δ is the expected utility given tht decision nodes re executed ccording to δ Given sst x to the set X of ll chnce vriles, let δ(x) denote the sst to decision vriles dictted y δ e.g., sst to D 1 determined y it s prents sst in x e.g., sst to D 2 determined y it s prents sst in x long with whtever ws ssigned to D 1 etc. Vlue of δ : EU(δ) = Σ X P(X, δ(x)) U(X, δ(x)) Optiml Policies An optiml policy is policy δ* such tht EU(δ*) EU(δ) for ll policies δ We cn use the dynmic progrmming principle to void enumerting ll policies We cn lso use the structure of the decision network to use vrile elimintion to id in the computtion Computing the Best Policy We cn work ckwrds s follows First compute optiml policy for Drug (lst decision) for ech sst to prents (C,F,BT,TR) nd for ech decision vlue (D = md,fd,none), compute the expected vlue of choosing tht vlue of D set policy choice for ech vlue of prents to e the vlue of D tht TstResult hs mx vlue Chills BloodTst eg: δ D (c,f,t,pos) = md Disese Fever optionl U Drug 51 52

14 Computing the Best Policy Next compute policy for BT given policy δ D (C,F,BT,TR) just determined for Drug since δ D (C,F,BT,TR) is fixed, we cn tret Drug s norml rndom vrile with deterministic proilities i.e., for ny instntition of prents, vlue of Drug is fixed y policy δ D this mens we cn solve for optiml policy for BT just s efore only uninstntited vrs re rndom vrs (once we fix its prents) Computing the Best Policy How do we compute these expected vlues? suppose we hve sst <c,f,t,pos> to prents of Drug we wnt to compute EU of deciding to set Drug = md we cn run vrile elimintion! Tret C,F,BT,TR,Dr s evidence this reduces fctors (e.g., U restricted to t,md: depends on Dis) eliminte remining vriles (e.g., only Disese left) left with fctor:u() = Σ Dis P(Dis c,f,t,pos,md)u(dis) Disese Chills Fever BloodTst TstResult optionl U Drug Computing the Best Policy Computing Expected Utilities We now know EU of doing Dr=md when c,f,t,pos true Cn do sme for fd,no to decide which is est Disese Chills Fever BloodTst TstResult optionl U Drug The preceding illustrtes generl phenomenon computing expected utilities with BNs is quite esy utility nodes re just fctors tht cn e delt with using vrile elimintion EU = Σ A,B,C P(A,B,C) U(B,C) = Σ A,B,C P(C B) P(B A) P(A) U(B,C) Just eliminte vriles in the usul wy A C B U 55 56

15 Optimizing Policies: Key Points Optimizing Policies: Key Points If decision node D hs no decisions tht follow it, we cn find its policy y instntiting ech of its prents nd computing the expected utility of ech decision for ech prent instntition no-forgetting mens tht ll other decisions re instntited (they must e prents) its esy to compute the expected utility using VE the numer of computtions is quite lrge: we run expected utility clcultions (VE) for ech prent instntition together with ech possile decision D might llow policy: choose mx decision for ech prent instntition Optimizing Policies: Key Points Decision Network Notes When decision D node is optimized, it cn e treted s rndom vrile for ech instntition of its prents we now know wht vlue the decision should tke just tret policy s new CPT: for given prent instntition x, D gets δ(x) with proility 1(ll other decisions get proility zero) If we optimize from lst decision to first, t ech point we cn optimize specific decision y ( unch of) simple VE clcultions it s successor decisions (optimized) re just norml nodes in the BNs (with CPTs) Decision networks commonly used y decision nlysts to help structure decision prolems Much work put into computtionlly effective techniques to solve these Complexity much greter thn BN inference we need to solve numer of BN inference prolems one BN prolem for ech setting of decision node prents nd decision node vlue 59 60

16 Rel Estte Investment DBN-Decision Nets for Plnning Actt-2 Actt-1 Actt Mt-2 Mt-1 Mt Mt+1 Tt-2 Tt-1 Tt Tt+1 Lt-2 Lt-1 Lt Lt+1 U Ct-2 Ct-1 Ct Ct+1 Rt-2 Rt-1 Rt Rt A Detiled Decision Net Exmple A Detiled Decision Net Exmple Setting: you wnt to uy used cr, ut there s good chnce it is lemon (i.e., prone to rekdown). Before deciding to uy it, you cn tke it to mechnic for inspection. They will give you report on the cr, leling it either good or d. A good report is positively correlted with the cr eing sound, while d report is positively correlted with the cr eing lemon. However the report costs $50. So you could risk it, nd uy the cr without the report. Owning sound cr is etter thn hving no cr, which is etter thn owning lemon

17 Cr Buyer s Network l l Lemon Inspect Report U Rep: good,d,none g n l i l i l i l i Buy Utility l -600 l 1000 l -300 l if inspect Evlute Lst Decision: Buy (1) EU(B I,R) = Σ L P(L I,R,B)U(L,B) The proility of the remining vriles in the Utility function, times the utility function. Note P(L I,R,B) = P(L I,R), s B is decision vrile tht does not influence L. I = i, R = g: P(L i,g): use vrile elimintion. Query vrile L is only remining vrile, so we only need to normlize (no summtions). P(L,i,g) = P(L)P(g L,i) HENCE: P(L i,g) = normlized [P(l)P(g l,i),p( l)p(g l,i) = [0.5*.2, 0.5*0.9] = [.18,.82] EU(uy) = P(l i,g)u(uy,l) + P( l)p( l i,g) U(uy, l)-50 =.18* * = 662 EU( uy) = P(l i, g) U( uy,l) + P( l i, g) U( uy, l) 50 =.18* * = -350 So optiml δ Buy (i,g) = uy Evlute Lst Decision: Buy (2) Evlute Lst Decision: Buy (3) I = i, R = : P(L,i,) = P(L)P( L,i) P(L i,) = normlized [P(l)P( l,i),p( l)p( l,i) = [0.5*.8, 0.5*0.1] = [.89,.11] EU(uy) = P(l i, ) U(l,uy) + P( l i, ) U( l,uy) - 50 =.89* * = -474 EU( uy) = P(l i, ) U(l, uy) + P( l i, ) U( l, uy) 50 =.89* * = -350 So optiml δ Buy (i,) = uy I = i, R = n P(L, i,n) = P(L)P(n L, i) P(L i,n) = normlized [P(l)P(n l, i),p( l)p(n l, i) = [0.5*1, 0.5*1] = [.5,.5] EU(uy) = P(l i,n) U(l,uy) + P( l i,n) U( l,uy) =.5* *1000 = 200 (no inspection cost) EU( uy) = P(l i, n) U(l, uy) + P( l i, n) U( l, uy) =.5* *-300 = -300 So optiml δ Buy ( i,n) = uy Overll optiml policy for Buy is: δ Buy (i,g) = uy ; δ Buy (i,) = uy ; δ Buy ( i,n) = uy Note: we don t other computing policy for (i, n), ( i, g), or ( i, ), since these occur with proility

18 Evlute First Decision: Inspect Vlue of Informtion EU(I) = Σ L,R P(L,R I) U(L, δ Buy (I,R)) where P(R,L I) = P(R L,I)P(L I) g,l,l g, l, l P(R,L i) 0.2*.5 =.1 0.8*.5 =.4 0.9*.5 = *.5 =.05 δ Buy uy uy uy uy U( L, δ Buy ) = = = = -350 EU(i) =.1* * * * = = EU( i) = P(l i, n) U(l,uy) + P( l i, n) U( l,uy) =.5* *1000 = 200 So optiml δ Inspect ( i) = uy So optiml policy is: don t inspect, uy the cr EU = 200 Notice tht the EU of inspecting the cr, then uying it iff you get good report, is less the cost of the inspection (50). So inspection not worth the improvement in EU. But suppose inspection cost $25: then it would e worth it (EU = = > EU( i)) The expected vlue of informtion ssocited with inspection is 37.5 (it improves expected utility y this mount ignoring cost of inspection). How? Gives opportunity to chnge decision ( uy if d). You should e willing to py up to $37.5 for the report 69 70