Value of Information in Spreadsheet Monte Carlo Simulation Models

Transcription

1 Value of Information in Spreadsheet Monte Carlo Simulation Models INFORMS 010 Austin Michael R. Middleton, Ph.D. Decision Toolworks

2 Background Spreadsheet models are often used for what if sensitivity analysis Probability distributions may be assigned to input assumptions, and Monte Carlo simulation can develop the corresponding distribution for the output performance measure Problem: How can we compute Value of Information using the results of Monte Carlo simulation in a spreadsheet model? INFORMS 010 Austin

3 Value of Information Expected Value of Perfect Information, EVPI Value of Clairvoyance, Value of Information, VOI Important for determining an upper bound on the value of actual information gathering activities comparing the value of information for multiple uncertainties Requires a model of a decision problem with alternatives, probabilities for uncertainties, outcome values, and willingness to determine certain equivalents Monte Carlo simulation may be used to describe uncertainty associated with a single alternative or strategy INFORMS 010 Austin

4 Why is Value of Information important? Answer #1 Felliand Hazen, 1999 In realistically sized problems, simple one and two way SAs may not fully capture parameter interactions, raising the disturbing possibility that many published decision analyses might be overconfident in their policy recommendations. we re examined 5 decision analyses While we expected EVPI values to indicate greater problem sensitivity than conventional SA due to revealed parameter interaction, we in fact found the opposite: compared to EVPI, the one and two parameter SAs accompanying these problems dramatically overestimated problem sensitivity to input parameters. INFORMS 010 Austin

5 Why is Value of Information important? Answer # Hubbard (009) computed EVPI on more than 60 large decision models with a total of over,000 uncertain input variables calculations of information values justify some empirical measurement of uncertain values most of the time. a VBA macro in Excel was used to slice the continuous distribution into thousands of discrete units, computing EVPI for each unit, and totaling the unit EVPIs. Here I show how to compute EVPI for the results of Monte Carlo simulation in Excel without using a VBA macro. INFORMS 010 Austin 5

6 Terminology Risk neutral certain equivalent Probability weighted average, Expected Value, EV EVUU, Expected Value Under Uncertainty EVPP, Expected Value with Perfect Prediction EVPI, Expected Value of Perfect Information EVPI = EVPP EVUU INFORMS 010 Austin 6

7 Computing EVPI by rearranging a decision tree EVPI(Units Sold) = EVPP(Units Sold) EVUU INFORMS 010 Austin 7

8 Simple spreadsheet model with base case and extreme inputs Software project Predetermined Unit Price, constant for these analyses Judgments about Units Sold, Unit Variable Cost, and Fixed Costs Other alternative (not shown) yields $80, A B C D E F G H I Input Cells Input Assumptions Scenarios Unit Price $9 Minimum Base Case Maximum Worst Best Units Sold Unit Variable Cost $.00 $.00 $.00 $7.00 $7.00 $.00 Fixed Costs $,000 $,000 $,000 $6,000 $6,000 $,000 Net Cash Flow $86,000 $57,000 $11, A B C D E F G H I Input Cells Scenarios Unit Price 9 Minimum Base Case Maximum Worst Best Units Sold Unit Variable Cost 7 7 Fixed Costs Net Cash Flow =B*($B$ B5) B6 =H*($B$ H5) H6 =I*($B$ I5) I6 INFORMS 010 Austin 8

9 Tornado chart for simple spreadsheet model Single factor sensitivity analysis Net Cash Flow Corresponding Input Value Output Value Percent Input Variable Low Output Base Case High Output Low Base High Swing Swing^ Units Sold $6,500 $86,000 $108,500 $5, % Unit Variable Cost $7.00 $.00 $.00 $80,000 $86,000 $88,000 $8,000.0% Fixed Costs $6,000 $,000 $,000 $8,000 $86,000 $88,000 $, % SensIt 1.5 Units Sold Unit Variable Cost $7.00 $.00 Fixed Costs $6,000 $,000 $50,000 $60,000 $70,000 $80,000 $90,000 $100,000 $110,000 $10,000 Net Cash Flow INFORMS 010 Austin 9

10 Simple spreadsheet model with random inputs A B C D E F G H I Unit Price $9 RAND() Type Units Sold Normal Mean=,000 StDev=00 Unit Variable Cost $ Triangular Min=$ Mode=$ Max=$7 Fixed Costs $, Discrete Lookup Value Probability 0.0 $, Net Cash Flow $7, $, $, $5, $6, A B C D Unit Price 9 RAND() Units Sold =ROUND(NORMINV(D,000,00),0) =RAND() Unit Variable Cost =ROUND(IF(D5<(-)/(7-),+(D5*(7-)*(-))^0.5,7-((1-D5)*(7-)*(7-))^0.5),) =RAND() Fixed Costs =VLOOKUP(D6,G7:H11,,TRUE) =RAND() Net Cash Flow =ROUND(B*(B B5) B6,0) INFORMS 010 Austin 10

11 Setting up an Excel data table for Monte Carlo simulation 1 5 A B C D E F Trial Unit Price Units Sold Unit Variable Cost Fixed Costs Net Cash Flow ='RAND Model'!B ='RAND Model'!B ='RAND Model'!B5 ='RAND Model'!B6 ='RAND Model'!B A B C D E F Trial Unit Price Units Sold Unit Variable Cost Fixed Costs Net Cash Flow $ $5.75 $,000 $81,851 1 For 10,000 trials, select A:F1000. In Excel 007 & 010 choose Data > What If Analysis > Data Table INFORMS 010 Austin 11

12 Results of an Excel data table for Monte Carlo simulation 1 5 A B C D E F Trial Unit Price Units Sold Unit Variable Cost Fixed Costs Net Cash Flow ='RAND Model'!B ='RAND Model'!B ='RAND Model'!B5 ='RAND Model'!B6 ='RAND Model'!B8 1 =TABLE(,A) =TABLE(,A) =TABLE(,A) =TABLE(,A) =TABLE(,A) =TABLE(,A) =TABLE(,A) =TABLE(,A) =TABLE(,A) =TABLE(,A) =TABLE(,A) =TABLE(,A) =TABLE(,A) =TABLE(,A) =TABLE(,A) 1 5 A B C D E F Trial Unit Price Units Sold Unit Variable Cost Fixed Costs Net Cash Flow $9 096 $5.9 $,000 $89,197 1 $9 15 $.1 $,000 $9,560 $9 09 $.91 $,000 $88,99 $9 077 $5.1 $6,000 $85,19 Data table is volatile. For subsequent analysis, copy and paste special values. INFORMS 010 Austin 1

13 Frequency Better Histogram Units Sold Frequency Better Histogram 0 $.50 $.00 $.50 $.00 $.50 $5.00 $5.50 $6.00 $6.50 $7.00 $7.50 Unit Variable Cost Better Histogram Better Histogram EV = $8,908 Frequency $1,000 $,000 $,000 $,000 $5,000 $6,000 $7,000 Fixed Costs Frequency $0,000 $60,000 $80,000 $100,000 $10,000 Net Cash Flow INFORMS 010 Austin 1

14 Decision tree analogy for Monte Carlo simulation EVPI Approx. EVPI(Units Sold) = Approx. EVPP(Units Sold) EVUU INFORMS 010 Austin 1

15 To determine EVPP(Units Sold), first Copy the Units Sold and Net Cash Flow data to another sheet 1 A B C Trial Units Sold Net Cash Flow 1 15 $9, $88, $85,19 Sort the simulation results with Units Sold as key 1 A B C Trial Units Sold Net Cash Flow $1, $10, $118,81 INFORMS 010 Austin 15

16 Compute Conditional EV(Sim) for each bracket Syntax: OFFSET(reference, rows, cols, [height], [width]) 100 brackets * 100 trials each = 10,000 trials total INFORMS 010 Austin 16

17 Compute Conditional EVPP for each bracket INFORMS 010 Austin 17

18 Compute EVPP and EVPI INFORMS 010 Austin 18

19 EVPI(Units Sold) for values of the other alternative Set up an Excel Data Table $60K to $110K, step $5K Formula for output: =J9 Select L:M1 Data > What If Analysis > Data Table I J K L M EV(Sim) $8,908 EV(Alt) EVPI =J9 EV(Alt) $80,000 $60,000 $65,000 EVUU $8,908 $70,000 $75,000 EVPP $86,58 $80,000 $85,000 EVPI $1,60 $90,000 $95,000 $100,000 $105,000 $110, I J K L M EV(Sim) $8,908 EV(Alt) EVPI $1,60 EV(Alt) $80,000 $60,000 $0 $65,000 $ EVUU $8,908 $70,000 $178 $75,000 $59 EVPP $86,58 $80,000 $1,60 $85,000 $,5 EVPI $1,60 $90,000 $1,58 $95,000 $569 $100,000 $156 $105,000 $ $110,000 $0 INFORMS 010 Austin 19

20 Three EVPIs for values of the other alternative $,000 $,500 $,000 $,500 Value of Information Units Sold Unit Variable Cost Fixed Costs EVPI $,000 $1,500 $1,000 $500 $0 $50,000 $60,000 $70,000 $80,000 $90,000 $100,000 $110,000 $10,000 EV(Alt) INFORMS 010 Austin 0

21 Excel features used in this project Worksheet functions in what if model RAND, NORMINV, IF, VLOOKUP, ROUND Monte Carlo simulation Data Table EVPI calculations Data Sort, AVERAGE, OFFSET, MAX Diagrams and charts Drawing tools, XY charts Better Histogram add in INFORMS 010 Austin 1

22 References Felli, J.C., Hazen, G.B Do sensitivity analyses really capture problem sensitivity? An empirical analysis based on information value. Risk, Decision and Policy () Accessed June 8, 010, ensitivity.pdf Hubbard, D., Samuelson, D.A Modeling Without Measurements: How the decision analysis culture s lack of empiricism reduces its effectiveness. OR/MS Today 6(5) 6 1. Accessed June 8, 010, /risk.html Middleton, M.R Risk Analysis and the Expected Value of Perfect Information. Proceedings, American Institute for Decision Sciences, Western Regional Conference. Accessed June 8, 010, INFORMS 010 Austin

23 Value of Information in Spreadsheet Monte Carlo Simulation Models Michael R. Middleton, Ph.D. Decision Toolworks INFORMS 010 Austin PowerPoint Slides, Slides PDF File, and Excel Workbook