TreeAge Pro 2 Day Healthcare Training Day 1

Transcription

1 TreeAge Pro 2 Day Healthcare Training Day 1 Using TreeAge Pro for Health Economic Modeling 2012 TreeAge Software, Inc. Agenda Day 1 Introduction Build Cost Effectiveness Model Analyze a Cost Effectiveness Model Sensitivity Analysis Exercise Build Decision Tree Markov Models TreeAge Pro Healthcare Training 2 1

2 Agenda Day 2 Markov Decisions and Time Dependence Heterogeneity and Event Tracking (Microsimulation) Sensitivity Analysis and Microsimulation Advanced Modeling Techniques TreeAge Pro Healthcare Training 3 Introduction Modeling/analysis goals Create a model to represent a disease process and available treatment options Evaluate treatment options independently and choose the optimal path Measure effects of uncertainty on treatment strategy selection TreeAge Pro Healthcare Training Introduction 4 2

3 Introduction Benefits of TreeAge Pro Visual modeling tool for easier model building and presentation Healthcare modeling analysis and reporting Sensitivity analysis Monte Carlo simulation Multiple measurements can be used to evaluate the same model Cost, effectiveness, cost effectiveness, other TreeAge Pro Healthcare Training Introduction 5 TreeAge Pro User Interface Tree Diagram Editor Perspectives Projects View Model Input Views TreeAge Pro Healthcare Training Introduction 6 3

4 TreeAge Pro Interface Tree Diagram Editor Primary modeling window for building model structure Multiple tabs for models and analysis output Zoom in/out, multi select, noteboxes Views For editing/viewing components of the model (e.g., parameters) Open via Views toolbar list Move, maximize, minimize, detach Perspectives Collection of views orientation stored when you exit software Can reset to original orientation or overwrite saved orientation TreeAge Pro Healthcare Training Introduction 7 TreeAge Pro Interface Model Input Views: Specific view(s) for each model input Some tied to tree (e.g., tree properties, variable properties) Some tied to node (e.g., node properties, variable definitions) Other Views: Projects View for managing files Model Overview/Tree Explorer for navigating in large trees Evaluator for testing calculations Context sensitive help Many more TreeAge Pro Healthcare Training Introduction 8 4

5 Build Cost Effectiveness Model Module 1: Build Cost Effectiveness Model Goals: Build model structure Create and use variables Set tree preferences Introduce clones 9 Build Cost Effectiveness Model Primary elements of a model Model structure Mimic everything that could happen Numeric values Add probabilities and values Tree Preferences Control how you want to evaluate/view the model 10 5

6 Model Structure Recreates the complete path of a patient Includes all possible events and outcomes Consists of a collection of nodes, where each one represents one step within the overall model flow Starts with a single root node Usually a decision node Facilitates comparison of treatment strategies Each node can have multiple branches to the right, but only a single parent 11 Node Types Decision Node: Branches are alternative strategies Run analyses here to compare strategies Chance Node: Branch for each possible outcome Probabilities are associated with each branch Mutually exclusive and exhaus ve ( = 100%) Use # for complementary probability (once) Terminal Node: Complete the scenario (root to terminal node) Payoffs place a value on all events within that scenario 12 6

7 Node Types Logic Node: Like a chance node but with logical expressions (true/false) instead of probabilities Expressions checked from top down until one is true Label Node: Like a chance node with a single branch with probability 100% Markov Node: Start of Markov model Will discuss in later module 13 Model Structure Decision Node Chance Nodes Terminal Nodes Root node Strategies Probabilities Payoffs 14 7

8 Numeric Values Required for probabilities, payoffs, etc. Consist of any combination of Numbers Model inputs: Variables, distributions, tables, trackers Built in functions: If, Discount, Min, Max, etc. Operators: +,, *, /, ^, &, TreeAge Pro cannot determine numeric values for your projects Sources: medical literature, trials, NIH, AHRQ, CDC Some must be estimated 15 Tree Preferences Control how a model is Calculated Calculation method Active, enabled payoff sets Optimal path min vs. max Etc. Displayed Show variables in tree Numeric formatting Fonts Etc. 16 8

9 Build Cost Effectiveness Model Model we will build The current standard is to treat a specific type of tumor with radiation We want to study a new treatment that combines surgery and radiation We estimate that the new treatment will increase the probability of eradicating the tumor from 60% to 80% A person s life expectancy is 10 years if the tumor is eradicated, but only 3 years if not The costs associated with radiation and surgery are $30K and $50K respectively The follow up costs post treatment are $2K per year 17 Build Cost Effectiveness Model How do we create a model from the information we have? Look for the decision Two treatment options indicates root decision node with branch for each strategy Possible outcomes Either treatment could eradicate the tumor or not Chance node with branches for two results Life expectancy provided Becomes measurement of effectiveness Markov model is not necessary Numbers Become parameters for probabilities or values 18 9

10 Build Cost Effectiveness Model Information: The current standard is to treat a specific type of tumor with radiation We want to study a new treatment that combines surgery and radiation Model: Root node is decision with a branches for each treatment option 19 Build Cost Effectiveness Model Instructions: 1. Create new model from toolbar icon (blank tree). 2. Enter node label text for the root decision node. 3. Double click on root node to add two branches. 4. Enter node label text for each strategy. 5. Adjust width of node as desired for text formatting

11 Build Cost Effectiveness Model Information: We estimate that the new treatment will increase the probability of eradicating the tumor from 60% to 80% Model: Both strategies are chance nodes with branches for tumor eradicated or tumor not eradicated Different probabilities for each set of branches 21 Build Cost Effectiveness Model Instructions: 1. Double click on top strategy node to add two branches. 2. Enter node labels. 3. Ignore the bottom strategy for now; we will create it later

12 Build Cost Effectiveness Model Information: A person s life expectancy is 10 years if the tumor is eradicated, but only 3 years if not The costs associated with radiation and surgery are $30K and $50K respectively The follow up costs post treatment are $2K per year Model: Each of these numeric parameter values factors into the payoffs (scenario values) All parameters should be entered as variables 23 Variables Why use variables? Clarity Isolate your parameters and formulas Consistency define once, use many times Efficiency changing a single variable can affect multiple numeric expressions in the model Transparency easier to understand the meaning of each value Sensitivity analysis covered later Clones covered later Always use variables! 24 12

13 Variables Variables are named values like in algebra Variable name: Each variable has a single name Reference the variable name anywhere in the model to calculate and return its value Calculated dynamically during analysis when referenced 32 letters/numbers/underscores (no punctuation) Stick to a naming style/convention Not case sensitive although you can enter case 25 Variables Variable definitions: Displayed beneath node within box Variable definitions evaluated when referenced Most variables will be defined once at the root node Reference anywhere in the model using the same definition Variables can be defined at any node Use different definitions for different parts of the model Useful for clones covered later Variable reference looks for closest definition from that node to the left MyVar2 = 20 at terminal node 26 13

14 Build Cost Effectiveness Model Instructions: 1. Right click on root node and choose Define Variable > New Variable from the context menu. 2. Enter variable name cfollowupannual. 3. Enter a description and comment if desired. 4. Click OK. 5. Enter definition 2K in the Define Variable dialog and click OK. 6. You will see the variable definition beneath the root node. 7. Repeat steps 2 5 for the variables below. cradiation = 30K csurgery = 50K 27 Variables Variable Properties View: Maintain variables in the tree Options to edit, add, delete, categorize, report variables Edit in Excel available with Excel Module (in TP Suite) Variable Definitions View: Maintain variable definitions Contents change with selected node Defined vs. Undefined vs. Inherited Cut/Copy/Paste to copy or move variable definitions to other nodes 28 14

15 Build Cost Effectiveness Model Instructions: 1. Select root node. 2. Select or open the Variable Properties View. 3. Click the + icon. 4. Enter the variable name efferadicated. 5. Check the box Define numerically at root. 6. Enter 10 in the definition field and click OK. 7. Repeat steps 3 6 for the variable below. effnoteradicated = 3 peradicaterad = 0.6 peradicateradsurg = Enter peradicaterad as the probability value for the Eradicates tumor within the Radiation strategy. 29 Build Cost Effectiveness Model At this point, our model should look like this We still need to Setup tree preferences Terminate the scenarios for the top strategy Build the bottom strategy 30 15

16 Build Cost Effectiveness Model Information: A person s life expectancy is 10 years if the tumor is eradicated, but only 3 years if not The costs associated with radiation and surgery are $30K and $50K respectively The follow up costs post treatment are $2K per year Model: Change tree s calculation method to cost effectiveness Terminate the two scenarios in the top strategies Add the appropriate values for cost and effectiveness 31 Tree Preferences Control calculation and display of model Many categories (see filter) Menu: Tree > Tree Preferences to open dialog Calculation Method: Determines which payoff sets will be used to evaluate the model and determine the optimal strategy Usually either simple or cost effectiveness Numeric Formatting: Controls display of outputs (calculated payoffs and EVs) Set for units, decimal places, labels, etc. Many more 32 16

17 Tree Preferences Simple Calculation Method: Enter a single payoff at each terminal node Single expected value (EV) calculated at all upstream nodes Optimal decision is maximum (eff) or minimum (cost) EV Cost effectiveness Calculation Method: Enter separate payoff values for cost and effectiveness at each terminal node Cost and effectiveness EVs calculated separately at all upstream nodes To select optimal strategy, must balance cost and effectiveness, looking at efficiency & tradeoffs Will examine further when analyzing model 33 Tree Preferences Not limited to two payoffs Only two are active for CE calc method Can use extra payoffs for different cost, effectiveness, other outcomes Same model can then be used to analyze problem with different outcome measurements Change active payoffs for different measurements and repeat analyses Cost: Total cost vs. patient cost vs. insurer cost, etc. Health/Utility: LY, QALY, etc. Other: Positive tests, cases avoided, cases identified, etc

18 Build Cost Effectiveness Model Instructions: 1. Choose Tree > Tree Preferences from the menu. 2. Select the category Calculation > Calculation Method. 3. Select Cost effectiveness (use default active payoffs). 4. Select the category Calculation > Numeric Formatting. 5. Enter settings as seen below. 6. Click OK to save the changes. 35 Build Cost Effectiveness Model Terminal nodes terminate the scenario Must account for all values (cost, eff) that contribute to that scenario Any node on path from the root node to that terminal node For cost effectiveness model, enter separate values for cost and effectiveness For top terminal node Costs: Treatment Cost: cradiation Followup Cost: cfollowupannual * efferadicated Effectiveness: efferadicated For second terminal node Same except using effnoteradicated 36 18

19 Build Cost Effectiveness Model Instructions: 1. Right click on top node. 2. Choose Change Type > Terminal. Edit Payoff dialog opens automatically. 3. Enter expression for Cost: Click ellipsis button to open formula editor for Cost payoff. Select variables to enter the following expression cradiation + (cfollowupannual*efferadicated) 4. Enter expression for Effectiveness Type the following expression efferadicated 5. Click OK. 6. Repeat steps 2 5 for second terminal node (effnoteradicated). 37 Formula Editor The formula editor is available anywhere you see the ellipsis button Create expressions by providing lists of Model inputs (variables, trackers, tables, distributions) Built in functions (Discount, Round, ProbToProb, etc.) Operators (+,, &&, etc.) Keywords (_stage, _trial, _sample, etc.) Value created in Expression field is passed back to the model 38 19

20 Content Assist Content Assist Enter partial text in model Press CONTROL+SPACEBAR to show all numerical elements (variables, functions, etc.) that match the partial text Select an item from the list Placed in expression within model 39 Build Cost Effectiveness Model The two strategies are very similar Why build it twice? Copy the top strategy s subtree and paste it onto the bottom strategy Make necessary changes to the values used in the new treatment strategy 40 20

21 Build Cost Effectiveness Model Instructions: 1. Select top strategy node. 2. Choose Subtree > Select Subtree from the menu. 3. Choose Edit > Copy from the menu. 4. Select bottom strategy node. 5. Choose Edit > Paste from the menu. An exact duplicate of the top strategy s subtree will be copied to the bottom strategy. 41 Build Cost Effectiveness Model Instructions (bottom strategy): 1. Change the probability for Tumor eradicated node to peradicateradsurg. 2. Add csurgery to the cost payoff values at each terminal node. The model is now complete! Training model: Example02 Variables.trex 42 21

22 Clones Before analyzing the model, we will introduce clones In our example, we copied the subtree from one strategy to the other What if the subtree was extremely complex and required significant revision? Clones create exact duplicates of a subtree that stay consistent even as the subtree is updated We will rebuild the second strategy of our tree using clones 43 Clones Instructions: 1. Open the Example 2 model. 2. Select the New treatment node. 3. Choose Subtree > Select Subtree from the menu. 4. Choose Subtree > Delete from the menu. 5. Select the Standard treatment node. 6. Choose Subtree > Create Clone Master from the menu. 7. Enter the name Treatment outcomes. 8. Select the New treatment node. 9. Choose Subtree > Attach Clone. The clone copy will be attached

23 Clones You have now created a clone master and attached a clone copy The clone master is identified by a dark node line and a clone index number beneath the node marker The clone copy is identified with a reference to the clone master index 45 Clones The Clones View allows you to view and edit the clone masters and copies When a clone master is destroyed, all clone copies are also destroyed When a clone copy is destroyed, you can replace it with an independent copy of the clone master 46 23

24 Clones The values in the clone copy and master are now the same How can we really compare strategies? Variables allow you flexibility to send different values into clone master vs. clone copy Same variable references within clone master Pass different variable definitions from outside (to left) of clone into the clone master/copy Recommendation: Define strategy specific parameters at root Define generic variables at strategy nodes using parameter variables from above Do not defined generic variables at root node 47 Clones Two values need to be different for the clone master and clone copy Probability of eradicating the tumor Cost of treatment Note that the clone master now uses generic variables peradicategeneric and ctreatmentgeneric Each strategy s parameters are passed to the clone master/copy via the generic variables defined at the strategy nodes 48 24

25 Clones Now the treatment outcomes subtrees are linked Roll back generates the same values for the strategies as before the clones (next module) Clone copy subtrees cannot currently be expanded If needed, temporarily detach clone copy to see complete independent subtree (don t save change) 49 Analyze Cost Effectiveness Model Module 2: Analyze Cost Effectiveness Model Goals: Understand how each strategy s expected value is calculated Compare strategies on basis of cost effectiveness (which is best?) Consider dominance among strategies (which are rejected) Introduce net benefits calculations for CE analysis TreeAge Pro Healthcare Training Module 2 Analyze Cost-Effectiveness Model 50 25

26 Analyze Cost Effectiveness Model Want to choose the optimal treatment strategy Must first calculate expected values (EVs) for each strategy Compare the strategies cost and effectiveness EVs using standard cost effectiveness analysis techniques TreeAge Pro Healthcare Training Module 2 Analyze Cost-Effectiveness Model 51 Expected Value (EV) Best estimate for the overall value of the strategy Reflects all possible outcomes based on each one s likelihood Sum of each outcome s value weighted by its probability Example: 20% chance of dying immediately 30% chance of living 10 years 50% chance of living 20 years EV = (0.2 * 0) + (0.3 * 10) + (0.5 * 20) = 13 For CE model, cost and effectiveness EVs are calculated separately TreeAge Pro Healthcare Training Module 2 Analyze Cost-Effectiveness Model 52 26

27 Analyze Cost Effectiveness Model TreeAge Pro calculates EVs at terminal nodes then calculates remaining EVs from right to left Terminal nodes EV: Calculate the payoff expressions Top terminal node Cost: cradiation + (cfollowupannual*efferadicated) = 30K + (2K * 10) = 50K Effectiveness: efferadicated = 10 TreeAge Pro Healthcare Training Module 2 Analyze Cost-Effectiveness Model 53 Analyze Cost Effectiveness Model Path probability: Probability of reaching that node within the scenario Product of probabilities of every chance node branch in path from strategy to terminal node P = TreeAge Pro Healthcare Training Module 2 Analyze Cost-Effectiveness Model 54 27

28 Analyze Cost Effectiveness Model Chance nodes: Weighted averages of the EVs of the node s branches Top strategy/chance node Cost: (0.6*50K) + (0.4*36K) = 30K K = 44.4K Eff: (0.6*10) + (0.4*3) = = 7.2 TreeAge Pro Healthcare Training Module 2 Analyze Cost-Effectiveness Model 55 Analyze Cost Effectiveness Model Decision nodes: Compare EVs for all strategies and choose optimal path Simple calculation method: Minimum or maximum based on tree preferences Cost effectiveness calculation method: Need to balance cost and effectiveness Rollback uses net benefits with willingness to pay (WTP) parameter from tree preferences Cost effectiveness analysis TreeAge Pro Healthcare Training Module 2 Analyze Cost-Effectiveness Model 56 28

29 Analyze Cost Effectiveness Model Cost Effectiveness Analysis (CEA) Standard health economic theory Two goals: Optimize effectiveness (maximize usually) Optimize cost (minimize) CEA context: Efficient use of limited resources Recommend interventions if additional effectiveness comes at a reasonable cost TreeAge Pro Healthcare Training Module 2 Analyze Cost-Effectiveness Model 57 Analyze Cost Effectiveness Model CEA: Calculate Incremental Cost Effectiveness Ratio (ICER) How much are we paying for each additional unit of effectiveness? Compare to a willingness to pay (WTP) threshold Is the ICER too high? Budget constraints: Sometimes compare to ceiling cost value Overall cost of treatment may be too high TreeAge Pro Healthcare Training Module 2 Analyze Cost-Effectiveness Model 58 29

30 Analyze Cost Effectiveness Model ICER Calculation: ICER In our model = IC/IE = (C comparator C baseline )/(E comparator E baseline ) ICER = ($97.2K $44.4K)/(8.6LY 7.2LY) = ($52.8K)/(1.4LY) = ~ $37.7K/LY To switch from the standard treatment to the new treatment, it costs ~ $37.7 for each extra LY If we are willing to pay at least that much per extra LY, we can recommend the new treatment TreeAge Pro Healthcare Training Module 2 Analyze Cost-Effectiveness Model 59 Analyze Cost Effectiveness Model Instructions: 1. Select root node. 2. Choose Analysis > Cost Effectiveness Analysis from the menu. 3. Click Yes. A Cost Effectiveness Analysis graph will be created. 4. Click the Text Report link. The cost effectiveness calculations are displayed. TreeAge Pro Healthcare Training Module 2 Analyze Cost-Effectiveness Model 60 30

31 Analyze Cost Effectiveness Model Cost Effectiveness Analysis graph: Plots strategies on cost and effectiveness axes Line segments form the cost effectiveness frontier Slope = ICER If ICER <= WTP, move to next strategy Want to be here Any strategies here would be dominated (rejected) TreeAge Pro Healthcare Training Module 2 Analyze Cost-Effectiveness Model 61 Graphs Edit Chart link modify the appearance of any graph Text Report link see the numeric data behind the graph File > Save allows you to save analysis output File type RPTX graph and underlying data File types JPEG, PNG, etc. image of the graph For publishing TreeAge Pro Healthcare Training Module 2 Analyze Cost-Effectiveness Model 62 31

32 Analyze Cost Effectiveness Model Cost Effectiveness Rankings report: Shows ICER calculations IC IE ICER TreeAge Pro Healthcare Training Module 2 Analyze Cost-Effectiveness Model 63 Analyze Cost Effectiveness Model Dominance: A strategy is dominated when other strategies provide better cost effectiveness Dominated strategies are then rejected as treatment options Absolute Dominance: Less effective (IE < 0) More costly (IC > 0) Extended Dominance: More effective (IE > 0) More costly (IC > 0) Less efficient (ICER > WTP) TreeAge Pro Healthcare Training Module 2 Analyze Cost-Effectiveness Model 64 32

33 Analyze Cost Effectiveness Model Example 4 model demonstrates both types of dominance Same as the tree we built but with three new strategies to evaluate Top two strategies collapsed via Subtree > Collapse Subtree TreeAge Pro Healthcare Training Module 2 Analyze Cost-Effectiveness Model 65 Analyze Cost Effectiveness Model CEA with dominance Dominated strategies are above and to the left of the cost effectiveness frontier TreeAge Pro Healthcare Training Module 2 Analyze Cost-Effectiveness Model 66 33

34 Analyze Cost Effectiveness Model Other Tx 2 rejected by absolute dominance Less effective and more costly than Standard Treatment Other Tx 3 rejected by extended dominance Lower ICER, more efficient to move from Standard treatment to New Treatment Higher ICER, less efficient to move from Standard treatment to Other Tx 3 Blended combination of New Treatment and Standard Treatment would get more effectiveness at same cost TreeAge Pro Healthcare Training Module 2 Analyze Cost-Effectiveness Model 67 Analyze Cost Effectiveness Model CE Rankings also shows dominance Absolute dominance negative IE and ICER Extended dominance ICER declines TreeAge Pro Healthcare Training Module 2 Analyze Cost-Effectiveness Model 68 34

35 Analyze Cost Effectiveness Model Which strategy is preferred? First reject dominated strategies Then compare the ICERs to the WTP value If WTP < $37.7K, recommend Standard treatment If $37.7K < WTP < $132K, recommend New treatment If WTP > $132K, recommend Other Tx 1 TreeAge Pro Healthcare Training Module 2 Analyze Cost-Effectiveness Model 69 Net Benefits Net Benefits combines cost, effectiveness and WTP into a single measurement Builds the ICER threshold (WTP) into calculations, as the weight on effectiveness Strategy with largest NB is most cost effective Calculations: NMB = (E * WTP) C = (LY * $/LY) $ = $ NHB = E (C / WTP) = LY ($ / ($/LY)) = LY Allows roll back to identify most cost effective strategy Simplifies presentation of more complex analyses looking for most cost effective strategy i.e., Sensitivity analysis TreeAge Pro Healthcare Training Module 2 Analyze Cost-Effectiveness Model 70 35

36 Analyze Cost Effectiveness Model Note that we can identify the preferred strategy from roll back using CE parameters in Tree Preferences Optimal strategy identified by Net benefits calculations using WTP parameter Change to Invert incremental only if effectiveness should be minimized (e.g., # of infections or # of deaths) TreeAge Pro Healthcare Training Module 2 Analyze Cost-Effectiveness Model 71 Sensitivity Analysis Module 3: Sensitivity Analysis Goals: Examine the effect of uncertainty on analysis results Study an individual parameter s uncertainty and identify thresholds Modeling exercise Study combined parameter uncertainty to determine overall confidence in conclusions 72 36

37 Sensitivity Analysis Sensitivity Analysis studies how uncertainty in a model s parameter inputs affect its analysis outputs And by extension, your conclusions/strategy selection Two types of Sensitivity Analysis are supported by TreeAge Pro Deterministic Sensitivity Analysis Individual parameters via VARIABLES (range) Probabilistic Sensitivity Analysis Many parameters via DISTRIBUTIONS (samples) Both are recalculation loops, with uncertain parameters changing between recalculations 73 Sensitivity Analysis Deterministic One Way Sensitivity Analysis: Identify a single parameter for analysis Must be a numeric parameter and not a formula Set a range (min, max) for the uncertainty related to its value Set the number of intervals for recalculations within the range Recalculate the model several times changing the parameter value from the bottom to the top of the range How do the results (and possibly conclusions) change? 74 37

38 Sensitivity Analysis Deterministic Instructions: 1. Open Example 2 tree. 2. Select root node. 3. Choose Analysis > Sensitivity Analysis > 1 Way from the menu. 4. Enter the sensitivity analysis parameters Select the variable peradicateradsurg. Range: Intervals: 8. Click OK. 75 Sensitivity Analysis Deterministic Generates EV calculations for each value of the variable given the range and intervals Cost, effectiveness for each strategy ICER for each parameter value Threshold: var ~= 0.75, ICER ~= 50K 76 38

39 Sensitivity Analysis Deterministic x vs. Avg. Eff. graph: Sensitivity analysis on effectiveness only Threshold at peradicateradsurg value 0.6 Point where effectiveness is equal for two strategies We really want threshold for cost effectiveness 77 Sensitivity Analysis Deterministic Sensitivity analysis on cost effectiveness is more complex At what point in variable range do we have a change in the recommended strategy based on cost effectiveness (threshold) Need to consider cost, effectiveness and WTP x. vs. ICER graph shows approximate threshold Net Benefits graph and thresholds report best identify threshold 78 39

40 Sensitivity Analysis Deterministic X vs. ICER graph: Shows how ICER changes with variable ICER undefined when IE = 0, presented as zero Can approximate threshold where ICER = WTP 79 Sensitivity Analysis Deterministic Net Benefits: Prompts for WTP, required for Net Benefits calcs Net benefits is higher for most cost effective strategy (C, E, WTP combo) Identifies CE threshold If var >= 0.749, recommend surgery + radiation If var < 0.749, recommend radiation only Thresholds report 80 40

41 Sensitivity Analysis Deterministic 2 way sensitivity analysis: See how changing two parameters affects recommended strategy Requires Net Benefits Each axis taken by a variable Region patterns show recommended strategy 81 Sensitivity Analysis Deterministic Instructions: 1. Open Example 2 tree. 2. Select root node. 3. Choose Analysis > Sensitivity Analysis > 2 Way from the menu. 4. Enter the sensitivity analysis parameters 1. Var1: peradicateradsurg; Range: 0 1; Intervals: Var2: peradicaterad; Range: 0 1; Intervals: WTP: 50K 5. Click OK

42 Sensitivity Analysis Deterministic 2 way sensitivity analysis: New treatment favored more with increased peradicateradsurg Standard treatment favored more with increased peradicaterad 83 Sensitivity Analysis Deterministic Tornado Diagram: Run a collection of 1 way sensitivity analyses See which variables have the largest impact on EV Uses Net Benefits with additional option for ICER Be careful: Do not overanalyze these graphs Details from 1 way sensitivity analyses are not fully presented in tornado diagram 84 42

43 Sensitivity Analysis Deterministic Instructions: 1. Open Example 2 tree. 2. Select root node. 3. Choose Analysis > Sensitivity Analysis > Tornado Diagram from the menu. 4. Enter the sensitivity analysis parameters 1. Var1: peradicaterad; Range: ; Intervals: Var2: peradicateradsurg; Range: ; Intervals: Var3: cradiation; Range: 25K 35K, Intervals: 4 4. Var4: csurgery; Range: 40K 60K, Intervals: 4 5. Var5: cfollowupannual; Range: 1.8K 2.2K, Intervals: 4 6. WTP: 50K 5. Click OK. 85 Sensitivity Analysis Deterministic Tornado Diagram: Shows band for range of EV for preferred strategy Dotted line shows base case EV Links for 1 way net benefit graphs Link for ICER graph Dark line indicates a strategy change 86 43

44 Modeling Exercise We are studying treatments for a disease that affects only elderly individuals Without the disease, the average person will live for 10 years With the disease, the average person will live only 5 years We are studying two competing drugs for treating this disease Each drug must be taken for at least 1 year If either drug cures the disease, it must be taken on an ongoing basis If either drug fails to cure the disease, no further treatment can be provided 87 Modeling Exercise Drug 1: Annual cost is $9K Cures disease in 70% of patients Drug 2: Annual cost is $12K Cures disease in 80% of patients Which, if either, drug is the most cost effective treatment option given a WTP of $50K/LY? 88 44

45 Modeling Exercise How will the model structure begin to form the basis for the decision analysis? Start with decision node with branches for each of three strategies What are the important parameters? Life expectancy with disease = 5 Life expectancy if cured = 10 Prob of cure with Drug 1 = 0.7 Prob of cure with Drug 2 = 0.8 Annual cost of Drug 1 = 9K Annual cost of Drug 2 = 12K 89 Modeling Exercise Given WTP = $50K/LY Which is the most cost effective treatment option? CEA/Rankings Drug 1 ICER = $18.8K Drug 2 ICER = $65.4K At what price for Drug 1would there be a strategy change? Sensitivity analysis on cost of Drug 1 for range 7K 11K Drug 1 at price $10,055, Drug 2 becomes favored Tx Assume we do not know the LY after cure. Identify thresholds in the range Sensitivity analysis on life expectancy after cure Shift from No Tx to Drug 1 at 6.19 Shift from Drug 1 to Drug 2 at

46 Sensitivity Analysis Probabilistic Probabilistic Sensitivity Analysis (PSA): Consider the combined uncertainty related to any number of parameters How does this uncertainty affect the overall confidence in your base case conclusions? Percent of simulation iterations that confirm Confidence intervals around main outputs Like ICER No thresholds for individual parameters 91 Sensitivity Analysis Probabilistic Monte Carlo simulation: Introduces randomness, sampling into analysis Required for Probabilistic Sensitivity Analysis (PSA) TreeAge Pro supports several forms of simulation Probabilistic Sensitivity Analysis Samples: 2nd order, parameter uncertainty Microsimulation, random walks Trials: 1st order, individual variability Two Dimensional Samples & Trials in same analysis Three Dimensional Partial EVPI 92 46

47 Sensitivity Analysis Probabilistic Deterministic vs. Probabilistic Deterministic Sensitivity Analysis Parameter Variables: range, intervals Probabilistic Sensitivity Analysis Parameter Distributions random samples Usually focused on 1 uncertainty at a time All uncertainties sampled simultaneously Repeat analysis Identical results Repeat analysis Different individual results Similar aggregate values 93 Sensitivity Analysis Probabilistic Prepare model for PSA: Set parameter values using distributions instead of simple variables Only create distributions for parameters you want to study via PSA PSA calculation loop: Sample parameter value from each distribution Substitute sampled values into the model Calculate expected values for strategies/payoff sets (C and E) Repeat for predefined number of samples Results: Set of EV calculations reflecting different parameter sets Mean EVs generally will confirm base case Individual EVs may not confirm base case Reflection of confidence in base case Confidence intervals 94 47

48 Sensitivity Analysis Probabilistic We will start with the Example 02 model Open and save under new name We will introduce distributions to measure parameter uncertainty for Probability of eradicating tumor with radiation Probability of eradicating tumor with radiation + surgery Cost of surgery 95 Sensitivity Analysis Probabilistic Instructions: 1. Choose Views > Distributions from the toolbar. 2. Click the + icon in the Distributions View to create a new distribution. 3. Enter the distribution information at right

49 Distributions Distribution type: Shape of distribution Required parameters are specific to each distribution type Example: Normal mean and standard deviation Parameters: Numeric values for sampling (sort of like a range) Warning: It can be difficult to determine the appropriate type and parameters for each of your model s uncertainties 97 Distributions Distribution type: Shape of distribution Required parameters are specific to each distribution type Example: Normal mean and standard deviation Parameters: Numeric values for sampling (sort of like a range) Warning: It can be difficult to determine the appropriate type and parameters for each of your model s uncertainties 98 49

50 Distribution Types Normal Standard bell curve Uniform Equal likelihood of all values in range Option to limit to integers Triangular Provide min, max and most likely Often easiest to create, understand Beta Frequently used for probabilities Restricted to between 0 and 1 Dirichlet Sample multiple complementary probabilities, like interrelated betas Table You determine each values that can be sampled and how frequently Good for known empirical data 99 Distributions Sampling rate: Resample per EV/group of trials PSA parameter uncertainty Parameter sample affects entire cohort Resample per trial For Microsimulation New sample for each individual trial within the cohort Resample per Markov cycle For Microsimulation New sample for each cycle in Markov model (less common)

51 Distributions Must reference distribution within model for Monte Carlo simulation (PSA) Reference distribution by either index or name Recommend references by name Distribution functions require reference by index Example: DistForce(index), DistTrim(index; min; max) Parameter approximation: Certain distribution types parameters can be approximated from other statistics In our first distribution, we estimated the Alpha and Beta parameters from the mean and standard deviation 101 Distributions Use Graph icon to sample and graph distribution: Mimics sampling that will be performed during PSA Generates a histogram of samples Samples centered around mean (0.6) with variation Beta is not a normal bell curve

52 Sensitivity Analysis Probabilistic Instructions: 1. Open/select the Distributions View. 2. Click the + icon in the Distributions View to create a new distribution. 3. Create two more distributions. dist_peradicateradsurg Beta parameters approximated from mean 0.8 and std dev 0.1 dist_csurgery Normal distribution with mean 50,000 and std dev 10, Sensitivity Analysis Probabilistic Distributions created, but they need to be integrated into the model Two options Replace references to variables with references to distributions within model Define existing referenced variables using distributions rather than numeric values Can still run deterministic sensitivity analysis

53 Sensitivity Analysis Probabilistic Instructions: 1. Select the root node. 2. Open/select the Variable Definitions View. 3. Redefine the following three variables 1. csurgery = dist_csurgery 2. peradicaterad = dist_peradicaterad 3. peradicateradsurg = dist_peradicateradsurg 105 Sensitivity Analysis Probabilistic Non PSA calculations will use mean values for distributions (no sampling) Our distributions means are equal to the original numeric estimates Our EV calculations will not change Be careful with non PSA analyses after adding distributions

54 Sensitivity Analysis Probabilistic Instructions: 1. Select the root node. 2. Choose Analysis > Monte Carlo Simulation > Sampling. 3. Enter 1000 samples. 4. Click Begin. 107 Sensitivity Analysis Probabilistic How many iterations do you need for a Monte Carlo simulation? Is 1000 enough? Depends on number of distributions and complexity of model If successive simulations generate mean values that are very close enough iterations Good rule for all simulation types (samples, trials, etc.)

55 Sensitivity Analysis Probabilistic Simulation results: Aggregate results: Mean Standard deviation % intervals Each iteration s results May or may not confirm base case Outputs Text reports Graphs Provide interpretation Can save result set in *.rptx file (consistency in paper) 109 Sensitivity Analysis Probabilistic Selected PSA output options Values, Distributions: Shows each strategy s EV calculations and distribution samples for each iteration Output Distributions: Shows distribution of EV outputs for each strategy or for incrementals ICER distribution can be interesting Sometimes invalid in cases where IE can be zero (wild fluctuation in ICER) CE Analysis: CEA from simulation means ICER may not match regular non PSA CEA ICER CE Scatterplot: Shows scatter of each iteration s cost and effectiveness values for each strategy

56 Sensitivity Analysis Probabilistic Remember out goal to see how combined uncertainty affects the overall confidence in our base case conclusions For this ICE Scatterplot Strategy Selection Frequency Acceptability Curve 111 Sensitivity Analysis Probabilistic ICE Scatterplot Compares a pair of strategies showing IC and IE on graph Edit graph to change y axis scale to include zero Line from origin to each point is like the ICER slope in CEA graph Points below right of WTP line (64.9%) recommend New Treatment ICER <= WTP

57 Sensitivity Analysis Probabilistic Strategy Selection Frequency Shows the percentage of iterations that favor each strategy at single WTP ($50K) Shows same percentage as ICE scatterplot Easier to see using Net Ben Changed scale via Edit Chart options 113 Sensitivity Analysis Probabilistic CE Acceptability Curve Shows the percentage of iterations that favor each strategy over WTP range ($0 $100K) Added line via Edit Chart options

58 Markov Models Module 4: Markov Models Goals: Understand the concepts behind Markov models Build a simple Markov model TreeAge Pro Healthcare Training Module 4 Markov Models 115 Markov Models Markov models: Follow a cohort of patients into the future Track disease progression over time Breaks down overall progression into individual cycles that repeat Model represents events within a cycle Also called state transition model Without Markov model Would have to create model structure for all events over lifetime Would lead to a gigantic model TreeAge Pro Healthcare Training Module 4 Markov Models

59 Markov Models Markov Model structure Starts with Markov node Branches are health states which start each cycle Subtree of each health state models all possible outcomes for an individual starting a cycle in that state Terminal nodes return individual to a health state (same or different) to start next cycle Accumulate cost and effectiveness within each cycle Continue until finished with analysis Report accumulated cost and effectiveness TreeAge Pro Healthcare Training Module 4 Markov Models 117 Markov Models Markov Model: Consists of the Markov node and everything to the right Usually part of a larger decision tree for strategy selection Evaluates to a single cost and effectiveness measure Feeds back into decision analysis like a terminal node Markov node cannot be placed within another Markov subtree TreeAge Pro Healthcare Training Module 4 Markov Models

60 Markov Models We will build a simple Markov model Markov node: 1 year cycles (implied) Terminate model after 20 years Markov health states: Alive & Dead Entire cohort starts in Alive state (initial probabilities) For each Alive cycle, accumulate State rewards Effectiveness of 1 LY Cost of $50K Transition subtree: At each annual cycle, there is a 10% chance of death TreeAge Pro Healthcare Training Module 4 Markov Models 119 Markov Models Cohort: Markov models follow a hypothetical cohort which is split among health states and transitions Cohort is homogeneous Use a hypothetical cohort of 1 to measure expected value for a single person Cohort starts each cycle split among the health states For each state, cohort % is then split further to reflect the many events that can occur in a cycle At end of cycle, cohort is returned to the health states (in different fractions) to start the next cycle TreeAge Pro Healthcare Training Module 4 Markov Models

61 Markov Models Cycle Divides the overall analysis into smaller time periods for modeling Almost always a fixed cycle length Cycle length is known, but not specified as a parameter All probabilities and rewards (cost, eff) must have values consistent with the cycle length Model is usually run for a specific number of cycles (e.g., 20 1 year cycles) Built in keyword _stage is used as reference to the cycle count _stage = 0 during first cycle TreeAge Pro Healthcare Training Module 4 Markov Models 121 Markov Models Markov node: Starts the Markov model Termination condition indicates when to stop processing Evaluated before each cycle analysis stops when true Frequently a function of _stage To run for 20 cycles: _stage = 20 (_stage = 0, 1,, 18, 19) Can run until entire cohort is dead (StateProb function) Be careful if prob of death never reaches 100% (will run forever) Multiple conditions: & = AND, = OR TreeAge Pro Healthcare Training Module 4 Markov Models

62 Markov Models Information: Markov node: 1 year cycles (implied) Terminate model after 20 years Instructions: 1. Create new model from toolbar icon (blank tree). 2. Change root node to type Markov. 3. Enter node label text. 4. Open the Markov Info View. 5. Edit the default termination condition. TreeAge Pro Healthcare Training Module 4 Markov Models 123 Markov Models Health States: Direct branches from Markov node Starting point for each cycle Track the changing distribution of the cohort among a number of mutually exclusive states Initial probabilities divide the cohort among the health states before the first cycle Proportion of cohort in health states will be different for next cycle based on events that occur within the cycle TreeAge Pro Healthcare Training Module 4 Markov Models

63 Markov Models Health States: State rewards accumulate value (cost, eff) by cycle Cost of treating person with that state s condition i.e., $10K/year to treat person with diabetes Utility (for QALYs) associated with the health state i.e., 0.80 rather than 1 for person in bad health Rewards Initial: for first cycle (often same as incremental) Incremental: for every subsequent cycle Final: after the last cycle (usually 0) TreeAge Pro Healthcare Training Module 4 Markov Models 125 Markov Models Information: Markov states: Alive & Dead Entire cohort starts in Alive state Instructions: 1. Double click on the Markov node to add two branches (Markov health states). 2. Enter node label text for each Markov state. 3. Enter the initial probability beneath each state. TreeAge Pro Healthcare Training Module 4 Markov Models

64 Markov Models Information: Markov states: State rewards for Alive state Effectiveness of 1 LY Cost of $50K Instructions: 1. Edit the Tree Preferences set Calc Method to Cost Effectiveness and set Numeric Formatting. 2. Select the Alive state node. 3. Open the Markov Info View. 4. Enter the State Rewards values initial and incremental rewards. TreeAge Pro Healthcare Training Module 4 Markov Models 127 Markov Models Transition Subtrees: Model structure for what can happen within a cycle Events (e.g., surgery, screening test, stroke, etc.) Each Markov state has its own transition subtree Structure and transition probabilities drive the cohort through the transition subtree Terminal nodes at end of subtree direct cohort to health states to start the next cycle Changes the cohort split among health states by cycle Absorbing states (usually Dead) do not have a transition subtree or jump states TreeAge Pro Healthcare Training Module 4 Markov Models

65 Markov Models Transition Subtrees: Transition rewards (cost, eff) are associated with events in subtree i.e., operation, adverse event, screening test cost and/or disutility Allocated to cohort that passes through that node in the subtree (not everyone starting cycle in state) Use extra payoff to count transitions to the dead state TreeAge Pro Healthcare Training Module 4 Markov Models 129 Markov Models Information: Transition subtree: At each annual cycle, there is a 10% chance of death Instructions: 1. Double click on the Alive node to add two branches in the transition subtree. 2. Enter the node label text for each branch. 3. Enter the probability for the Live (#) and Die (0.1) branches. TreeAge Pro Healthcare Training Module 4 Markov Models

66 Markov Models Information: Count deaths via transition reward Instructions: 1. Tree Preferences: 1. Select option to calculate extra payoffs. 2. Set enabled payoffs to Set custom payoff labels. 2. Set transition reward for Die node to 1 for payoff set 3. TreeAge Pro Healthcare Training Module 4 Markov Models 131 Markov Models Need to terminate the transition subtrees Instructions: 1. Right click on the Live and change the node type to Terminal. 2. Select the jump state Alive when prompted. 3. Repeat for the Die branch and select the jump state Dead. 4. Repeat for the Dead state no jump state needed for absorbing state. TreeAge Pro Healthcare Training Module 4 Markov Models

67 Markov Models Model is now complete Example07a MarkovSimple.trex Markov state node Transition subtree starts here Markov node Termination condition Jump state (for next cycle) Transition probability Markov state rewards Initial probability TreeAge Pro Healthcare Training Module 4 Markov Models