An efficient method for computing the Expected Value of Sample Information. A non-parametric regression approach

Transcription

1 ScHARR Working Paper An efficient metho for computing the Expecte Value of Sample Information. A non-parametric regression approach Mark Strong,, eremy E. Oakley 2, Alan Brennan. School of Health an Relate Research (ScHARR), University of Sheffiel, 30 Regent Street, Sheffiel S 4DA, UK. 2. School of Mathematics an Statistics, University of Sheffiel, Hicks Builing, Hounsfiel Roa, Sheffiel S3 7RH, UK. Corresponing author: m.strong@sheffiel.ac.uk March 25, 205 Abstract Expecte value of sample information (EVSI) is typically compute via a neste two-level Monte Carlo scheme in which plausible atasets are generate in an outer loop, an then conitional on each ataset, samples are generate from the posterior istribution of the parameters in an inner loop. Because the moel is run at each iteration of the inner loop, this scheme can easily become computationally burensome. The two-level Monte Carlo approach will also be ifficult if the prior istribution of the moel parameters is not conjugate to the ata likelihoo. In this case generating the inner loop samples will typically require Markov chain Monte Carlo (MCMC), an the repeate application of MCMC for each sample ataset as consierably to the computational buren. We present here a metho for calculating EVSI that requires only the single set of moel evaluations that is generate in a stanar probability sensitivity analysis (PSA). Our metho is base on a non-parametric regression of the moelle net benefits on ata samples that are generate from the PSA, an follows closely the non-parametric regression metho for computing partial EVPI that we have previously escribe. Our propose EVSI metho makes no assumptions regaring the form of the moel, oes not require the use of MCMC, an oes not require that the parameter prior is conjugate to the ata likelihoo. All that is require is that the ata likelihoo can be sample (but not necessarily evaluate).

2 Strong, Oakley, Brennan Efficient computation of EVSI 2 Introuction Expecte value of sample information (EVSI) is typically compute via a neste two-level Monte Carlo scheme in which plausible atasets are generate in an outer loop, an then conitional on each ataset, samples are generate from the posterior istribution of the parameters in an inner loop. Because the moel is run at each iteration of the inner loop, this scheme can easily become computationally burensome. The two-level Monte Carlo approach will also be ifficult if the prior istribution of the moel parameters is not conjugate to the ata likelihoo. In this case generating the inner loop samples will typically require Markov chain Monte Carlo (MCMC), an the repeate application of MCMC for each sample ataset as consierably to the computational buren. Computationally simpler approaches are sometimes available, but these rely either on the moel being of a certain form (Aes et al., 2004; Welton et al., 203), or on assumptions of Normality of the mean incremental net benefits (Eckermann an Willan, 2007). We present here a metho for calculating EVSI that requires only the single set of moel evaluations that is generate in a stanar probability sensitivity analysis (PSA). Our metho is base on a non-parametric regression of the moelle net benefits on ata samples that are generate from the PSA, an follows closely the non-parametric regression metho for computing partial EVPI escribe in Strong et al. (203). Our propose metho for EVSI makes no assumptions regaring the form of the moel, oes not require the use of MCMC, an oes not require that the parameter prior is conjugate to the ata likelihoo. All that is require is that the ata likelihoo can be sample (but not necessarily evaluate). 2 Metho The expecte value of sample information (EVSI) is the expecte ifference between the value of the optimal ecision base on some sample of ata, informative for some subset of inputs, an the value of the ecision mae only with prior information (Raiffa, 968; Claxton an Posnett, 996; Aes et al., 2004). To express this we first introuce some notation. We assume that we are face with D ecision options, inexe =,..., D, an have built a moel NB(, θ) that aims to preict the net benefit of ecision option given a vector of p input parameter values θ = (θ,..., θ p ). The true values of the input parameters are assume to be unknown an we represent beliefs about the input parameters via their joint istribution p(θ). We inex a sample rawn from the joint istribution of the parameters with a brackete superscript, θ (n), for sample raws n =,..., N. We envisage that we can collect ata that will be informative for some subset of parameters. We consier the (as yet uncollecte) ata as a vector of ranom variables, an enote this as upper case X. We use lower case x for some arbitrary realise (or sample) vector of values from the istribution of X. We use the brackete superscript notation to inex a sample of ata vectors x (n), n =,..., N. The expecte value of our optimal ecision, mae only with current information is max E θ {NB(, θ)}. ()

3 Strong, Oakley, Brennan Efficient computation of EVSI 3 If we ha ata X that were informative for (some subset of) the inputs, then the optimal ecision woul be that with the greatest net benefit, after averaging over the joint istribution of the inputs conitional on the ata, θ X. The expecte net benefit woul be max E θ X {NB(, θ)}. (2) But, since X is uncollecte, we must average over possible atasets, E X [max E θ X {NB(, θ)}, (3) where the istribution of X can be obtaine by the marginalisation of p(x, θ) = p(x θ)p(θ). This expression suggests a straightforwar Monte Carlo sampling scheme for X, i.e. sample first a value θ from the prior p(θ) an then sample X from the ata likelihoo p(x θ = θ ). The EVSI is then the ifference between equation (3) an equation (), EVSI = E X [max E θ X {NB(, θ)} max E θ {NB(, θ)}. (4) In most applications the X will only be informative for either a single parameter, or a small subset of parameters, θ i. The istribution for the remaining parameters θ i remains unchange on learning X. The partition θ = (θ i, θ i ) is implicit in the above expressions. At this point we also note that we can re-express (4) as follows. Firstly, we write max E θ {NB(, θ)} = E θ {NB(, θ)} (5) where an hence EVSI = E X [max = arg max E θ {NB(, θ)}, (6) E θ X {NB(, θ)} = E X [max E θ X {NB(, θ)} max E θ {NB(, θ)} = E X [max E θ X {NB(, θ)} E θ X {NB(, θ)} E θ {NB(, θ)}. (7) The reason for the re-expression will become apparent when we iscuss Monte Carlo sampling schemes for estimating EVSI. 2. The Monte Carlo approach to computing EVSI A probabilistic sensitivity analysis (PSA) takes N samples from the joint istribution of the input parameters, {θ (),..., θ (N) }, an generates a corresponing set of N net benefits {NB(, θ () ),..., NB(, θ (N) )} for each ecision option. From this the usual Monte Carlo solution to the secon term in equation (4) is max E θ {NB(, θ)} max N N NB(, θ (n) ). (8) n=

4 Strong, Oakley, Brennan Efficient computation of EVSI 4 The first term in equation (4) requires more work, an unless there are analytic solutions to the expectations the usual approach is to use a neste two-level Monte Carlo metho (Aes et al., 2004). Here, the estimator is given by E X [max E θ X {NB(, θ)} K k= max NB(, θ (j,k) ), (9) where θ (j,k) are samples rawn from the posterior istribution of θ x (k), an x (k) are generate by first sampling θ (k) from p(θ) an then x (k) from p(x θ = θ (k) ). Subtracting equation (8) from equation (9) results in the two-level Monte Carlo EVSI estimator ÊVSI = K k= max j= NB(, θ (j,k) ) max j= N N NB(, θ (n) ). (0) However, if we arrange our sampling scheme such that it reflects equation (7) we obtain instea where ÊVSI = K = K max k= k= max NB(, θ (j,k) ) K j= = arg max NB(, θ (j,k) ) j= K k= k= n= NB(, θ (j,k) ) j= NB(, θ (j,k) ). () j= NB(, θ (j,k) ). (2) This allows us to exploit the positive correlation between max j= NB(, θ(j,k) ) an j= NB(, θ (j,k) ), an hence obtain an estimator with lower variance than equation (0). The first problem with the neste two-level scheme is the requirement to evaluate the net benefit function (i.e. to run the moel) at each iteration of the inner loop, resulting in K moel evaluations. If the moel is slow to run, an/or if an K are large (in orer to obtain aequate precision), then the scheme will be computationally burensome. A secon potential problem is the requirement to sample from the posterior istribution of the input parameters, conitional on the sample ata, i.e. obtaining the j =,..., samples θ (j,k) from each p(θ x (k) ) in the inner loop. If p(x θ) an p(θ) are conjugate then the posterior istribution will be of a known form, an sampling from it will be straightforwar. However, if conjugate forms are not appropriate, we may be require to resort to MCMC to generate the j =,..., samples θ (j,k) from p(θ x (k) ). The MCMC step must be repeate for each of the k =,..., K sample ata values, an this will a consierably to the computational buren. Setting up the MCMC sampler (e.g. via writing BUGS coe (Lunn et al., 2009)) an checking the MCMC chain(s) for convergence also requires investment in moeller time. We note at this point that in some restricte cases we can avoi entirely the inner loop Monte Carlo step. If the moel is linear or multi-linear (i.e. of sum-prouct form) in the parameters, an if the parameters are inepenent of one another (an retain this inepenence after upating with ata), an if we can analytically compute the posterior expectations of the parameters given the ata, then we can simply plug in the expecte parameter values into the net benefit equation to obtain the expecte net benefit. See Brennan et al. (2007) or Aes et al. (2004) for a fuller iscussion. j=

5 Strong, Oakley, Brennan Efficient computation of EVSI Non-parametric regression metho The problem with the two-level Monte Carlo scheme is the nee to compute the inner expectation in the first term in (4) via Monte Carlo. Not only oes this require moel runs for each outer loop, but it is this step that requires the potentially problematic sampling from the conitional istribution p(θ X). We therefore propose to estimate this expectation as follows. Firstly, we recognise that we can express the net benefit for ecision option evaluate at θ (k) as a sum of the conitional expectation that we require, an a mean-zero error term, To see why ε has zero mean we rearrange to give an take expectations with respect to both X an θ, NB(, θ) = E θ X {NB(, θ)} + ε. (3) ε = NB(, θ) E θ X {NB(, θ)}, (4) E(ε) = E{NB(, θ)} E X [E θ X {NB(, θ)} = E{NB(, θ)} E{NB(, θ)} = 0. (5) Next, we recognise that the expectation E θ X {NB(, θ)} can be thought of as an unknown function of X. We enote this function g(, X), an substituting this into equation (3) gives NB(, θ) = g(, X) + ε. (6) In many instances the ata X will be high imensional (e.g. censore time-to-event ata in a stuy that measures survival), an if this is so we write E θ X {NB(, θ)} as a function of some low imensional summary statistic of the ata T (X), We iscuss choice of summary statistic in the next section. NB(, θ) = g{, T (X)} + ε. (7) Lastly, for each ecision option, we treat the net benefits NB(, θ) as noisy ata through which we can learn about the target function g{, T (X)}. Thus, we can think of this as a regression problem. However, we immeiately recognise that the target function g{, T (X)} has unknown form, an we have no esire to impose any particular form. We coul begin by fitting a stanar linear moel, with power an interaction terms to moel the non-linearity between the net benefits an the inputs of interest, but we choose instea to aopt the more flexible non-parametric regression approach offere by the Generalise Aitive Moel (GAM). GAM moels assume that the expectation of the epenent variable is a smooth, but usually unknown, function of the inepenent variable, which is exactly what we nee here. For an introuction to GAM moels see Hastie an Tibshirani (986) or Woo (2006). The moels are easy to fit using the freely available software R (R Development Core Team, 203). Obtaining the necessary ata for the regression analysis procees as follows. We assume we have at our isposal a PSA sample of size K. This consists of a set of K samples from the istribution of the input parameters {θ (),..., θ (K) }, an K corresponing evaluations of the moel {NB(, θ () ),..., NB(, θ (K) )} for ecision options =,..., D.

6 Strong, Oakley, Brennan Efficient computation of EVSI 6 In orer to generate realisations of the inepenent variable T (X) that correspon to the epenent variable realisations {NB(, θ () ),..., NB(, θ (K) )} we sample, for each value θ (k), a ataset x (k) from the likelihoo p(x θ = θ (k) ). To give an example, we imagine a hypothetical net benefit function NB(, θ) = θ 2. The parameter θ represents a proportion (e.g. of people in the population who have a certain characteristic) an current knowlege about the proportion is expresse via a Beta(40,200) istribution. We want to know the value of oing a stuy with 500 participants to learn about the proportion. The number of people in the stuy with the characteristic of interest, x, is moelle using a Binomial(θ, 500) istribution. The PSA sample is comprise of samples {θ (),..., θ (K) } with the corresponing samples from the net benefit function {NB(, θ () ),..., NB(, θ (K) )}. For each sample θ (k) we generate a sample of ata x (k) from X θ (k) Binomial(θ (k), 500). The ata here are scalar an we therefore choose T (x) = x. We fit a GAM regression of NB(, θ) on T (x) an extract the fitte values, which are our estimates of E θ x {NB(, θ)}. In this hypothetical example we can also calculate E θ x {NB(, θ)} analytically, so can assess the accuracy of the GAM regression metho. Figure shows a scatter plot of sample values of the incremental net benefit, NB(, θ) = θ 2, versus sample values of x. The two lines show the posterior expecte net benefit E θ x {NB(, θ)} as a function of the sample values of x. The soli line shows the GAM moel fitte values an the ashe line shows the analytically calculate values T(x) NB(θ) Posterior expectation E{NB(θ) x} GAM moel fitte values Analytically calculate values Figure : Hypothetical example. GAM moel fitte values of the posterior expecte incremental net benefit versus analytic values. After fitting the GAM moel we then extract the regression moel fitte values. The fitte values are estimates of g{, T (x () )},..., g{, T (x (K) )}, our target quantity. We enote the

7 Strong, Oakley, Brennan Efficient computation of EVSI 7 GAM fitte values ĝ{, T (x () )},..., ĝ{, T (x (K) )} an the estimate EVSI is then given by withêvsi = K k= max ĝ{, T (x (k) )} max K ĝ{, T (x (k) )}. (8) k= Note that we use max K K k= ĝ{, T (x(k) )} as our unbiase Monte Carlo estimator of the secon term in equation (4) rather than max K K k= NB(, θ(k) ). By choosing this as our estimator we exploit the positive correlation between the two terms in equation (8) an hence estimate the EVSI with increase precision. We also note at this point that EVSI (calculate by any metho) is invariant to the re-expression of net benefits as incremental net benefits, relative to some chosen baseline option (which is therefore efine as having an absolute net benefit of zero). This reuces the number of regression problems from D to D. Because we are averaging over k we can think of this as a single loop Monte Carlo metho. The size of K will etermine the precision of the estimate of the EVSI an a metho for estimating the stanar error of this approximation is given in Appenix A. Example R coe is available at [url here - for peer review see file. The sampling scheme in Box Box - GAM regression base EVSI algorithm Firstly, generate a PSA sample of size K: Sample θ (k), kwith =,..., K from the istribution of the parameters, p(θ) Run the economic moel to obtain net benefits NB(, θ (k) ) Calculate incremental net benefits INB(, θ (k) ) Then o: For each k =,..., K generate a ata sample x (k) from p(x θ (k) ) Calculate summary statistic T (x (k) ) for each ata sample x (k) Regress INB(, x) on T (x) for each > Set ĝ{, T (x (k) )} = 0 for all k Extract GAM moel fitte values ĝ{, T (x (k) )} for each > Calculate EVSI via equation (8). The coe in Box 2 illustrates the simplicity of the GAM regression approach in R (R Development Core Team, 203). We have chosen the implementation of GAM that is provie by the mgcv package (Woo, 2006). In the example there are two ecision options, with the vector object INB holing the incremental net benefits from the PSA. One of the moel parameters, θ, is a proportion (say, for example, the proportion of patients in a population who get better from some isease without treatment). The aim of the EVSI analysis is to etermine the expecte value of oing a stuy with 200 participants in orer to learn about this proportion. The number of patients, x, who get better from the isease without treatment in the stuy is assume to follow a Binomial(θ, 200) istribution. The ata here are scalar, so we choose the summary statistic to be T (x) = x.

8 Strong, Oakley, Brennan Efficient computation of EVSI 8 Box 2 - example R coe for estimating EVSI via GAM regression ### # Two ecision problem # Future stuy of 200 participants will upate θ # Data, x, are Binomially istribute given θ # Samples from θ are hel in vector object theta of length K # Corresponing samples from X θ are hel in vector object x of length K # The incremental net benefits are hel in vector object INB of length K ### library(mgcv) x <- rbinom(k, 200, theta) moel <- gam(inb te(x)) g.hat <- moel$fitte evsi <- mean(pmax(0,g.hat)) - max(0,mean(g.hat)) # to get gam() function # generate K atasets # fit GAM moel # extract fitte values # calculate EVSI 2.3 Choice of summary statistic T ( ) If we expect that the stuy ata x to be informative for a single economic moel parameter θ i, an if x is scalar we choose T (x) = x. If x is vector value, then we choose T (x) to be a sample estimator for θ i. This leas to quite natural summary statistics. So, for example, if we wish to calculate the expecte value of a two-arm, binary outcome trial to upate beliefs about an os ratio, then we our choice of T (x) woul be the sample os ratio. If we wish to upate beliefs about p > economic moel parameters {θ,..., θ p }, then we woul calculate p summary statistics {T (x),..., T p (x)}, where each T i (x) is the sample estimator for θ i. For example, if we wish to calculate the expecte value of a stuy to learn about the shape an scale parameters of a Weibull istribution, {θ, θ 2 }, an x are censore time-to-event ata, then we woul choose {T (x), T 2 (x)} to be the sample estimates {ˆθ, ˆθ 2 } erive from a Weibull survival moel. In the case of vector of summary statistics, {T (x),..., T p (x)}, we fit the multivariate regression moel, NB(, θ) = g {, T (x),..., T p (x)} + ε. (9) 3 Stanar error of EVSI calculate via the regression metho In the GAM metho our estimate of the posterior expecte net benefit for ecision option, conitional on ataset k is given by the value of the regression moel, evaluate at at T (x (k) ) (i.e. the k th fitte value). We enote this value ĝ{, T (x (k) )}, an the vector of k =,..., K fitte values as ĝ = [ĝ{, T (x () i ÊVSI = K )},..., ĝ{, T (x (K) i )}. The estimate EVSI is given by [ max ĝ{, T (x (k) )} ĝ{, T (x (k) )}, (20) k=

9 Strong, Oakley, Brennan Efficient computation of EVSI 9 where = arg max K ĝ{, T (x (k) )}. (2) k= Any GAM moel can be re-expresse as a linear moel with coefficients ˆβ. Coefficients ˆβ are mappe onto the fitte values ĝ via a preiction matrix P, where ĝ = P ˆβ. (22) Helpfully, P is returne by the preict.gam function in the mgcv package. The covariance matrix for ˆβ is returne as the $Vp component of the object returne by the gam function call. We enote this matrix ˆV β. The coefficients ˆβ are approximately Normally istribute an we can therefore we can express uncertainty about the true values of the target quantity g via the Normal istribution g N(ĝ, P ˆV β P T ). (23) For each ecision option, we raw a large number (say,000) of sample values of g from the above istribution. We enote these samples g (s) (s =,..., S). For each g (s) we calculate the EVSI via equation (20), replacing ĝ with g (s). We enote the sample EVSI values ẽ(s). The sample stanar eviation of ẽ (s) is an estimate of the stanar error we require. Note that for reasons of computational efficiency we woul not sample from (23) irectly, but instea woul (s) sample values ˆβ (s =,..., S) from the much lower imensional istribution N( ˆβ, ˆV β ) an then compute g (s) = P ˆβ(s). References Aes, A. E., Lu, G. an Claxton, K. (2004). Expecte value of sample information calculations in meical ecision moeling, Meical Decision Making, 24 (2): Brennan, A., Kharroubi, S., O Hagan, A. an Chilcott,. (2007). Calculating partial expecte value of perfect information via Monte Carlo sampling algorithms, Meical Decision Making, 27 (4): Claxton, K. an Posnett,. (996). An economic approach to clinical trial esign an research priority-setting, Health Economics, 5 (6): Eckermann, S. an Willan, A. R. (2007). Expecte value of information an ecision making in hta, Health Economics, 6 (2): Hastie, T. an Tibshirani, R. (986). Generalize aitive moels, Stat Sci, (3): Lunn, D., Spiegelhalter, D., Thomas, A. an Best, N. (2009). The BUGS project: Evolution, critique, an future irections, Statistics in Meicine, 28 (25): Oakley,. E., Brennan, A., Tappenen, P. an Chilcott,. (200). Simulation sample sizes for Monte Carlo partial EVPI calculations, ournal of Health Economics, 29 (3):

10 Strong, Oakley, Brennan Efficient computation of EVSI 0 R Development Core Team (203). R: A Language an Environment for Statistical Computing, R Founation for Statistical Computing, Vienna, Austria, ISBN Raiffa, H. (968). Decision Analysis. Introuctory Lectures on Choices Uner Uncertainty, Reaing, Massachusetts: Aison-Wesley. Strong, M., Oakley,. E. an Brennan, A. (203). Estimating multi-parameter partial Expecte Value of Perfect Information from a probabilistic sensitivity analysis sample: a non-parametric regression approach, Meical Decision Making. Welton, N.., Maan,.., Calwell, D. M., Peters, T.. an Aes, A. E. (203). Expecte Value of Sample Information for multi-arm cluster ranomize trials with binary outcomes, Meical Decision Making. Woo, S. (2006). Generalize Aitive Moels: Hall/CRC. An Introuction with R, Chapman an