7 Analyzing the Results 57

Transcription

1 7 Analyzing the Results 57 Criteria for deciding Cost-effectiveness analysis Once the total present value of both the costs and the effects have been calculated, the interventions can be compared. If one intervention achieves more and at no greater cost than the alternatives or if it costs less to achieve the same, it is clearly the superior choice. If one intervention is both more costly and more effective, it is not immediately possible to draw a clear conclusion. Instead: Calculate the incremental cost-effectiveness of the more expensive option (the extra effectiveness and the extra cost above the alternative) and consider whether the extra achievements are worth the additional costs. It is not necessary to know exactly what these additional achievements are worth but only whether it is likely to be more than the additional costs. This is an example of the switching value technique. Alternatively, estimate how much additional investment would be required for the less expensive intervention to achieve a similar impact to the more expensive intervention, and compare the incremental costs of both interventions. Appraisals, as opposed to evaluations, are done before project implementation and afford the opportunity to define the alternative interventions as having the same scale, either in terms of level of inputs or outcomes. In this particular case the ratio of cost to effectiveness is a suitable indicator to use: the intervention with the lowest cost-effectiveness ratio is the preferred option.

2 58 Cost-benefit analysis Comparisons of project options in cost-benefit analysis can be done using one of three related indicators: the cost-benefit ratio (CBR), the net present value (NPV), and the internal rate of return (IRR). The cost-benefit ratio is equivalent in construction to the cost-effectiveness ratio in that the discounted stream of project costs is divided by the discounted stream of benefits. The NPV of an investment which costs C t in year t and generates benefits B t in year t and has a life of n years at a discount rate of i per year is: n B t C t NPV = (1 + i ) t t = 0 The IRR is the value of i that is consistent with a NPV of zero. If the cost and benefit measures truly capture their full value, then NPV > 0, or CBR > 1 or IRR > the discount rate would automatically imply that the project is worth supporting. Since this is in practice difficult to do, the case for supporting such an intervention can be strengthened by examining the CBR, NPV or IRR of other alternatives designed to meet the same objectives. The three techniques are equivalent although they do differ in how sensitive they are to different assumptions. The IRR, for example, is highly sensitive when the ratio of capital to recurrent costs is low, a characteristic of most nutrition projects. A small change in assumptions about key variables such as prices of inputs or the value of benefits can make a dramatic difference to the IRR. Where a single crucial piece of information is missing or its value is highly uncertain, it is possible to recast the question and ask what value this variable should have in order for the choice to shift from one option to another the so-called switching value. For example,

3 59 What is the minimum value a life would have to have to make this project worthwhile doing? What is the minimum increase in productivity that must be achieved to justify investment in this project? Dealing with uncertainty Some degree of uncertainty will be attached to any prediction of costs, effectiveness and benefits. This is due to limitations in our knowledge of the value of underlying variables and in our understanding of relationships between variables. For some variables we have good data on current values but know they can vary in poorly defined ways making predictions difficult (input prices might be an example). Other variables are likely to be fairly stable over time but we are uncertain what their value actually is (e.g. discount rates). Some technical relationships are well-established (e.g. the relationship between breastfeeding and mortality), others are not (e.g. productivity implications of vitamin A deficiency, mortality implications of anemia). Coverage is relatively easy to predict, compliance often is not. The existence of uncertainty means that the estimates of costs and benefits may not be correct. This does not always matter. The probable difference between the predicted and actual costs and benefits may be small, or it may be quite large but not affect the relationship between costs and effects, or, it may affect the relationship between costs and affects but in such a way that the cost-effectiveness ordering remains the same. It makes sense then to focus attention on those variables which might reasonable take any one of a wide range of values, or those variables to which the cost-benefit results (or relative cost-effectiveness orderings) are likely to be particularly sensitive. An example of the latter is the discount rate. Many of the benefits of nutrition programs accrue a number of years in the future after investment, and estimates of their value will depend crucially on the value of the discount rate used. The larger the discount rate, the smaller the

4 60 value of the benefits, and the less attractive the NPV, CBR or IRR. The distribution of benefits over time can be very different between projects, making decisions based on cost-effectiveness analysis sensitive to the choice of discount rate. Similar arguments apply to the decision about project horizon, the number of years after project implementation for which project benefits will be included. Clearly uncertainty may have important implications for the conclusions drawn and must be accommodated in some fashion in the analysis. While the value of some variables may not be known with certainty, it is usually possible to define upper and lower limits beyond which it is unlikely they would be. It is sometimes even possible to define the probabilities attached to a range of values they might have. The first kind of information is used in sensitivity analysis, the second in risk analysis. Sensitivity analysis Sensitivity analysis explores how plausible changes in assumptions about uncertain variables affects the conclusions. The cost-effectiveness (or cost-benefit) of the alternative interventions is recalculated for each new assumption (or set of new assumptions) to see if this affects the conclusions. If it does not, the conclusions are considered robust. If it does, then it is important to specify the conditions under which the different conclusions will hold. Recording values of cost and benefit components in a spreadsheet simplifies the process of calculation, and facilitates the exploration of alternative assumptions. For discount rates, it is illuminating to recalculate the results using rates of 3%, and 10% two values commonly employed in health project analysis in addition to whatever rate is considered appropriate in the particular country.

5 61 Risk analysis Risk analysis provides a tool for estimating the probability of project outcomes (or costs) based on estimates of the probability of project parameters. A series of random selections of values from the probability distribution of parameters are made to generate a probability distribution of project effects or costs (monte carlo analysis), from which the mean and variance of project cost-effectiveness or cost-benefit can be determined. Cost-effectiveness or cost-benefit comparisons are then expressed in probability terms: for example, there is an x% chance that intervention A is superior to intervention B. Barnum (1995) provides an example from the health sector. Purposeful bias If, at the outset, it is clear that one intervention stands out from the rest, the case for it can be strengthened by adopting a purposeful bias in the calculations choosing a value for uncertain variables that penalizes the attractive option. If, despite this, the intervention still emerges as the preferred option, the conclusion can be accepted with a high degree of confidence. On the other hand, it is unhelpful to adopt a bias in assumptions which favors the option which ends up being recommended as a result of the costeffectiveness or cost-benefit analysis. Such a strategy complicates interpretation and may lead to rejection of conclusions which were in fact valid but made over-inflated claims. For example, Basta, et al., (1979) calculated the benefit-cost ratio of an iron supplementation intervention for male rubber plantation workers to be a dramatic 260:1, but this result involved not incorporating the cost of identifying who is anemic, attributing the entire value of the extra latex to labor productivity, not incorporating into the calculation the costs of attrition, and not adjusting for the result that only piece work laborers seem to have differential gains from taking iron versus placebo (Behrman, 1992). The decision-maker is left wondering

6 62 how much less the real ratio is and whether it might even be below 1, invalidating the conclusions entirely. Other considerations Scope A project may be cost-effective but not big enough to address the whole problem. For example, hospital-based breastfeeding promotion may be highly cost-effective but it will never reach those women who do not give birth in hospitals; sugar fortification may be a very efficient way to reduce vitamin A fortification but it will not be effective in those groups not consuming centrally processed (fortified) sugar. The appropriate way to tackle this is to look at what combination of interventions is most cost-effective to tackle the defined level of problem. This will usually mean starting with the most efficient strategy first and then exploring the most cost-effective ways to tackle the remaining problem. Equity For many nutrition programs income redistribution will be the justification for government involvement. Estimating the distributional effects of a project will be a crucial element of the analysis of those projects. The Bank has a large literature on how to estimate equity impacts of social sector projects. Sustainability The fiscal requirements or recurrent costs of maintaining projects are important determinants of sustainability. During project design and exploration of alternatives, ways of minimizing these costs should be explored. The appraisal should look for realistic assessments of the likelihood of getting adequate recurrent funding, and assess alternate ways to substitute capital for recurrent inputs. Examples include better quality equipment that does

7 63 not require as much maintenance; periodic assessments rather than cumbersome record keeping, curriculum reform rather than in-service training. Institutional and political factors Limited institutional capacity or institutional resistance to change can sabotage otherwise well-designed projects. Political instability is also likely to undermine project effectiveness in a number of ways. Both are likely to be reflected in delays or short-falls of funding (particularly recurrent funding) with attendant loss of benefits.