George Mason University The School of Public Policy and the School of Information Technology & Engineering. Cost Analysis

Transcription

1 Cost Analysis S.J. Balut Cost Analysis and Research Division Institute for Defense Analyses Th. R. Gulledge The School of Public Policy and The Department of Systems Engineering & Operations Research 01/22/ Decisions For our purposes, a decision is judgement based on: Experience Intuition Analysis 01/22/

2 Analysis is: Analysis A practical philosophy on how best to aid a decisionmaker with a complex of choice under uncertainty. Application of the scientific method problems of economic choice. Characteristics of analysis: Systematic Analytical Cannot supplant decisionmaking Cannot prove optimality Continuous dialogue between analyst and decisionmaker 01/22/ Analysis (Continued) Elements of analysis: Continuous iterative cycle Explores alternatives Costs Effectiveness Compare in light of the consequences Present to decisionmaker Brings to bear value judgements Rejects some alternatives Makes decisions Asks for more alternatives within framework established by decisions already made 01/22/

3 The Iterative Nature of Analysis* Clarifying the problem Determining objectives and criteria Opening new alternatives Searching out and designing alternatives Questioning assumptions Iteration Collecting data and Information Interpreting results Building and testing models Evaluating costs and effectiveness Examining alternatives for feasibility * Taken from E.S. Quade, Analysis for Policy Decisions. North-Holland, /22/ Policy Analysis Policy Analysis is a form of applied research carried out to acquire a deeper understanding of sociotechnical issues and to bring about better solutions. The general framework for policy analysis is presented by E.S. Quade: Quade, E.S. Analysis for Public Decisions, Third Edition. New York: North-Holland, /22/

4 Policy Analysis Studies Operations Research, Systems Analysis, Cost- Benefit Analysis, Economic Analysis, and Cost-Effectiveness Analysis are often employed in Policy Analysis Studies. Policy Analysis is broader than the above, taking into account the political and organizational difficulties associated with public decisions and their implementation. 01/22/ Wicked Problems Policy Analysis problems fall in the most difficult category. As noted by Quade, The problems of public policy are thus likely to be "wicked" problems; that is, they may have no definite formulation or stopping rule to tell the problem solver when he has a good solution. Moreover, a proposed solution is not true or false but good or bad. There may be neither an immediate nor even an ultimate test of a solution - the set of solutions may seem endless. Even for the comparatively simple problems involved in locating as freeway or changing a school curriculum, there is not a solution but a resolution - and this must rely on political 01/22/

5 Examples of Public Policy Problems Analysis alone is clearly not sufficient to provide a solution to such questions as: How much of a county budget should be allocated to welfare and what portion of this to birth control clinics in several cities? Are metropolitan transportation needs better served by a rapid transit system or my more and higher-performance freeways? Is there some legislative action that might end the increase in juvenile delinquency? 01/22/ Cost-Benefit Analysis* Cost-Benefit Analysis is an estimation and evaluation of net benefits associated with alternatives for achieving defined goals. Cost-Benefit Analysis is a generic term embracing a wide range of evaluative procedures which lead to a statement assessing costs and benefits relevant to project alternatives. The variety of problems addressed and the ingenuity which must be exercised in estimating costs and benefits make it particularly difficult, if not impossible, to design an all-purpose Cost-Benefit Analysis procedure. * Peter G. Sassone and William A. Schaffer. Cost-Benefit Analysis: A Handbook. San Diego: Academic Press, /22/

6 Operations Research* Operational Research is the attack of modern science on complex problems arising in the direction and management of large systems of men, machines, materials, and money in industry, business, government, and defense. Its distinctive approach is to develop a scientific model of the system, incorporating measurements of factors such as chance and risk, with which to predict and compare the outcomes of alternative decisions, strategies, or controls. The purpose is to help management determine its policy and actions scientifically. * British Operational Research Society, Operational Research Quarterly, Vol. 13 #3 (1962), /22/ Systems Analysis* Systems Analysis may be defined as inquiry to assist decisionmakers in choosing preferred future courses of action by 1. systematically examining and reexamining the relevant objectives and the alternative policies or strategies for achieving them; and 2. comparing quantitatively where possible the economic costs, effectiveness (benefits), and risks of the alternatives. It is more of a research strategy than a method or technique; and in its present state of development it is more like an art than a science, although scientific methods are utilized wherever possible. In sum, systems analysis may be viewed as an approach to, or a way of looking at, complex problems of choice, usually under conditions of uncertainty. * Gene H. Fisher. Cost Considerations in Systems Analysis. New York: American Elsevier, /22/

7 Economic Analysis Economic Analysis is a tool that is used in the search of more efficient courses of action. The elements of an Economic Analysis, and a discussion of economic choice in general, are presented in Chapter 7 of C.J. Hitch and R.N. McKean. The Economics of Defense in the Nuclear Age, R-346. Santa Monica: The RAND Corporation, /22/ Economic Analysis (1972) The definition of Economic Analysis is given in DoD Instruction , Economic Analysis and Program Evaluation for Resource Management, October 18, Economic Analysis is a systematic approach to the problem of choosing how to employ scarce resources and an investigation of the full implications of achieving a given objective in the most efficient and effective manner. As of 1972 Economic Analysis and Systems Analysis are synonymous within the Department of Defense. 01/22/

8 NES Dictionary Cost The amount paid or payable for the acquisition of materials, property, or services... Gene H. Fisher* The cost of something is measured by the resources used to attain it. The cost of attaining an objective at some point in time is measured by the resources not available for use in attaining alternative objectives. Economic Costs are benefits lost. It is for this reason that economic costs are often referred to as Alternative Costs or Opportunity Costs. It is in the alternatives, the foregone opportunities, that the real meaning of cost must always be found. * Gene H. Fisher. Cost Considerations in Systems Analysis. New York: American Elsevier, /22/ Effectiveness* In cases where there is a single goal to accomplish, the effectiveness of any alternative is the extent to which the goal is achieved. It will be helpful to define the two concepts: Effectiveness Scale: A scale indicating the degree to which the objective has been achieved. Effectiveness: The position on the effectiveness scale that the alternative has reached. * E.S. Quade, Analysis for Public Decisions, 3rd Edition. New York: North -Holland, 1987, p /22/

9 Effectiveness* Effectiveness is the performance or output received from an approach or program. Outputs are the products, functions, tasks, services, or capabilities and organization exists to produce, accomplish, attain, or maintain. Output measures are useful descriptors of functions, tasks, or missions performed by an organization, and of capabilities possessed. * DoD Instruction (Enclosure 3), /22/ Cost-Effectiveness Analysis* Cost-Effectiveness Analysis is a method for examining alternative means of accomplishing a desired military objective/mission for the purpose of selecting weapons and forces which will provide the greatest military effectiveness for the cost. * NES Dictionary 01/22/

10 The Economic Analysis Process* Define Purposes, Goals, and Objectives Formulate Assumptions Determine Alternatives Determine Costs Quantifiable Quantifiable Determine Benefits Non-Quantifiable Compare and Rank alternatives Using Techniques of Economic Analysis Non-Quantifiable Evaluate Uncertainties Data, Information, Experience, Judgment, and Analysis Are Required at Each Step in the Process DECISION Factors Other that Benefits and Costs * Source: Department of Energy, Economic Analysis: Methods, Procedures, Life-cycle Costing, Reviewing, and Validating, DOE/MA-0063, January, 1982, 01/22/ Key Point If the output is "valued" by the market, then the analysis is a Cost-Benefit Analysis. If the output is not "valued" by the market, then the analysis is a Cost-Effectiveness Analysis. The inputs are always "valued" by the market. 01/22/

11 Characteristics of Military Systems* In a consumer-oriented industry, the variables considered in decisionmaking can often be reduced to a common denominator, usually related to cost. One attribute of a military system is the absence of a market mechanism to determine the relation between a proposed system's cost and its military worth (effectiveness). Thus, cost and effectiveness are frequently incommensurable; i.e., there is no generally accepted standard for measuring the relation between the value of a military system and its cost. * See Hitch and McKean, Chapter 9. 01/22/ Summary of Key Point Value = ( Benefit in this use ) ( Benefit in next best use ) Output Units Measure Input Units* Measure Market Valued Dollars Revenue Dollars Cost Non-Market Valued Physical Effectiveness Dollars Cost * In some cases cost may not be measured in dollars. For example, certain strategic resources (e.g., key scientists), if lost, may involve large opportunity costs. The assignment of a dollar value to this cost may be difficult, but for the purposes of this presentation we assume that such an assignment is possible. 01/22/

12 Cost-Benefit Analysis Consider a project that involves a change from the status quo. Let: v i = valuation ith individual attaches to change: i=1,2,...,n v i > 0 Change increases ith individual's welfare and "v" is the v i < maximum amount that would be paid for the change 0 Change decreases ith individual's welfare and "v" is the least amount that would be accepted to agree to the change Project sanctioned if: v i > 0 i 01/22/ Cost-Effectiveness Analysis Consider the same problem, except add a constraint E on effectiveness or C on cost. Let: m = number of gainers 1 m = number of losers 2 m + m = n 1 2 The, Cost-Effectiveness Analysis calculations take the form of either: Maximize m m 1 i = 1 2 St : vi < C i= 1 v i 01/22/ , Minimize m m 2 i= 1 v i 1 St : vi > E i = 1 12

13 Cost-Benefit Analysis Cost Analysis Cost-Effectiveness Analysis Systems Analysis Policy Analysis 01/22/ Cost Estimating NES Dictionary Cost and price estimating is defined as the art of predetermining the lowest realistic cost and price of an item or activity which assures a normal profit. Gene H. Fisher*... "making an estimate" of the cost of something implies taking a rather detailed set of specifications and "pricing them out." * Gene H. Fisher. Cost Considerations in Systems Analysis. New York: American Elsevier, /22/

14 Cost Estimating and Analysis* "Differences in Disciplines" "The first issue is the difference in disciplines between cost estimating and analysis... Even though the two disciplines are functionally separate, they are interrelated so that one depends on the other. Cost estimating, the process of predicting or forecasting the cost of a work activity or a work output, depends on the output of cost analysis, the process of studying and organizing past cost and future estimates." "...the difference between estimating and analysis must continually be put in perspective. Each has its own function and purpose in the lifecycle of a program. While they are interrelated their purposes should never be misconstrued to accept one as a form of the other." * Quotes from N. Albert, "Merger Concerns and Objectives," Dollars and Sense, March/April, /22/ Summary of Terminology Policy Analysis is the broadest analytical framework. Policy Analysis includes a consideration of all intangibles. Systems Analysis is a subset of policy analysis in that distributional effects are excluded from consideration. Cost-Benefit Analysis and Cost-Effectiveness Analysis fall under the Systems Analysis Framework. Cost-Effectiveness Analysis is an analytical approach that uses cost and effectiveness as decision criteria. Since 1968 Cost-Effectiveness Analysis has been known as Economic Analysis within the DoD. Cost-Benefit Analysis is an analytical approach that uses costs and market valued benefits as decision criteria. Cost Analysis is a subset of Cost-Benefit Analysis and Cost- Effectiveness Analysis. 01/22/

15 Hierarchy Policy Analysis (PA) Encompasses SA, adding a concern for distributional effects (who pays, who benefits, implementation, acceptance, etc.) Seeks to do all that SA does, but to see that it is done with equity. Systems Analysis (SA) Grew out of OR and may be considered to encompass it, adding economic considerations and inquiry into goals and their interactions with means. Seeks to see that the right thing is done, and not only better, but more cheaply. Cost-Effectiveness (CE) Belongs with SA (involves economic considerations but not distributional effects). Is a tool or procedure of SA Is an incomplete SA Almost stands alone provided... Operations Research (OR) (Originally) sought no more than to do something better, to use scientific methods to get the most out of available resources 01/22/ Cost-Effectiveness and Politicians* Consider the following quote by Quade * I regard the acceptance of cost-effectiveness analysis in an environment populated with politicians as remarkable --- for politics is geared to winning the voters by outlining the benefits of programs before the means and costs are determined. * E.S. Quade, A History of Cost-Effectiveness, P Santa Monica: The RAND Corporation, /22/

16 Background Economic Theory and Defense Decisionmaking A Stroll Through History People Events Trends 01/22/ History of Analysis Throughout 1940s 1950s/early 1960s Late 1960s/early 1970s 1970s/1980s 1990s Draw on economic theory Rise of Operations Research Rise of Systems Analysis Fall of Systems Analysis Emphasis on Cost Analysis Rise of Cost and Operational Effectiveness Analysis 01/22/

17 Rise of Operations Research 1941 Operations Research invented 1942 ASWORG established During WWII After WWII Operations Research evolved Separation of mission responsibility broke down 1946 RAND established 1947 Key West Agreement - DoD established 1948 "Weapon Systems Analysis" 1949 "Systems Analysis" 01/22/ Rise of Systems Analysis 1950 RAND Cost Analysis Department established Novick pioneered Cost Analysis 1958 Defense Reorganization Act - PERT developed Late 1950s DoD Civilians losing control 1960 RAND suggested applying economic analysis to government decisions 1961 McNamara appointed Hitch - Hitch established DASD (Systems Analysis) - Beginning of centralized decisionmaking using cost-effectiveness as criteria "Whiz Kids" won battles 1965 President Johnson extended PPBS to all Federal Agencies - Systems Analysis office elevated to ASD 01/22/

18 Fall of Systems Analysis 1965 Flag officers began speaking out against costeffectiveness 1968 Schlessinger warned Congress of limitations 1969 Laird de-emphasized role of Systems Analysis - Term "Economic Analysis" substituted Laird/Packard resolve to fix procurement ills 1972 Shift towards "Program Evaluation" - ASD(SA) changed to ASD (Program Analysis & Evaluation) Through 1970s Cost focus shifted from planning to programming and budgeting 01/22/ Emphasis on Cost Analysis 1971 Design to Cost 1972 CAIG established 1974 VAMOSC 1981 Carlucci Initiatives 1984 ICE 1985 Service Cost Analysis Centers 01/22/

19 Rise of Cost and Operational Effectiveness Analysis 1991 DoD Instruction requires COEAs 1992 Weapon System Cost Analysis Division established 1997 Evaluation of Alternatives 01/22/ Institutional Requirements for Cost-Effectiveness Analysis Department of Defense* "cost and operational effectiveness analyses shall be prepared and considered at milestone decision reviews of acquisition category I programs, beginning with Milestone I, Concept Demonstration Approval." "...(they) shall be tailored and implemented in support of acquisition category II, III, and IV programs as deemed appropriate by the DoD component Acquisition Executives." Non-Defense Part of Government** "Benefit-cost analysis is recommended as the technique to use in a formal economic analysis of Government programs or projects. Cost-effectiveness analysis... can be appropriate when the benefits from competing alternatives are the same or when a policy decision has been made that the benefits must be provided." * DoD Instruction , Part 4, Section E, "Cost and Operational Effectiveness Analysis, February 23, **Office of Management and Budget Circular No. A-94, Revised, October, /22/

20 DoD Instruction This document outlines policy guidance and establishes a framework for consistent application of: 1. Economic analysis on proposed programs, projects, and activities, 2. Program evaluations of on-going activities, and 3. Establishes the Defense Economic Analysis Council. 01/22/ Efficiency and Effectiveness* The determination of efficiency and effectiveness is implicit in the assessment of the cost effectiveness of alternative approaches, and is accomplished by: Systematically identifying the benefits and other outputs and costs associated with alternative programs, missions, and functions and/or of alternative ways for accomplishing a given program. Sensitivity Analysis on key decision variables and assumptions on which decisions are based. Evaluating alternative methods of financing investments, such as lease or buy. Using the benefits and costs to compare the relative merits of alternatives as an aid in making trade-offs between alternatives, recommending the cost-effective alternative, and in establishing or changing priorities. * DoD Instruction , p. 2 01/22/

21 Two Clear-Cut Roles In the Economic Analysis literature, two individuals are referred to: The analyst, and The decisionmaker. This division usually does not exist in practice; the roles of the analyst and the decisionmaker invariably overlap. 01/22/ Role of the Economic Analyst The function of the analyst is to present a schedule of alternatives and not to optimize in the sense of recommending the selection of a particular alternative. A crucial role for the analyst is that of deciding which alternatives should be considered by the decisionmaker. 01/22/

22 The Analytical Steps Define the objective(s), Construct the alternatives, Measure the cost of the alternatives, Measure the effectiveness of the alternatives, Discuss the findings: Quantitative results, Omitted factors, Risks and uncertainties, Judgement calls required of decisionmakers, Recommendations (two views). 01/22/ Example of Equal Effectiveness and Equal Cost Comparisons* EFF z z x x v v EFF v x v x z v x z z x v z COST COST z x v System A System B System C * Taken from DoD Instruction , Part 4, Section E, 23 February /22/

23 The Presentation to the Decisionmaker Under the assumption of a fixed budget constraint, the analyst would eliminate from consideration all systems but the one that yielded the greatest effectiveness. If effectiveness were predetermined, the analyst would attempt to minimize cost. Since neither scenario is likely, the analyst must assume that a range of levels of effectiveness and cost might be acceptable to the decisionmaker. 01/22/ The Best Alternative The determination of a rule to tell you what is best is seldom solved even in the most ordinary and prosaic circumstances. Consumer Reports does not tell you which washing machine to buy -- it merely lists dollar costs and some important characteristics and indicators for performance that you should consider. There may be quite a problem, however, in deciding which characteristics and indicators are significant to your choice. In the immensely more complicated problems of public policy, we can seldom do much better than take a similar approach, that is, display the various consequences, costs, and spillover associated with each alternative and their sensitivities to changes and uncertainties -- the approach we have earlier called the score-card approach -- and put the criterion problem squarely up to the decision-makers (Quade*, pp ) * E.S.Quade, Analysis for Public Decisions, 3rd Edition. New York: North-Holland, /22/

24 Honest Open Qualities of a Good Analysis Alternatives not arbitrarily suppressed Quantitative factors selected without bias Calculations accurate Explicit Verifiable Self-correcting Objective Unbiased Not self serving 01/22/ Limitations of Analysis Analysis is necessarily incomplete, Measures of effectiveness are inevitably approximate, No satisfactory way exists to predict the future, Analysis falls short of scientific research, Analysis tends to ignore factors that cannot be reduced to numbers, and tends to over-emphasize those that can, The approach may bias against the new in favor of the old, Cost-Effectiveness analysis appears biased against new systems to replace those now in operation. 01/22/

25 Some Common Fallacies Underemphasis of problem formulation, Inflexibility in the face of evidence, Adherence to cherished beliefs, Parochialism, Communications failures, Over-concentration on the model, Excessive attention to detail. Neglect of the question, Incorrect use of the model, 01/22/ Some Common Fallacies (Continued) Disregard of the limitations, Concentration on statistical uncertainty, Inattention to uncertainties, Use of side issues as criteria, Substitution of the model for the decisionmaker, Neglect of the subjective elements, Failure to reappraise the work. 01/22/