Analytical Tools for Affordability Analysis. David Tate Cost Analysis and Research Division Institute for Defense Analyses

Transcription

1 Analytical Tools for Affordability Analysis David Tate Cost Analysis and Research Division Institute for Defense Analyses

2 Report Documentation Page Form Approved OMB No Public reporting burden for the collection of information is estimated to average 1 hour per response, including the time for reviewing instructions, searching existing data sources, gathering and maintaining the data needed, and completing and reviewing the collection of information. Send comments regarding this burden estimate or any other aspect of this collection of information, including suggestions for reducing this burden, to Washington Headquarters Services, Directorate for Information Operations and Reports, 1215 Jefferson Davis Highway, Suite 1204, Arlington VA Respondents should be aware that notwithstanding any other provision of law, no person shall be subject to a penalty for failing to comply with a collection of information if it does not display a currently valid OMB control number. 1. REPORT DATE MAY REPORT TYPE 3. DATES COVERED to TITLE AND SUBTITLE Analytical Tools for Affordability Analysis 5a. CONTRACT NUMBER 5b. GRANT NUMBER 5c. PROGRAM ELEMENT NUMBER 6. AUTHOR(S) 5d. PROJECT NUMBER 5e. TASK NUMBER 5f. WORK UNIT NUMBER 7. PERFORMING ORGANIZATION NAME(S) AND ADDRESS(ES) Institute for Defense Analyses,Cost Analysis and Research Division,4850 Mark Center Drive,Alexandria,VA, PERFORMING ORGANIZATION REPORT NUMBER 9. SPONSORING/MONITORING AGENCY NAME(S) AND ADDRESS(ES) 10. SPONSOR/MONITOR S ACRONYM(S) 12. DISTRIBUTION/AVAILABILITY STATEMENT Approved for public release; distribution unlimited 11. SPONSOR/MONITOR S REPORT NUMBER(S) 13. SUPPLEMENTARY NOTES Presented at the 12th Annual Acquisition Research Symposium held May 13-14, 2015 in Monterey, CA. 14. ABSTRACT 15. SUBJECT TERMS 16. SECURITY CLASSIFICATION OF: 17. LIMITATION OF ABSTRACT a. REPORT unclassified b. ABSTRACT unclassified c. THIS PAGE unclassified Same as Report (SAR) 18. NUMBER OF PAGES 24 19a. NAME OF RESPONSIBLE PERSON Standard Form 298 (Rev. 8-98) Prescribed by ANSI Std Z39-18

3 What is Affordability Analysis? Starting with 2010 s Better Buying Power memorandum, OSD has issued policy requiring acquisition programs to present affordability analyses at Milestone reviews This requirement is now part of Department of Defense Instruction The Defense Acquisition Guidebook (DAG) was updated in July 2013 to reflect the new requirement and provide guidance 5/19/2015 1

4 Affordability Analysis Content Affordability Analysis shows each program s planned development and production costs over time, in the context of The cost and schedules of the other programs in the relevant acquisition portfolio The projected available funding over the life cycles of those programs This task is assigned to Service leadership Not the program s responsibility Should reflect Service long-term planning 5/19/2015 2

5 Recommended Submission Formats (DAG) 300,000 CY $ 250,000 CY14$ B 200,000 All Portfolios Under Component TOA POM ,000 TOA RDT&E Procurement 100,000 O&M 50,000 MILPER 0 FY 83 FY 93 FY 03 FY 13 FY 23 FY 33 FY 43 Procurement Procurement + RDT&E + RDT&E Portfolios $B POM14-18 FY43 BY12 TWV Portion of O&M TWV O&M Requirements 3.50 $B POM14 18 F Transportation (Procurement + RDT&E) ,200 2, ,200 2, ,917 New System Total Lifecycle Costs (total reserved profile) 1, Light Tactical Vehicle (Not JLTV) Protection and Other Medium Tactical Vehicles Heavy Tactical Vehicles Joint Light Tactical Vehicle (JLTV) 5/19/2015 3

6 What Tools do Affordability Analysts Need? Reconcile inconsistent submissions Predict annual costs for alternative plans Estimate the consequences of various possible funding levels Assess affordability risk For the portfolio For each program 5/19/2015 4

7 Reconciliation of Inconsistencies Which programs are in the portfolio? How much total funding is available? How much of that total will each program get year by year? How many units will that buy? If Service plans or estimates have changed, need to be able to propagate those changes to other portfolios as well 5/19/2015 5

8 Estimated Cost of Alternative Plans If the current plan is Fiscal Year Quantity Cost then what would the annual costs be if instead we do Fiscal Year Quantity Cost???????? This is a hard problem. 5/19/2015 6

9 Why Is It Hard? Cost progress (aka learning curves ) Fixed costs at contractor and program levels Nonrecurring and non-end-item costs Production rate effects and incentives Causal ambiguity in historical data Schedule changes cause cost changes Cost changes also cause schedule changes Technical / management issues can cause both 5/19/2015 7

10 Competing Theories and Models Fixed/Variable apportioning (e.g., Balut et al.) Plant capacity varies with workload Program share of fixed costs is proportional to variable costs, some of which have learning Cobb-Douglas production function (Womer) Unit cost as a function of learning and rate Learning with forgetting (Benkard) Learning depreciates over time Discretionary capital investment (Rogerson) 5/19/2015 8

11 Estimated Impact of Change in Budget If there isn t enough money in the budget to do what we had planned, what happens? Programs stretch lower production rates If necessary, some may be canceled In order to predict the impact of a given schedule, we need a heuristic that can estimate how the portfolio manager would react to the new budget Requires costing ability described above Should also work for unexpected surplus funds 5/19/2015 9

12 Affordability Risk Assessment Affordability is often treated as a yes/no question, but reality is messier Cost estimates are uncertain Program outcomes are uncertain Budgets are uncertain Service priorities change over time New programs start The question of interest is not Is this program affordable?, but rather What is likely to happen if this is the plan? 5/19/

13 Risk Assessment Support Sensitivity analysis Vary one input at a time, see what happens Does not directly answer What is likely? Monte Carlo estimation Vary all uncertain inputs according to userspecified probability distributions Analyze the distribution of outcomes Requires credible driving distributions for many parameters and program characteristics 5/19/

14 Analyst Support Tool Notional Design Organize programs into portfolios Multiple alternative ways to partition the world Coordinate across multiple data sources SAR / DAES / PB / POM Individual program affordability analysis submissions Perform what-if and sensitivity analyses Alternative schedules Alternative budgets Revised cost estimates New programs 5/19/

15 Current Status: Software APASS: the Acquisition Portfolio Affordability Support System Web application SQL Server database Migrating to D3 graphics from Google API Data from multiple (conflicting) sources, organized by portfolio sets for analysis at the portfolio level To date, MDAP and pre-mdap data only 5/19/

16 IDA I A Portfolio Set Acquisition Portfolio Analysis Support System Home Army Equ1pment Dollars O Then-Year Year 2013 Show Serv1ce: Army Portfolio Set: Equipment Source: SAR By Budget Portfolio Set 0 Aifordabihty Portfolio Set Chart Type ~ Workspace Projections ~ C Reload Categories rlrdt&e l'liprocurement E:Jo&M D MILCON Years to Show Start Year: Earlrest 199~ End year: Latest 2034 Max Dollars :;:,... ~ ol!1 8 $8,000 $6,000 ~ $4,000 0 ~ ~ ~ $2,000 Total FY2013 Dollars Soldier and Squad Mission Command Indirect Fires Aviation Air and Missile Defense Dshow Actuals Source SAR lr $0 "'....,., co > ~ ~ ~ ~ ~! Fa ~ ~ ~ g) co ~ f? ~ ~ ~ ~ ~ 2 2 ~ fj <'V "' ~ <'V <'V <'V <'V <'V <'V <'V <'V <'V <'V Fiscal Year 5/19/

17 IDA I A Portfolio with Budget Acquisition Portfolio Analysis Support System Dollars 0 Then-Year Year 2013 Show 0 By Budget Portfolio Set 0 Affordat>ility Portfolio Set Serv1ce: Army Portfolio Set: Equ1pment Portfolio: Av1at1on Source: SAR Chart Type Workspace Projections ~Reload Categories i!:jrdt&e 12JProcurement Co&M DMILCON $5,000 Total FY2013 Dollars Budget UH-80M Black Hawk MQ-1 C Gray Eagle Years to Show Start Year: Eartrest 199~ End year Latest 2026 Max Dollars. rlshow Actuals ::;... "' <s "" 0 $3,750 Q ~ $2,500 0 ~ u_ 0; ~ $1,250 CH-47F AH-84E RemanufacturE AH-84E New Build Include 12lBudget li"jfit to budget Source SAR $0 "' ~ II) <o "- <0 0) ~ ~ ~ ~ ~ ~ ~ 0 ~ "' ~ ~ II) <o "- <0 0 RJ ;; ~ ~ ~ ~ ~ ~ ~ & &"' f;j ('(J ('(J ('(J ~ ('(J ('(J ('(J ('(J ('(J ~ ('(J ~ ('(J f? Fiscal Year 5/19/

18 IDA I Results of Fitting to Budget $5,000 ~ ~ $3,750 t? Jg 0 Q 1: (!) E $2,500 ~ l> ct (\')... C> ~ $1,250 ~ FY2013 Procurement Dollars Budget FY 2030: $1500M Budget UH-60M Black Hawk MQ-1 C Gray Eagle CH-47F AH-64E RemanufacturE AH-64E New Build $0 ~ ~ ~ ~ ~ ~ ~ m o ~ ~ ~ ~2? ~ ~fj~~~rfjrfj Fiscal Year 5/19/

19 Summary We are developing software tools to support Affordability Analysis (and oversight of Affordability Analysis) The current focus is on near-term ability to view and compare disparate data sources and alternative scenarios Spot discrepancies Produce reconciled views Provide what if? assessment of alternatives Secondary focus on estimating the impact on portfolios of alternative budgets 5/19/

20 BACKUP 5/19/

21 What is Affordability? Since the late 1990s, the military services have all spent large sums of money on programs that did not deliver their intended military capability Many of these programs spent billions and delivered nothing at all The purpose of Affordability Analysis is to avoid starting or continuing programs that cannot be produced and supported within reasonable expectations for future budgets. DoDI , Enclosure 8, Affordability Analysis and Investment Constraints (2015) 5/19/

22 DAG Format 1 TOA Top Line & Color of Money All Portfolios Under Component TOA CY $ 300,000 CY14$B 250,000 POM , , ,000 50,000 TOA RDT&E Procurement O&M MILPER 0 FY 83 FY 93 FY 03 FY 13 FY 23 FY 33 FY 43 5/19/

23 DAG Format 2 Service Portfolios Procurement + RDT&E Portfolios Procurement + RDT&E Portfolio $B POM14-18 FY43 5/19/

24 DAG Format 4 Portfolio Detail F Transportation (Procurement + RDT&E) 3.50 $B POM ,200 2,200 2,135 2, New System Total Lifecycle Costs (total reserved profile) , , Light Tactical Vehicle (Not JLTV) Medium Tactical Vehicles Joint Light Tactical Vehicle (JLTV) Protection and Other Heavy Tactical Vehicles 5/19/

25 Benkard Formulation Cn average unit cost in year n Ln production quantity in year T1 theoretical first unit cost β learning rate parameter δ annual forgetting rate γ production rate parameter n γ n k Cn = TL 1 n Lkδ k = 1 β n 5/19/