Use of Joint Confidence Level (JCL) Data for Programmatic Success

Transcription

1 Use of Joint Confidence Level (JCL) Data for Programmatic Success ISPA/SCEA Conference June 2011 Rey Carpio - Tecolote Research, Inc Los Angeles Washington, D.C. Boston Chantilly Huntsville Dayton Santa Barbara Albuquerque Colorado Springs Ft. Meade Ft. Monmouth Goddard Space Flight Center Ogden Patuxent River Silver Spring Washington Navy Yard Cleveland Dahlgren Denver Houston Montgomery New Orleans Oklahoma City Tampa Tacoma Vandenberg AFB Warner Robins ALC PRT-80, 21 March 2011 Approved for Public Release 1of 16

2 Consumer Price Index News Soundbite BEIJING China said inflation remained high last month, fueling further doubt about the government's ability The consumer-price index (CPI) rose 4.9%, Last month's rise was in line with the 4.9% rise in January, and was just slightly higher than economists' expectations. China's economic data are distorted by the timing of its Lunar New Year holiday, which fell earlier in the month this year than in Despite the 4.9%, inflation pressure remains intense, The implication is that tightening pressure is still on, and before inflation is tamed, consumers will continue to be pressed. China's leaders have pledged to tame consumer prices, and expressed confidence in their ability to do so. Beijing's success or failure could have global ramifications. The government faces a delicate balance, wanting to cool prices without clamping down so hard that they strangle economic growth in China, which has driven global growth in recent years. INDEX.Index.index.indeX.INDEX PRT-80, 21 March 2011 Approved for Public Release 2of 16

3 Another Soundbite NASA Program Manager reported low JCL index which is fueling further doubt about the program s ability to tackle what Congress officials have called their main scientific priority this year The JCL index (JCLI) rose 4.9% Last month's rise was in line with the 4.9% rise in January, and was just slightly higher than NASA HQ s expectations. Program's February confidence data are distorted by the timing of Continuing Resolution, which persisted this year than in Despite the 4.9%, pressure to succeed remains intense, The implication is that tightening pressure is still on, and before technical risk is tamed, Program will continue to be pressed. Program management have pledged to manage technical risks, and expressed confidence in their ability to do so. Program's success or failure could have NASA-wide ramifications. The Program faces a delicate balance, wanting to control costs without clamping down so hard that they strangle technical growth, which has driven scientific growth in recent years. JCL is an INDEX PRT-80, 21 March 2011 Approved for Public Release 3of 16

4 JCL Index A PM Perspective Viewed as an INDEX and used as an INDEX Index trend provides a measurement of change in the overall confidence level of the program s plan with respect to the cost, schedule, and threats managed by the Program Definition: statistical estimate constructed using information that captured uncertainty, risk, cost, and schedule of program representative elements whose data are collected periodically JCL can be one of the most closely watched program indices PRT-80, 21 March 2011 Approved for Public Release 4of 16

5 Monthly PM s Status Mtg Ok, let s start with project schedule then, the cost estimates, and have the risk people give us the state of project risk PRT-80, 21 March 2011 Approved for Public Release 5of 16

6 Project Scheduler the project schedule doesn t look that good we are couple months behind schedule and recovery is not in sight PRT-80, 21 March 2011 Approved for Public Release 6of 16

7 Cost Estimates what do you mean, we re over our budget?... what about that S-curve stuff you told me last month where are we now? PRT-80, 21 March 2011 Approved for Public Release 7of 16

8 Risk Analysis Boss, can I just show you the risk list and we can discuss whichever one you feel like talking about I can easily talk to you about this one but not that one PRT-80, 21 March 2011 Approved for Public Release 8of 16

9 the moment Boss, you heard from your staff schedule, cost, and risk. and we are ready to hear what you have to say cause we don t see the picture PRT-80, 21 March 2011 Approved for Public Release 9of 16

10 Integrate, integrate, schedule is behind, cost estimate is ugly, and risks are uglier what s my trade space and which risks are driving me off the cliff show me an integrated metric (index) on where I am where I m going what s the net effect of it all? PRT-80, 21 March 2011 Approved for Public Release 10 of 16

11 How JCL Index is put together Requirements Total Project Cost (w/ LOE) p Technology Resources Effort Work to be done (i.e. estimated touch Productivitylabor + material cost) Availability Capacity to do work External Events Schedule A plan to do work /1 10/9 9/15 11/30 Project Completion Date $ Budget Presented at the 2011 ISPA/SCEA Joint Annual Conference and Training = Source Workshop of Uncertainty PRT-80, 21 March 2011 Approved for Public Release 11 of 16

12 The Situation: Satellite XYZ Program JCL index: (100 means tracking to Target) Current JCL = 22.3% Target JCL = 55% Index = 100 * (Current JCL/Target JCL)) Analysis shows two test activities are scheduled and they are driving cost and schedule PM needs to decide whether to keep or eliminate Test #2 Test #2 Specifics Schedule Test #2 is about 3 months long Cost about $150M Risk depends Test #2 content is shake & bake of the satellite If Test #2 is taken out, there will be impact to reliability, deployment mechanisms, and other aspects PRT-80, 21 March 2011 Approved for Public Release 12 of 16

13 Scatter Plot If Test #2 is deleted ($150M and 3 months), do we improve confidence index for the plan and JCL frontier After Index: Compare previous JCL and the new JCL Curves Point of marginal benefit Trade space Up/out reporting data Before and After JCL comparison Manage stakeholder expectations Down/in management data Capture/compare to historical norm Trending over time Before Index: PRT-80, 21 March 2011 Approved for Public Release 13 of 16

14 Using the Schedule/Cost CDF/PDF If Test #2 is deleted, then what is the possible range of schedule on the curve at a given, or range of confidence level Plan is shorter and planned costs for Test 2 ($150M) are deleted Increased uncertainty Cost range shifting to the left, but Schedule range shifting to the right Confidence Confidence Level Level (CDF) (CDF) 100.0% 100.0% 90.0% 90.0% 80.0% 80.0% 70.0% 70.0% 60.0% 60.0% Current Current Plan Plan Total Total Cost Cost Current Plan Current Plan 50.0% 50.0% 40.0% 40.0% 30.0% 30.0% 20.0% 20.0% 10.0% 10.0% 0.0% 0.0% Scenario Scenario 819 1,025 1,232 1,438 1,645 1,851 2,058 2,264 2,471 2, ,025 1,232 1,438 1,645 1,851 2,058 2,264 2,471 2,678 TY$K TY$K Probability Histogram Confidence Level (CDF) Probability Histogram Confidence Level (CDF) 4.5% 4.5% 4.0% 4.0% 3.5% 3.5% Probability (Histogram) 3.0% 3.0% 2.5% 2.5% 2.0% 2.0% 1.5% 1.5% 1.0% 1.0% 0.5% 0.5% 0.0% 0.0% Probability (Histogram) Net effects of increased technical risk, accelerated schedule, and cost reduction May include additional risk mitigation cost/task/schedule to accommodate the deletion of Test #2 Comparison Is the Index getting better? Cost CDF/PDF will undergo similar analysis PRT-80, 21 March 2011 Approved for Public Release 14 of 16

15 Annual Risk (Fan) Charts Test #2 is deleted from year #3 ($150M and 3 months) Annual comparison of plans doesn t provide enough information Compare previous river chart with new river chart Relative and absolute movement of the Bound and JCL target level Before - $78M Total Delta for 55% Provides a perspective, with relation to time, on program uncertainty After - $88M Total Delta for 55% PRT-80, 21 March 2011 Approved for Public Release 15 of 16

16 It s an INDEX Just like Consumer Price Index JCL is an INDEX Treat it like an index Develop the before and after pictures Search for answers for cost and schedules Identify the risks trade space (risk-cost-schedule dimensions) JCL is an INDEX PRT-80, 21 March 2011 Approved for Public Release 16 of 16