- 1. MANAGING COST, SCHEDULE, AND TECHNICAL PERFORMANCE RISK AT Y-12 Increasing the Probability of Program Success 1 Niwot Ridge Consulting
- 2. Starting with DOE Guidance 2
- 3. The DOE Risk Management Lifecycle 3
- 4. DOE G 413.3-7A Risk Process 4 DOE G 413.3-7A, page 9
- 5. Connecting the Dots Between the Elements of the Performance Measurement Baseline 5 Risk SOW Cost WBS Schedule TPM PMB
- 6. There are two types of Uncertainty Uncertainty about the functional and performance aspects of the program’s technology that impacts the produceability of the product or creates delays in the schedule Uncertainty about the duration and cost of the activities that deliver the functional and performance elements of the program independent of the technical risk 6 Technical Programmatic
- 7. Risk Assessment and Management Techniques Vary with Maturity† 7 Add a Risk Factor or Percentage to the critical paths A “bottom line” Monte Carlo or Range analysis Detailed Monte Carlo for each WBS element Expert Opinions in a Database with assessment Detailed Bayesian Network Analysis Increasing Detail and Difficulty IncreasingPrecisionandValue ¨ There are several approaches to building a Risk Tolerant Performance Measurement Baseline ¤ First recognize where you are on the curve ¤ Then recognize there is value in moving further up the curve † Ron Coleman Litton TASC, 33rd ADoDDCAS, Williamsburg, VA
- 8. Risk is Different from Uncertainty Knowing this Difference is Critical to Success ¨ Cost estimating methodology risk resulting from improper models of cost ¨ Cost factors such as inflation, labor rates, labor rate burdens, etc ¨ Configuration risk (variation in the technical inputs) ¨ Schedule and technical risk coupling ¨ Correlation between risk distributions ¨ Requirements change impacts ¨ Budget Perturbations ¨ Re–work, and re–test phenomena ¨ Contractual arrangements (contract type, prime/sub relationships, etc) ¨ Potential for disaster (labor troubles, shuttle loss, satellite “falls over”, war, hurricanes, etc.) ¨ Probability that if a discrete event occurs it will invoke a project delay 8 Risk stems from known probability distributions Uncertainty stems from unknown probability distributions
- 9. Schedule Risk Management … ¨ Seeks to anticipate and address uncertainties that threaten the goals and timetables of a project ¨ Recognizes unmitigated risks lead rapidly to delays in delivery dates and budget overages that undermine confidence in the schedule and in the project manager ¨ Is process oriented, guided by DOE G 413.3-7 ¨ Accepts a certain level of risk, regular and rigorous risk analysis and risk management techniques serve to defuse problems before they arise ¨ Defines an Integrated Master Plan that reflects the development phases and the hierarchical structure of the system. : Risk Based Planning
- 10. A sample Risk Management System at Johnson Space Flight Center 10
- 11. Connecting the Dots, Again 11 Risk SOW Cost WBS Schedule TPM PMB Named Deliverables defined in the WBS BCWS at the Work Package, rolled to the Control Account TPMs attached to each critical deliverables in the WBS and identified in each Work Package in the IMS, used to assess maturity in the IMP The Products and Processes that produce them in a “well structured” decomposition in the WBS Schedule contains all the Work Packages, BCWS, Risk mitigation plans, and rolls to the Integrated Master Plan to measure increasing maturity Technical and Programmatic Risks Connected to the WBS and IMS
- 12. The Basics of Probabilistic Schedule12
- 13. All Schedule Activities Are Random Variables, with Probability Distributions 13
- 14. Cost does not have a linear relationship with schedule. Basic Principles of Probabilistic Cost14
- 15. Keys to Cost Estimating Success 15 ¨ Start with guidance on cost estimating. ¨ Tailor the guidance to fit the problem domain. ¨ Verify the processes work and add value. ¨ Improve the fidelity of estimates with feedback. ¨ Adjust estimating parameters to match actuals.
- 16. Basic Principles with Probabilistic Cost Estimating Relationships (CER) 16 ¨ Cost estimates involve many CERs ¤ Each of these CERs has uncertainty (standard error) ¤ CER input variables have uncertainty (technical uncertainty) ¨ Combine CER uncertainty with technical uncertainty for many CERs in an estimate ¤ Usually cannot be done arithmetically; must use simulation to roll up costs derived from Monte Carlo samples n Add and multiply probability distributions rather than numbers n Statistically combining many uncertain, or randomly varying, numbers ¤ Monte Carlo simulation n Take random sample from each CER and input parameter, add and multiply as necessary, then record total system cost as a single sample n Repeat the procedure thousands of times to develop a frequency histogram of the total system cost samples n This becomes the probability distribution of total system cost
- 17. The Cost Probability Distributions as a function of the weighted cost drivers 17 $ Cost Driver (Weight) Cost = a + bXc Cost Estimate Historical data point Cost estimating relationship Standard percent error boundsTechnical Uncertainty Combined Cost Modeling and Technical Uncertainty Cost Modeling Uncertainty
- 18. Basic Principles of connecting cost models with the IMS involve three steps 18 ¨ Step 1: Define “likely–to–be” program ¤ Using deterministic inputs from the Independent Technical Assessment (ITA) ¨ Step 2: Quantify the probability distributions describing the modeling uncertainty of all CERs, cost factors, and other estimating methods ¤ Specifically, the type of distribution (normal, triangular, lognormal, beta, etc.) ¤ The mean and variance of the distribution ¨ Step 3: Quantify the correlation between all WBS elements that are estimated using CERs and other methods ¤ If unknown, assess whether No correlation, Mild correlation, or High correlation, for example: n None: r = 0, Mild: r = ±0.2, High: r = ± 0.6 ¤ Correlation affects the overall cost variance
- 19. Basic Principles 19 ¨ Step 4: Set up and run the cost estimate in a Monte Carlo framework (e.g., Crystal Ball, @RISK), resulting in a “baseline” estimate ¤ This will provide a probability distribution of the cost based on cost estimating model uncertainty only ¤ Report the MEAN as the baseline expected cost ¨ Step 5: Now incorporate technical uncertainty and discrete risks ¤ Step 5a: Set up a new estimate which also contains any “discrete risk” events that are to be guarded against n Quantify appropriate modeling uncertainties and correlations, as in Steps 2 and 3, for these discrete risks ¤ Step 5b: Define the probability distributions for all CER input variables n Also may need to quantify correlation between CER input variables
- 20. Basic Principles of connecting cost models with the IMS involve three steps 20 ¨ Step 6: Re–run the Monte Carlo simulation with random CER input variables and discrete risk events, resulting in a final “risk–adjusted” estimate ¤ Results in a new risk–adjusted cost probability distribution. ¤ Wider and shifted to the right Baseline vs. Risk-Adjusted Estimates 0 50 100 150 200 250 300 350 FY$M Likelihood
- 21. Baseline versus Risk Adjusted Cost Estimates Almost Always Shows an Increase In Cost 21 Baseline vs. Risk-Adjusted Estimates 0 50 100 150 200 250 300 350 FY$M Likelihood
- 22. S-Curve for Cost Modeling 22 Cumulative Distribution Function 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% $60 $80 $100 $120 $140 $160 $180 $200 FY00$M CumulativeProbability