SlideShare ist ein Scribd-Unternehmen logo
1 von 19
Downloaden Sie, um offline zu lesen
Joint Cost and Schedule Probabilistic
 Estimating and Budgeting Policy at
               NASA

           February 2009
What?



• NASA wants to establish realistic
  program and project budgets that reflect
  a reasonable chance of achieving
  technical, cost and schedule objectives
  and commitments




                                             2
Why?


• We need to meet the expectations of
  stakeholders
• Getting it wrong
  – Damages our reputation as good stewards of
    public resources
  – Inhibits our ability to obtain funds for future
    programs and projects
  – Has adverse impacts of other programs and
    projects in our portfolio



                                                      3
How?



• Establish budgets based on the
  probability that projects have a better
  than 50/50 change of meeting cost and
  schedule targets




                                            4
Why Are Cost Estimates Uncertain?


– Beyond-State-of-the-Art      – Tight Schedules
  Technology                       • Undeveloped Technology
    •   Cooling                    • Software Development
    •   Processing                 • Supplier Viability
    •   Survivability          – System Integration
    •   Power                      • Multi-Contractor Teams
    •   Laser Communications       • System Testing
– Partners failing to meet     – Limited Resources
  commitments                  – Program Funding Stretch-out
– Launch conflicts             – Premature Commitment to
                                 R&D Phase
                               – Unforeseen Events


            These are just a few examples
                                                               5
What a Cost Estimate Looks Like
  Percentile     Value                                                  Program Alpha
   10%           516.81        10,000 Trials                            Cumulative Chart
                                      1.000
   20%           538.98
                                                            “S-Curve”
   30%           557.85                     .750

   40%           575.48
                                            .500
   50%           592.72
   60%           609.70                     .250


   70%           629.19                     .000

   80%           650.97                            462.43      537.16          611.89
                                                                             BY04$M
                                                                                             686.62    761.35

   90%           683.01
                                                                         Program Alpha

                               10,000 Trials                            Frequency Chart
Statistics           Value                  .020
Trials               10,000                                                                “Density Curve”
Mean                 596.40                 .015
                                  Density




Median               592.72                 .010
Mode                 ---
Standard Deviation     63.18                .005
Range Minimum        450.19
Range Maximum        796.68                 .000
                                                   462.43      537.16          611.89        686.62    761.35
                                                                              BY04$M
                                                                                                                6
                                                                                                                 6
So, Where Should We Set the
                 Budget?
• There is no “best” answer
• The “answer” depends on the amount of budget
  available or requested, and the amount of risk the
  decision-maker is willing to take
• A risk-averse decision-maker would probably choose a
  budget reflecting a cost with a high probability of
  realization
   – To minimize the probability of a cost overrun
• A risk-tolerant decision-maker might budget at a lower
  number, channeling a program manager to greater risk
  management
• Budget decisions should consider the risk across the
  entire portfolio of programs


                                                           7
NASA’s Policy on Setting Budgets
            Based on Probability
• In March of 2006, NASA Administrator
   –   “Griffin determined that NASA’s standard practices will be to budget projects at a 70%
       confidence level based on the independent cost estimate. Any proposed deviations
       from this standard must be brought forward for consideration to the appropriate
       management council.”
   –   “… initiate a pattern of honest dealing between Program and Project Managers, HQ,
       the Congress, and the WH, and to avoid the pattern of finger-pointing for cost overruns
       and schedule slips that have plagued the industry in the past”.
• Since this policy was declared, nearly all the major projects
  that came before the Agency Program Management Council
  were budgeted less than 70 percent probability of success
  (confidence level). Many reasons exist, but a few follow
   – No clear agreement on the estimate that should be used
   – Difficulty reconciling project estimates (typically based on
     engineering build up) with the Independent Estimates that were
     done using parametric models
   – Estimates were insensitive to schedule uncertainty
   – Belief that projects should be challenged
• NASA changed the policy (and formally documented it) on                                        8
  January 16, 2009
Management Intent of New Policy

• Confidence level policy needs to be more explicit
  about meeting schedule commitments in addition to
  cost -- move to a joint cost and schedule probability of
  success
• Probability of success should be based on project’s
  estimate/plan
• Confidence policy needs to be more flexible
   – Set the confidence policy at the program level
   – Allow program to solve issues by changing content and use
     resources within the programs’ control
   – Use other resources, but with better accountability
• At KDPs
   – Projects must develop and defend good plans at KDPs
   – SRB to assess project plans vice developing and presenting
     independent PCEs                                             9
A Summary of the New Confidence
             Level Policy

• All space flight and information
  technology programs shall develop a
  joint cost and schedule probabilistic
  analysis and be baselined or
  rebaselined and budgeted such that
  there is a 70 percent probability of
  achieving the stated life cycle cost and
  launch schedule


                                             10
New Confidence Level Policy
               (Continued)

• Applicable decision authorities may approve
  a different joint confidence level
• Projects are to be baselined or rebaselined
  and budgeted at confidence level consistent
  with the program’s confidence level
• At a minimum, projects are to be funded at a
  level that is equivalent to a confidence level
  of 50 percent, or as approved by the
  applicable decision authority

                                                   11
New Confidence Level Policy
               (Concluded)


• Programs or project’s proposed cost and
  schedule baselines are to be assessed by an
  independent review team
• Mission Directorates or Mission Support
  Offices are to confirm that program and
  projects’ life cycle cost estimates and annual
  budget submissions are consistent



                                                   12
An Example of Joint Cost and Schedule Probability




                                     Front
                                           i
                                   the co er Curve re
                                          mbina          p
                                sched
                                      ule wi      tion o resents
                                             th 65%     f cost
                                                     joint c and
                                                            onfide
                                                                   nce




                                                                         13
                                                                         Page 13
Draft Implementation Plan

• OCE established four working groups to implement
  the new Acquisition NPD. Must include
   –   Flow-down to lower level documents
   –   Communication
   –   Training and consulting
   –   Done by October 1, 2009
• JCL Working Group has met twice
   – Includes representatives from ESMD, SMD, SOMD, CFO,
     OCE and PA&E.
   – Plan to add selected center and/or project resource
     managers
   – Developed draft implementation approach (follows)

                                                           14
JCL Proposed Approach

• All programs and projects must immediately begin to
  develop and maintain probabilistic joint cost and
  schedule estimates and confidence levels (JCLs)
   – Include projects within formation as part of Program JCL
     computation. Deemed a practical consideration to get the agency
     to the end-state
• Communicate implementation details by way of
  Strategic Planning Guidance (SPG) and education
  workshops
• Use phased approach
   – PA&E to help the six programs (and associated projects) develop
     the JCLs during FY 2009
   – PA&E to provide immediate education to key development centers:
     GSFC, JPL, JSC, MSFC, & KSC. Provide two half day sessions at
     HQ and LaRC
   – Provide consulting support and tools to remainder during FY 2010


                                                                        15
Proposed Approach (Continued)

•   Salient provisions of Strategic Planning Guidance
     – PA&E to help six programs to develop and submit JCLs
     – Remaining projects within Development Phases (B, C, D), must submit
       minimal documentation to demonstrate that projects are resourced
       adequately, in lieu of JCL documentation
     – Proposed directed projects about to enter Phase A or competed projects
       about to enter Phase B, must undergo a Basis of Estimate review with PA&E
       to demonstrate the proposed time and resources are “within the bounds of
       reasonableness” and not likely to have adverse impact to programs
•   At KDP B & C decision milestone events
     – MD’s parent program must demonstrate the program complies with JCL
       policy with addition of the new project
     – Project must develop and defend its implementation plan (a resource loaded
       schedule)
     – SRB to provide an assessment of program and project JCL
•   At KDP D decision milestone event:
     – Only programs and projects that are on PA&E’s “help” list must produce a
       JCL
     – SRB’s develop JSL assessments for those programs and projects that have
       been helped. Otherwise, SRB performs a more general assessment of
       project’s plan
                                                                                    16
Proposed Approach (Concluded)
• Programs and projects JCL updates
   – Must be done at KDPs B, C & D
   – Should be done whenever there is a change in Operating
     Plan (change in requirements, cost, phasing of resources, or
     schedule)
   – Must be done in support of annual budget submit
• Programs and project report JCL
   – Whenever PMC directs (informed by BRP)
   – As part of annual budget submit




                                                                    17
Scenari             Project A           FY1         FY2
   os               50%                FY3     . . . FYn                            LC time is
for JCL             Project A                                                     defined by LCC
                                                                                 for each project
 Extern
   al               UFE
                    ∑ Project
 OMB                A
                                                                                   Mgt Options to Meet
      $ from OMB    Project B                                                      External Commitments
                    50%                                                         1.Use UFE from other
      $ to OMB                                                                    projects
       (cuts)
                    Project B                                                   2.Make minor de-scopes
                              a                        b                        3.∆ phasing
                     ⋮
                    UFE
                              c                      f                            requirements between

Intern              ∑ Project a               .        b′              }          years
                                                                                  Mgt Options to Solve
                                                                                   Problems, but Change
  al                                                                               External Commitments
                    B         c′                    f′            N

                                                                 Σ₅₀Pn
                                                                                1. Reduce project
Progra
   m
                    (Fundin
                              N

                                Σ₅₀P FY₁
                                i =
                                              .
                                              .                  i =              content
                      g)
                    Project N    1
                                                                  1             2. Slip schedule
   N                             N            .
                                              .                   N             3. Reduce ops $ or
        excess
      Program N
                    Internal
                                ΣμP FY₁
                                i =
                                                                 ΣμP FYn
                                                                 i =
                                                                                  duration
                                                                                4. Add Funds from
         UFE
                                                                                  projects in
 excess Program N
                    (Budget
                                 1
                                        .      .      .           1
                                                                                  formulation, New

                                Σ FY₁                            Σ FYn
       UFE
                                                                                  Starts, other
                    ∑UFE
                       )


                    Formulation
                                               .
                                              $$                           $$
                                                                                Notes:
                                                                                  programs, or Congress
                                                                                1.Programs must
                                                                                  maintain JCL
                    External
                                 Σ $$
                                                                                2.Projects in
                    $$                                                            Formulation are
                                                                                  included in JCL
                    New Start                  $$                          $$     calculation
 Outside            Extended Ops Σ $$
                    $$             $$                       $$
    of
 Program            $$                Σ $$
JCL Methodology and Tools

• Knowledge and tools to develop JCL are available
  now
   – Parametric approach. Estimate cost and schedule
     separately, then convolve to form JCL
   – Bottom-up approach via resource-loaded schedule
     (preferred method because it best reflects the PM’s plan)
• Tools include
   – Parametric
       • MS Project and Excel with Monte-Carlo simulation add ins
         (e.g., @ Risk and Crystal Ball)
       • PA&E-supplied Excel template
   – Bottom-up (resource-loaded schedule)
       • MS Project with add ins
       • PertMaster
   – PA&E-developed Excel template to develop Program JCL

                                                                    19

Weitere ähnliche Inhalte

Was ist angesagt?

Ed mangopanelpm challengefinal
Ed mangopanelpm challengefinalEd mangopanelpm challengefinal
Ed mangopanelpm challengefinalNASAPMC
 
Thomas.coonce
Thomas.coonceThomas.coonce
Thomas.coonceNASAPMC
 
Borchardt.heidemarie
Borchardt.heidemarieBorchardt.heidemarie
Borchardt.heidemarieNASAPMC
 
Osterkamp jeff
Osterkamp jeffOsterkamp jeff
Osterkamp jeffNASAPMC
 
Randall.taylor
Randall.taylorRandall.taylor
Randall.taylorNASAPMC
 
Charles.leising
Charles.leisingCharles.leising
Charles.leisingNASAPMC
 
Woods.edwards.pm challenge bpr presentation 2012 v1
Woods.edwards.pm challenge bpr presentation 2012 v1Woods.edwards.pm challenge bpr presentation 2012 v1
Woods.edwards.pm challenge bpr presentation 2012 v1NASAPMC
 
Noneman.steven
Noneman.stevenNoneman.steven
Noneman.stevenNASAPMC
 
Albert sierra
Albert sierraAlbert sierra
Albert sierraNASAPMC
 
Esker.linda
Esker.lindaEsker.linda
Esker.lindaNASAPMC
 
Smalley sandra
Smalley sandraSmalley sandra
Smalley sandraNASAPMC
 
John.baniszewski
John.baniszewskiJohn.baniszewski
John.baniszewskiNASAPMC
 
Harrison.g.poole.k
Harrison.g.poole.kHarrison.g.poole.k
Harrison.g.poole.kNASAPMC
 
Harvey elliott
Harvey elliottHarvey elliott
Harvey elliottNASAPMC
 
Thomas.a.greathouse.r
Thomas.a.greathouse.rThomas.a.greathouse.r
Thomas.a.greathouse.rNASAPMC
 
Sally.godfrey
Sally.godfrey Sally.godfrey
Sally.godfrey NASAPMC
 
Backup darren elliott
Backup darren elliottBackup darren elliott
Backup darren elliottNASAPMC
 
Blythe ortiz7120 5e handbooks 2 15-2012 final
Blythe ortiz7120 5e handbooks 2 15-2012 finalBlythe ortiz7120 5e handbooks 2 15-2012 final
Blythe ortiz7120 5e handbooks 2 15-2012 finalNASAPMC
 

Was ist angesagt? (20)

Ed mangopanelpm challengefinal
Ed mangopanelpm challengefinalEd mangopanelpm challengefinal
Ed mangopanelpm challengefinal
 
Thomas.coonce
Thomas.coonceThomas.coonce
Thomas.coonce
 
Borchardt.heidemarie
Borchardt.heidemarieBorchardt.heidemarie
Borchardt.heidemarie
 
Osterkamp jeff
Osterkamp jeffOsterkamp jeff
Osterkamp jeff
 
Randall.taylor
Randall.taylorRandall.taylor
Randall.taylor
 
Charles.leising
Charles.leisingCharles.leising
Charles.leising
 
Woods.edwards.pm challenge bpr presentation 2012 v1
Woods.edwards.pm challenge bpr presentation 2012 v1Woods.edwards.pm challenge bpr presentation 2012 v1
Woods.edwards.pm challenge bpr presentation 2012 v1
 
Noneman.steven
Noneman.stevenNoneman.steven
Noneman.steven
 
Albert sierra
Albert sierraAlbert sierra
Albert sierra
 
Esker.linda
Esker.lindaEsker.linda
Esker.linda
 
Smalley sandra
Smalley sandraSmalley sandra
Smalley sandra
 
John.baniszewski
John.baniszewskiJohn.baniszewski
John.baniszewski
 
Harrison.g.poole.k
Harrison.g.poole.kHarrison.g.poole.k
Harrison.g.poole.k
 
Harvey elliott
Harvey elliottHarvey elliott
Harvey elliott
 
Thomas.a.greathouse.r
Thomas.a.greathouse.rThomas.a.greathouse.r
Thomas.a.greathouse.r
 
Sally.godfrey
Sally.godfrey Sally.godfrey
Sally.godfrey
 
Symons
SymonsSymons
Symons
 
Backup darren elliott
Backup darren elliottBackup darren elliott
Backup darren elliott
 
D1186
D1186D1186
D1186
 
Blythe ortiz7120 5e handbooks 2 15-2012 final
Blythe ortiz7120 5e handbooks 2 15-2012 finalBlythe ortiz7120 5e handbooks 2 15-2012 final
Blythe ortiz7120 5e handbooks 2 15-2012 final
 

Andere mochten auch

Hulett david
Hulett davidHulett david
Hulett davidNASAPMC
 
Hazen michael
Hazen michaelHazen michael
Hazen michaelNASAPMC
 
Backup ronniegoodinchampionsinventors
Backup ronniegoodinchampionsinventorsBackup ronniegoodinchampionsinventors
Backup ronniegoodinchampionsinventorsNASAPMC
 
Canga.m.wood.j
Canga.m.wood.jCanga.m.wood.j
Canga.m.wood.jNASAPMC
 
Terry.cooke davies
Terry.cooke daviesTerry.cooke davies
Terry.cooke daviesNASAPMC
 
Ann over
Ann overAnn over
Ann overNASAPMC
 
Baldwin.kristen
Baldwin.kristenBaldwin.kristen
Baldwin.kristenNASAPMC
 
Richard.grammier
Richard.grammierRichard.grammier
Richard.grammierNASAPMC
 
David.fuller
David.fullerDavid.fuller
David.fullerNASAPMC
 
Bryan.oconnor
Bryan.oconnorBryan.oconnor
Bryan.oconnorNASAPMC
 
Ralph.basilio
Ralph.basilioRalph.basilio
Ralph.basilioNASAPMC
 
Millis.marc
Millis.marcMillis.marc
Millis.marcNASAPMC
 
Jeff.robinson
Jeff.robinsonJeff.robinson
Jeff.robinsonNASAPMC
 
Fujieh.maura
Fujieh.mauraFujieh.maura
Fujieh.mauraNASAPMC
 
Ralph.basilio
Ralph.basilioRalph.basilio
Ralph.basilioNASAPMC
 
Kline.robinson
Kline.robinsonKline.robinson
Kline.robinsonNASAPMC
 
White.p.johnson.k
White.p.johnson.kWhite.p.johnson.k
White.p.johnson.kNASAPMC
 
Adrian.mark
Adrian.markAdrian.mark
Adrian.markNASAPMC
 
Lockwood.scott
Lockwood.scottLockwood.scott
Lockwood.scottNASAPMC
 

Andere mochten auch (20)

Hulett david
Hulett davidHulett david
Hulett david
 
Hazen michael
Hazen michaelHazen michael
Hazen michael
 
Backup ronniegoodinchampionsinventors
Backup ronniegoodinchampionsinventorsBackup ronniegoodinchampionsinventors
Backup ronniegoodinchampionsinventors
 
Canga.m.wood.j
Canga.m.wood.jCanga.m.wood.j
Canga.m.wood.j
 
Terry.cooke davies
Terry.cooke daviesTerry.cooke davies
Terry.cooke davies
 
Ann over
Ann overAnn over
Ann over
 
Baldwin.kristen
Baldwin.kristenBaldwin.kristen
Baldwin.kristen
 
Richard.grammier
Richard.grammierRichard.grammier
Richard.grammier
 
David.fuller
David.fullerDavid.fuller
David.fuller
 
Bryan.oconnor
Bryan.oconnorBryan.oconnor
Bryan.oconnor
 
Ralph.basilio
Ralph.basilioRalph.basilio
Ralph.basilio
 
Millis.marc
Millis.marcMillis.marc
Millis.marc
 
Jeff.robinson
Jeff.robinsonJeff.robinson
Jeff.robinson
 
Fujieh.maura
Fujieh.mauraFujieh.maura
Fujieh.maura
 
Ralph.basilio
Ralph.basilioRalph.basilio
Ralph.basilio
 
Kline.robinson
Kline.robinsonKline.robinson
Kline.robinson
 
White.p.johnson.k
White.p.johnson.kWhite.p.johnson.k
White.p.johnson.k
 
Huebner
HuebnerHuebner
Huebner
 
Adrian.mark
Adrian.markAdrian.mark
Adrian.mark
 
Lockwood.scott
Lockwood.scottLockwood.scott
Lockwood.scott
 

Ähnlich wie Coonce.tom

Book.steve
Book.steveBook.steve
Book.steveNASAPMC
 
Thomas.coonce
Thomas.coonceThomas.coonce
Thomas.coonceNASAPMC
 
Risk based cost estimating for water infrastructure projects
Risk based cost estimating for water infrastructure projectsRisk based cost estimating for water infrastructure projects
Risk based cost estimating for water infrastructure projectsAdvisian
 
CQF Risk Brochure
CQF Risk BrochureCQF Risk Brochure
CQF Risk Brochurecberr
 
The regulatory response jeff balchin
The regulatory response   jeff balchinThe regulatory response   jeff balchin
The regulatory response jeff balchinScott Donald
 
Iii. principles of_capital_budgeting
Iii. principles of_capital_budgetingIii. principles of_capital_budgeting
Iii. principles of_capital_budgetingEzgi Kurt
 
Wwf Andrew Lockett Tim Dyke & Scott Logie Iof Analysts Group Segmentation V...
Wwf Andrew Lockett Tim Dyke & Scott Logie Iof Analysts Group   Segmentation V...Wwf Andrew Lockett Tim Dyke & Scott Logie Iof Analysts Group   Segmentation V...
Wwf Andrew Lockett Tim Dyke & Scott Logie Iof Analysts Group Segmentation V...Insight_SIG
 
Discounted Cash Flow Methodology for Banks and Credit Unions
Discounted Cash Flow Methodology for Banks and Credit UnionsDiscounted Cash Flow Methodology for Banks and Credit Unions
Discounted Cash Flow Methodology for Banks and Credit UnionsLibby Bierman
 
Project Controls Expo 18th Nov 2014 - "Cost Estimate Risk Analysis: For Capit...
Project Controls Expo 18th Nov 2014 - "Cost Estimate Risk Analysis: For Capit...Project Controls Expo 18th Nov 2014 - "Cost Estimate Risk Analysis: For Capit...
Project Controls Expo 18th Nov 2014 - "Cost Estimate Risk Analysis: For Capit...Project Controls Expo
 
Optimising Portfolios and Programmes in an ever changing world by "Matthew Wa...
Optimising Portfolios and Programmes in an ever changing world by "Matthew Wa...Optimising Portfolios and Programmes in an ever changing world by "Matthew Wa...
Optimising Portfolios and Programmes in an ever changing world by "Matthew Wa...Project Controls Expo
 
Jeff balchin (pwc) presentation
Jeff balchin (pwc) presentationJeff balchin (pwc) presentation
Jeff balchin (pwc) presentationScott Donald
 
CECL Key Concepts
CECL Key ConceptsCECL Key Concepts
CECL Key ConceptsBaker Hill
 
SKassegne - Effective Capex Management
SKassegne - Effective Capex ManagementSKassegne - Effective Capex Management
SKassegne - Effective Capex ManagementSultana Kassegne
 
An approach for costing, pricing, and managing ffp
An approach for costing, pricing, and managing ffpAn approach for costing, pricing, and managing ffp
An approach for costing, pricing, and managing ffpEric McCamey
 
Session 8 cost planning and definition (1)
Session 8  cost planning and definition (1)Session 8  cost planning and definition (1)
Session 8 cost planning and definition (1)Marwan AL Hashdei
 
Coonce.tom
Coonce.tomCoonce.tom
Coonce.tomNASAPMC
 
Noms Award User Group 2009
Noms   Award User Group 2009Noms   Award User Group 2009
Noms Award User Group 2009barnbyvilla
 
Maximizing Profits
Maximizing ProfitsMaximizing Profits
Maximizing Profitsbrunsgregory
 

Ähnlich wie Coonce.tom (20)

Book.steve
Book.steveBook.steve
Book.steve
 
Thomas.coonce
Thomas.coonceThomas.coonce
Thomas.coonce
 
Risk based cost estimating for water infrastructure projects
Risk based cost estimating for water infrastructure projectsRisk based cost estimating for water infrastructure projects
Risk based cost estimating for water infrastructure projects
 
CQF Risk Brochure
CQF Risk BrochureCQF Risk Brochure
CQF Risk Brochure
 
The regulatory response jeff balchin
The regulatory response   jeff balchinThe regulatory response   jeff balchin
The regulatory response jeff balchin
 
Present to Inspire - J
Present to Inspire - JPresent to Inspire - J
Present to Inspire - J
 
Iii. principles of_capital_budgeting
Iii. principles of_capital_budgetingIii. principles of_capital_budgeting
Iii. principles of_capital_budgeting
 
Wwf Andrew Lockett Tim Dyke & Scott Logie Iof Analysts Group Segmentation V...
Wwf Andrew Lockett Tim Dyke & Scott Logie Iof Analysts Group   Segmentation V...Wwf Andrew Lockett Tim Dyke & Scott Logie Iof Analysts Group   Segmentation V...
Wwf Andrew Lockett Tim Dyke & Scott Logie Iof Analysts Group Segmentation V...
 
Discounted Cash Flow Methodology for Banks and Credit Unions
Discounted Cash Flow Methodology for Banks and Credit UnionsDiscounted Cash Flow Methodology for Banks and Credit Unions
Discounted Cash Flow Methodology for Banks and Credit Unions
 
Project Controls Expo 18th Nov 2014 - "Cost Estimate Risk Analysis: For Capit...
Project Controls Expo 18th Nov 2014 - "Cost Estimate Risk Analysis: For Capit...Project Controls Expo 18th Nov 2014 - "Cost Estimate Risk Analysis: For Capit...
Project Controls Expo 18th Nov 2014 - "Cost Estimate Risk Analysis: For Capit...
 
Optimising Portfolios and Programmes in an ever changing world by "Matthew Wa...
Optimising Portfolios and Programmes in an ever changing world by "Matthew Wa...Optimising Portfolios and Programmes in an ever changing world by "Matthew Wa...
Optimising Portfolios and Programmes in an ever changing world by "Matthew Wa...
 
Jeff balchin (pwc) presentation
Jeff balchin (pwc) presentationJeff balchin (pwc) presentation
Jeff balchin (pwc) presentation
 
CECL Key Concepts
CECL Key ConceptsCECL Key Concepts
CECL Key Concepts
 
SKassegne - Effective Capex Management
SKassegne - Effective Capex ManagementSKassegne - Effective Capex Management
SKassegne - Effective Capex Management
 
An approach for costing, pricing, and managing ffp
An approach for costing, pricing, and managing ffpAn approach for costing, pricing, and managing ffp
An approach for costing, pricing, and managing ffp
 
Session 8 cost planning and definition (1)
Session 8  cost planning and definition (1)Session 8  cost planning and definition (1)
Session 8 cost planning and definition (1)
 
Coonce.tom
Coonce.tomCoonce.tom
Coonce.tom
 
Value chain analysis 4
Value chain analysis 4Value chain analysis 4
Value chain analysis 4
 
Noms Award User Group 2009
Noms   Award User Group 2009Noms   Award User Group 2009
Noms Award User Group 2009
 
Maximizing Profits
Maximizing ProfitsMaximizing Profits
Maximizing Profits
 

Mehr von NASAPMC

Bejmuk bo
Bejmuk boBejmuk bo
Bejmuk boNASAPMC
 
Baniszewski john
Baniszewski johnBaniszewski john
Baniszewski johnNASAPMC
 
Yew manson
Yew mansonYew manson
Yew mansonNASAPMC
 
Wood frank
Wood frankWood frank
Wood frankNASAPMC
 
Wood frank
Wood frankWood frank
Wood frankNASAPMC
 
Wessen randi (cd)
Wessen randi (cd)Wessen randi (cd)
Wessen randi (cd)NASAPMC
 
Vellinga joe
Vellinga joeVellinga joe
Vellinga joeNASAPMC
 
Trahan stuart
Trahan stuartTrahan stuart
Trahan stuartNASAPMC
 
Stock gahm
Stock gahmStock gahm
Stock gahmNASAPMC
 
Snow lee
Snow leeSnow lee
Snow leeNASAPMC
 
Seftas krage
Seftas krageSeftas krage
Seftas krageNASAPMC
 
Sampietro marco
Sampietro marcoSampietro marco
Sampietro marcoNASAPMC
 
Rudolphi mike
Rudolphi mikeRudolphi mike
Rudolphi mikeNASAPMC
 
Roberts karlene
Roberts karleneRoberts karlene
Roberts karleneNASAPMC
 
Rackley mike
Rackley mikeRackley mike
Rackley mikeNASAPMC
 
Paradis william
Paradis williamParadis william
Paradis williamNASAPMC
 
O'keefe william
O'keefe williamO'keefe william
O'keefe williamNASAPMC
 
Muller ralf
Muller ralfMuller ralf
Muller ralfNASAPMC
 
Mulenburg jerry
Mulenburg jerryMulenburg jerry
Mulenburg jerryNASAPMC
 
Mitskevich amanda
Mitskevich amandaMitskevich amanda
Mitskevich amandaNASAPMC
 

Mehr von NASAPMC (20)

Bejmuk bo
Bejmuk boBejmuk bo
Bejmuk bo
 
Baniszewski john
Baniszewski johnBaniszewski john
Baniszewski john
 
Yew manson
Yew mansonYew manson
Yew manson
 
Wood frank
Wood frankWood frank
Wood frank
 
Wood frank
Wood frankWood frank
Wood frank
 
Wessen randi (cd)
Wessen randi (cd)Wessen randi (cd)
Wessen randi (cd)
 
Vellinga joe
Vellinga joeVellinga joe
Vellinga joe
 
Trahan stuart
Trahan stuartTrahan stuart
Trahan stuart
 
Stock gahm
Stock gahmStock gahm
Stock gahm
 
Snow lee
Snow leeSnow lee
Snow lee
 
Seftas krage
Seftas krageSeftas krage
Seftas krage
 
Sampietro marco
Sampietro marcoSampietro marco
Sampietro marco
 
Rudolphi mike
Rudolphi mikeRudolphi mike
Rudolphi mike
 
Roberts karlene
Roberts karleneRoberts karlene
Roberts karlene
 
Rackley mike
Rackley mikeRackley mike
Rackley mike
 
Paradis william
Paradis williamParadis william
Paradis william
 
O'keefe william
O'keefe williamO'keefe william
O'keefe william
 
Muller ralf
Muller ralfMuller ralf
Muller ralf
 
Mulenburg jerry
Mulenburg jerryMulenburg jerry
Mulenburg jerry
 
Mitskevich amanda
Mitskevich amandaMitskevich amanda
Mitskevich amanda
 

Kürzlich hochgeladen

NIST Cybersecurity Framework (CSF) 2.0 Workshop
NIST Cybersecurity Framework (CSF) 2.0 WorkshopNIST Cybersecurity Framework (CSF) 2.0 Workshop
NIST Cybersecurity Framework (CSF) 2.0 WorkshopBachir Benyammi
 
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve Decarbonization
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve DecarbonizationUsing IESVE for Loads, Sizing and Heat Pump Modeling to Achieve Decarbonization
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve DecarbonizationIES VE
 
How Accurate are Carbon Emissions Projections?
How Accurate are Carbon Emissions Projections?How Accurate are Carbon Emissions Projections?
How Accurate are Carbon Emissions Projections?IES VE
 
Cybersecurity Workshop #1.pptx
Cybersecurity Workshop #1.pptxCybersecurity Workshop #1.pptx
Cybersecurity Workshop #1.pptxGDSC PJATK
 
The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...
The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...
The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...Aggregage
 
UiPath Studio Web workshop series - Day 8
UiPath Studio Web workshop series - Day 8UiPath Studio Web workshop series - Day 8
UiPath Studio Web workshop series - Day 8DianaGray10
 
UiPath Platform: The Backend Engine Powering Your Automation - Session 1
UiPath Platform: The Backend Engine Powering Your Automation - Session 1UiPath Platform: The Backend Engine Powering Your Automation - Session 1
UiPath Platform: The Backend Engine Powering Your Automation - Session 1DianaGray10
 
VoIP Service and Marketing using Odoo and Asterisk PBX
VoIP Service and Marketing using Odoo and Asterisk PBXVoIP Service and Marketing using Odoo and Asterisk PBX
VoIP Service and Marketing using Odoo and Asterisk PBXTarek Kalaji
 
Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...
Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...
Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...Will Schroeder
 
IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019
IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019
IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019IES VE
 
UiPath Community: AI for UiPath Automation Developers
UiPath Community: AI for UiPath Automation DevelopersUiPath Community: AI for UiPath Automation Developers
UiPath Community: AI for UiPath Automation DevelopersUiPathCommunity
 
Bird eye's view on Camunda open source ecosystem
Bird eye's view on Camunda open source ecosystemBird eye's view on Camunda open source ecosystem
Bird eye's view on Camunda open source ecosystemAsko Soukka
 
Building AI-Driven Apps Using Semantic Kernel.pptx
Building AI-Driven Apps Using Semantic Kernel.pptxBuilding AI-Driven Apps Using Semantic Kernel.pptx
Building AI-Driven Apps Using Semantic Kernel.pptxUdaiappa Ramachandran
 
AI Fame Rush Review – Virtual Influencer Creation In Just Minutes
AI Fame Rush Review – Virtual Influencer Creation In Just MinutesAI Fame Rush Review – Virtual Influencer Creation In Just Minutes
AI Fame Rush Review – Virtual Influencer Creation In Just MinutesMd Hossain Ali
 
AI You Can Trust - Ensuring Success with Data Integrity Webinar
AI You Can Trust - Ensuring Success with Data Integrity WebinarAI You Can Trust - Ensuring Success with Data Integrity Webinar
AI You Can Trust - Ensuring Success with Data Integrity WebinarPrecisely
 
UiPath Studio Web workshop series - Day 6
UiPath Studio Web workshop series - Day 6UiPath Studio Web workshop series - Day 6
UiPath Studio Web workshop series - Day 6DianaGray10
 
Basic Building Blocks of Internet of Things.
Basic Building Blocks of Internet of Things.Basic Building Blocks of Internet of Things.
Basic Building Blocks of Internet of Things.YounusS2
 
COMPUTER 10: Lesson 7 - File Storage and Online Collaboration
COMPUTER 10: Lesson 7 - File Storage and Online CollaborationCOMPUTER 10: Lesson 7 - File Storage and Online Collaboration
COMPUTER 10: Lesson 7 - File Storage and Online Collaborationbruanjhuli
 

Kürzlich hochgeladen (20)

NIST Cybersecurity Framework (CSF) 2.0 Workshop
NIST Cybersecurity Framework (CSF) 2.0 WorkshopNIST Cybersecurity Framework (CSF) 2.0 Workshop
NIST Cybersecurity Framework (CSF) 2.0 Workshop
 
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve Decarbonization
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve DecarbonizationUsing IESVE for Loads, Sizing and Heat Pump Modeling to Achieve Decarbonization
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve Decarbonization
 
How Accurate are Carbon Emissions Projections?
How Accurate are Carbon Emissions Projections?How Accurate are Carbon Emissions Projections?
How Accurate are Carbon Emissions Projections?
 
Cybersecurity Workshop #1.pptx
Cybersecurity Workshop #1.pptxCybersecurity Workshop #1.pptx
Cybersecurity Workshop #1.pptx
 
The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...
The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...
The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...
 
UiPath Studio Web workshop series - Day 8
UiPath Studio Web workshop series - Day 8UiPath Studio Web workshop series - Day 8
UiPath Studio Web workshop series - Day 8
 
UiPath Platform: The Backend Engine Powering Your Automation - Session 1
UiPath Platform: The Backend Engine Powering Your Automation - Session 1UiPath Platform: The Backend Engine Powering Your Automation - Session 1
UiPath Platform: The Backend Engine Powering Your Automation - Session 1
 
VoIP Service and Marketing using Odoo and Asterisk PBX
VoIP Service and Marketing using Odoo and Asterisk PBXVoIP Service and Marketing using Odoo and Asterisk PBX
VoIP Service and Marketing using Odoo and Asterisk PBX
 
Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...
Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...
Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...
 
IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019
IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019
IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019
 
UiPath Community: AI for UiPath Automation Developers
UiPath Community: AI for UiPath Automation DevelopersUiPath Community: AI for UiPath Automation Developers
UiPath Community: AI for UiPath Automation Developers
 
Bird eye's view on Camunda open source ecosystem
Bird eye's view on Camunda open source ecosystemBird eye's view on Camunda open source ecosystem
Bird eye's view on Camunda open source ecosystem
 
Building AI-Driven Apps Using Semantic Kernel.pptx
Building AI-Driven Apps Using Semantic Kernel.pptxBuilding AI-Driven Apps Using Semantic Kernel.pptx
Building AI-Driven Apps Using Semantic Kernel.pptx
 
AI Fame Rush Review – Virtual Influencer Creation In Just Minutes
AI Fame Rush Review – Virtual Influencer Creation In Just MinutesAI Fame Rush Review – Virtual Influencer Creation In Just Minutes
AI Fame Rush Review – Virtual Influencer Creation In Just Minutes
 
201610817 - edge part1
201610817 - edge part1201610817 - edge part1
201610817 - edge part1
 
AI You Can Trust - Ensuring Success with Data Integrity Webinar
AI You Can Trust - Ensuring Success with Data Integrity WebinarAI You Can Trust - Ensuring Success with Data Integrity Webinar
AI You Can Trust - Ensuring Success with Data Integrity Webinar
 
UiPath Studio Web workshop series - Day 6
UiPath Studio Web workshop series - Day 6UiPath Studio Web workshop series - Day 6
UiPath Studio Web workshop series - Day 6
 
Basic Building Blocks of Internet of Things.
Basic Building Blocks of Internet of Things.Basic Building Blocks of Internet of Things.
Basic Building Blocks of Internet of Things.
 
COMPUTER 10: Lesson 7 - File Storage and Online Collaboration
COMPUTER 10: Lesson 7 - File Storage and Online CollaborationCOMPUTER 10: Lesson 7 - File Storage and Online Collaboration
COMPUTER 10: Lesson 7 - File Storage and Online Collaboration
 
20230104 - machine vision
20230104 - machine vision20230104 - machine vision
20230104 - machine vision
 

Coonce.tom

  • 1. Joint Cost and Schedule Probabilistic Estimating and Budgeting Policy at NASA February 2009
  • 2. What? • NASA wants to establish realistic program and project budgets that reflect a reasonable chance of achieving technical, cost and schedule objectives and commitments 2
  • 3. Why? • We need to meet the expectations of stakeholders • Getting it wrong – Damages our reputation as good stewards of public resources – Inhibits our ability to obtain funds for future programs and projects – Has adverse impacts of other programs and projects in our portfolio 3
  • 4. How? • Establish budgets based on the probability that projects have a better than 50/50 change of meeting cost and schedule targets 4
  • 5. Why Are Cost Estimates Uncertain? – Beyond-State-of-the-Art – Tight Schedules Technology • Undeveloped Technology • Cooling • Software Development • Processing • Supplier Viability • Survivability – System Integration • Power • Multi-Contractor Teams • Laser Communications • System Testing – Partners failing to meet – Limited Resources commitments – Program Funding Stretch-out – Launch conflicts – Premature Commitment to R&D Phase – Unforeseen Events These are just a few examples 5
  • 6. What a Cost Estimate Looks Like Percentile Value Program Alpha 10% 516.81 10,000 Trials Cumulative Chart 1.000 20% 538.98 “S-Curve” 30% 557.85 .750 40% 575.48 .500 50% 592.72 60% 609.70 .250 70% 629.19 .000 80% 650.97 462.43 537.16 611.89 BY04$M 686.62 761.35 90% 683.01 Program Alpha 10,000 Trials Frequency Chart Statistics Value .020 Trials 10,000 “Density Curve” Mean 596.40 .015 Density Median 592.72 .010 Mode --- Standard Deviation 63.18 .005 Range Minimum 450.19 Range Maximum 796.68 .000 462.43 537.16 611.89 686.62 761.35 BY04$M 6 6
  • 7. So, Where Should We Set the Budget? • There is no “best” answer • The “answer” depends on the amount of budget available or requested, and the amount of risk the decision-maker is willing to take • A risk-averse decision-maker would probably choose a budget reflecting a cost with a high probability of realization – To minimize the probability of a cost overrun • A risk-tolerant decision-maker might budget at a lower number, channeling a program manager to greater risk management • Budget decisions should consider the risk across the entire portfolio of programs 7
  • 8. NASA’s Policy on Setting Budgets Based on Probability • In March of 2006, NASA Administrator – “Griffin determined that NASA’s standard practices will be to budget projects at a 70% confidence level based on the independent cost estimate. Any proposed deviations from this standard must be brought forward for consideration to the appropriate management council.” – “… initiate a pattern of honest dealing between Program and Project Managers, HQ, the Congress, and the WH, and to avoid the pattern of finger-pointing for cost overruns and schedule slips that have plagued the industry in the past”. • Since this policy was declared, nearly all the major projects that came before the Agency Program Management Council were budgeted less than 70 percent probability of success (confidence level). Many reasons exist, but a few follow – No clear agreement on the estimate that should be used – Difficulty reconciling project estimates (typically based on engineering build up) with the Independent Estimates that were done using parametric models – Estimates were insensitive to schedule uncertainty – Belief that projects should be challenged • NASA changed the policy (and formally documented it) on 8 January 16, 2009
  • 9. Management Intent of New Policy • Confidence level policy needs to be more explicit about meeting schedule commitments in addition to cost -- move to a joint cost and schedule probability of success • Probability of success should be based on project’s estimate/plan • Confidence policy needs to be more flexible – Set the confidence policy at the program level – Allow program to solve issues by changing content and use resources within the programs’ control – Use other resources, but with better accountability • At KDPs – Projects must develop and defend good plans at KDPs – SRB to assess project plans vice developing and presenting independent PCEs 9
  • 10. A Summary of the New Confidence Level Policy • All space flight and information technology programs shall develop a joint cost and schedule probabilistic analysis and be baselined or rebaselined and budgeted such that there is a 70 percent probability of achieving the stated life cycle cost and launch schedule 10
  • 11. New Confidence Level Policy (Continued) • Applicable decision authorities may approve a different joint confidence level • Projects are to be baselined or rebaselined and budgeted at confidence level consistent with the program’s confidence level • At a minimum, projects are to be funded at a level that is equivalent to a confidence level of 50 percent, or as approved by the applicable decision authority 11
  • 12. New Confidence Level Policy (Concluded) • Programs or project’s proposed cost and schedule baselines are to be assessed by an independent review team • Mission Directorates or Mission Support Offices are to confirm that program and projects’ life cycle cost estimates and annual budget submissions are consistent 12
  • 13. An Example of Joint Cost and Schedule Probability Front i the co er Curve re mbina p sched ule wi tion o resents th 65% f cost joint c and onfide nce 13 Page 13
  • 14. Draft Implementation Plan • OCE established four working groups to implement the new Acquisition NPD. Must include – Flow-down to lower level documents – Communication – Training and consulting – Done by October 1, 2009 • JCL Working Group has met twice – Includes representatives from ESMD, SMD, SOMD, CFO, OCE and PA&E. – Plan to add selected center and/or project resource managers – Developed draft implementation approach (follows) 14
  • 15. JCL Proposed Approach • All programs and projects must immediately begin to develop and maintain probabilistic joint cost and schedule estimates and confidence levels (JCLs) – Include projects within formation as part of Program JCL computation. Deemed a practical consideration to get the agency to the end-state • Communicate implementation details by way of Strategic Planning Guidance (SPG) and education workshops • Use phased approach – PA&E to help the six programs (and associated projects) develop the JCLs during FY 2009 – PA&E to provide immediate education to key development centers: GSFC, JPL, JSC, MSFC, & KSC. Provide two half day sessions at HQ and LaRC – Provide consulting support and tools to remainder during FY 2010 15
  • 16. Proposed Approach (Continued) • Salient provisions of Strategic Planning Guidance – PA&E to help six programs to develop and submit JCLs – Remaining projects within Development Phases (B, C, D), must submit minimal documentation to demonstrate that projects are resourced adequately, in lieu of JCL documentation – Proposed directed projects about to enter Phase A or competed projects about to enter Phase B, must undergo a Basis of Estimate review with PA&E to demonstrate the proposed time and resources are “within the bounds of reasonableness” and not likely to have adverse impact to programs • At KDP B & C decision milestone events – MD’s parent program must demonstrate the program complies with JCL policy with addition of the new project – Project must develop and defend its implementation plan (a resource loaded schedule) – SRB to provide an assessment of program and project JCL • At KDP D decision milestone event: – Only programs and projects that are on PA&E’s “help” list must produce a JCL – SRB’s develop JSL assessments for those programs and projects that have been helped. Otherwise, SRB performs a more general assessment of project’s plan 16
  • 17. Proposed Approach (Concluded) • Programs and projects JCL updates – Must be done at KDPs B, C & D – Should be done whenever there is a change in Operating Plan (change in requirements, cost, phasing of resources, or schedule) – Must be done in support of annual budget submit • Programs and project report JCL – Whenever PMC directs (informed by BRP) – As part of annual budget submit 17
  • 18. Scenari Project A FY1 FY2 os 50% FY3 . . . FYn LC time is for JCL Project A defined by LCC for each project Extern al UFE ∑ Project OMB A Mgt Options to Meet $ from OMB Project B External Commitments 50% 1.Use UFE from other $ to OMB projects (cuts) Project B 2.Make minor de-scopes a b 3.∆ phasing ⋮ UFE c f requirements between Intern ∑ Project a . b′ } years Mgt Options to Solve Problems, but Change al External Commitments B c′ f′ N Σ₅₀Pn 1. Reduce project Progra m (Fundin N Σ₅₀P FY₁ i = . . i = content g) Project N 1 1 2. Slip schedule N N . . N 3. Reduce ops $ or excess Program N Internal ΣμP FY₁ i = ΣμP FYn i = duration 4. Add Funds from UFE projects in excess Program N (Budget 1 . . . 1 formulation, New Σ FY₁ Σ FYn UFE Starts, other ∑UFE ) Formulation . $$ $$ Notes: programs, or Congress 1.Programs must maintain JCL External Σ $$ 2.Projects in $$ Formulation are included in JCL New Start $$ $$ calculation Outside Extended Ops Σ $$ $$ $$ $$ of Program $$ Σ $$
  • 19. JCL Methodology and Tools • Knowledge and tools to develop JCL are available now – Parametric approach. Estimate cost and schedule separately, then convolve to form JCL – Bottom-up approach via resource-loaded schedule (preferred method because it best reflects the PM’s plan) • Tools include – Parametric • MS Project and Excel with Monte-Carlo simulation add ins (e.g., @ Risk and Crystal Ball) • PA&E-supplied Excel template – Bottom-up (resource-loaded schedule) • MS Project with add ins • PertMaster – PA&E-developed Excel template to develop Program JCL 19