2. IBM Rational Software Conference 2009
The nature of innovation
Innovation is not the predictable realization of a new idea
Innovation is a risky plunge into the unknown, but with a clear objective and a
strong sense of general direction, a key idea
Executive Summit 2
3. IBM Rational Software Conference 2009
What I read on my vacation
Magellan knew exactly what he wished to achieve and
his general plan for getting there
Find a passage through South America
Claim the Moluccas (Slice Islands) for Spain
There were many unknowns and risks along the way
There was no detailed plan – he steered as he went
How would Magellan have
managed a software project?
Executive Summit 3
4. IBM Rational Software Conference 2009
The project model
Standard model Modern model
Distinct development phase Continuously evolving systems
No distinct boundary between development
Distinct handoff to maintenance
and maintenance
Requirements-design-code-test Sequence of released capabilities
sequence with ever increasing value
Phase and role specific tools Common platform of integrated process / tooling
Collocated teams Distributed, web based collaboration
Economic governance tailored
Standard engineering governance
to risk / reward profiles
Engineering practitioner led Business value and outcome led
Executive Summit 4
5. IBM Rational Software Conference 2009
Critical culture shifts
Conventional Governance Agile Governance
Activity-based management Results-based management
Mature processes, PMI/PMBOK More art than engineering
Plan in detail, then track variances Plan/steer/plan/steer…
Adversarial relationships Honest collaborative communication
Paper exchange, speculation Progressions/digressions, facts
Requirements first Architecture (risk mitigation) first
Assumes certainty in desired product Admits uncertainties
Avoid change Manage change
Early false precision Evolving artifacts
“More detail = higher quality” Scope (Problem specs)
Design (Solution specs)
Constraints (Planning specs)
Apply too much or too little process Right-size the process
Process is primary, blind adherence Desired results drive process
Manage variances
Executive Summit 5
6. IBM Rational Software Conference 2009
Schedule risk: Imagine you have 12 months to deliver
a business critical system
Your estimators tell you it will be done in 11 months
What do you do with the information?
Rest easy, believing there is no risk?
0 6 12
Executive Summit 6
7. IBM Rational Software Conference 2009
Maybe you realize that program parameters
(cost, schedule, effort, quality, …) are random variables
Area under curve describes probability of measurement falling in range
0.45
0.4
0.35 Likelihood of actual value
0.3
falling in range is area
under curve
0.25
0.2
0.15
0.1
0.05
0
00
60
40
20
00
80
20
00
80
20
00
40
80
60
40
60
0.
1.
2.
5.
6.
0.
1.
2.
2.
3.
4.
4.
4.
5.
0.
3.
Executive Summit 7
8. IBM Rational Software Conference 2009
Imagine you have 12 months to deliver a business
critical systems
So you ask for the distribution and discover there is some uncertainty
0 6 12
Executive Summit 8
9. IBM Rational Software Conference 2009
Imagine you have 12 months to deliver a business
critical systems
In fact there is less than 50% chance of making the date
48%
0 6 12
Executive Summit 9
10. IBM Rational Software Conference 2009
Then what?
Move out the date to improve likelihood of shipping?
95%
0 6 12 15
Executive Summit 10
11. IBM Rational Software Conference 2009
Then what?
Or move in the estimate by sacrificing quality or content?
95%
0 6 12
Executive Summit 11
12. IBM Rational Software Conference 2009
Managing variances in scope, solution, plans:
The real key to improving software economics
Sources of uncertainty 0.45
0.4
and variance
0.35
Lack of knowledge 0.3
Lack of confidence 0.25
0.2
Lack of agreement
0.15
Reduction of 0.1
variance reflects 0.05
Increased predictability 0
00
40
00
60
80
20
60
80
20
60
40
80
40
00
20
of outcome
0.
0.
2.
5.
0.
1.
1.
2.
3.
3.
4.
4.
2.
4.
5.
Increased knowledge about
Client needs
Technology capability
Team capability
Good decisions
Executive Summit 12
13. IBM Rational Software Conference 2009
Then what?
Determine the source of the variance
Over the project lifecycle, reduce the variance to improve likelihood of shipping
90%
0 6 12
Executive Summit 13
14. IBM Rational Software Conference 2009
Then what?
Over the lifecycle, reduce the variance further to improve likelihood of shipping
95%
0 6 12
Executive Summit 14
15. IBM Rational Software Conference 2009
Measure and steer
At onset of program
Report: Establish estimates/variances of effort, cost, establish initial plan
Collaborate: Set initial scope and expectations with stakeholders
Automate: Establish a collaborative development environment
At each iteration, improve estimates and report
Report: Values and variances of progress achieved, quality achieved, resources expended
Collaborate: With stakeholders to refine scope and plans
Automate: Manage changes to plans, baselines, test-beds
Initial Plan
Initial State
Initial Planned Path Uncertainty
in stakeholder
satisfaction space
Variance in
Actual Path estimate to
complete
Executive Summit 15
16. IBM Rational Software Conference 2009
Agile Governance = Managing Uncertainty = Managing Variance
A completion date is not a point in time, it is a probability distribution
0 6 12
Scope is not a requirements document, it is a continuous negotiation
Plans/Resource estimates
Scope
Product features/quality
A plan is not a prescription, it is an evolving,
moving target Uncertainty in
Stakeholder
Satisfaction Space
Initial State Initial Plan
Actual path and precision of Scope/Plan
Executive Summit 16
17. IBM Rational Software Conference 2009
Practices included as part of Rational Method
Composer
Executive Summit 17
18. IBM Rational Software Conference 2009
Agenda
Transitioning to Agile Software Delivery
Economic governance
Steering and managing uncertainty
Metrics and Measures
Instrumentation for gaining control, improving efficiency and optimizing value
Improving Software Economics
A framework for improvement priorities
Executive Summit 18
19. IBM Rational Software Conference 2009
Software and systems need a control framework
Performance Measurement
Value
Value Metrics
Business Objectives measures
e.g., ROI, ROA for SSD
Efficiency
Operational Effectiveness Metrics
feedback
Operational Objectives measures
e.g., Time to market, productivity
Control
Process Definition / Practices
measures Practice Practice
Rational Method Composer
Adoption/Maturity Artifacts
subjective Objective
Process Enactment / Governance Enforcement / Process Awareness
Jazz Platform
Executive Summit 19
20. IBM Rational Software Conference 2009
Meters for software and systems
development and delivery improvement
BRIDGE to MCIF
Value
Return on Investment (ROI)
Return on Assets (ROA) $0 Business $15
Product revenue profile Value
$M
Efficiency
Time to market, productivity
Program portfolio investment profile
Defect phase containment, scrap and rework rates 0% Operational 100%
Application service levels Efficiency
Defect densities, requirements churn, design churn
Skills improvement, training cost reduction
Control
Practice adoption, project checkpoints Process
0% 100%
Artifact time between gates Adoption
Collaboration, skills mix
Executive Summit 20
21. IBM Rational Software Conference 2009
Tailor to organizational and project context
Agree on business value measures: Cost, profit, return on assets, market share, etc.
Determine project mix type
Choose appropriate operational measures
Choose practices to achieve measures for project mix
Establish measures and feedback channels for closed loop control
Variance Examples
Low Medium High
Value Cost of operations Market share growth Profitability of one-of-a-
(Business Time to market for new features kind system
Measures)
Efficiency Cost per change request Cost per change request Architectural stability
(Operational Individual productivity Team Productivity Organizational productivity
Measure)
Controls Self check for practices Beta releases Stakeholder
demonstrations
Defect densities, removal rates
Practices Requirements management Agile planning Shared vision
Change management Test driven development Risk based lifecycle
Iterative development
Evolutionary Architecture
Executive Summit 21
22. IBM Rational Software Conference 2009
Select practices and measures based on business
and operational objectives
High # of defects (pre/post-ship) CEO Business Value High support or maintenance costs without
High maintenance costs (devt) of fixing defects high defect #s
Customer downtime Low pipeline conversion
IT Quality Goals Low customer satisfaction
CIO
Dev. Mgr. Development Quality Goals Operational Quality Goals
? ?
High defects both pre- Non-functional Req. Issues
and post-ship High # of Help Desk Calls
Reduce Post-Delivery Defects Deliver What Stakeholder Needs
Growing defect backlog High Incidence Resolution Times
H L M L
? ? ? ?
Improve Non-
Increase Defect Increase Defect Deliver on Customer
Functional Quality
Prevention Detection Requirements
M M H L M L Attributes M L
High post-ship or
customer-reported Measures: Measures:
Measures: Measures:
defects Post-ship problem reports Post-ship problem reports
Defect density Defect density, distribution
Customer satisfaction Customer satisfaction
High error rates in Defect arrival/closure Defect arrival/closure rates
Pipeline conversion? Support / maint. costs
fixes rates Defect removal effectiv. High Requirements Churn
Support / maint. costs Requirement test coverage
Defect backlog Fixes failing verification
Requirem. test coverage Test execution results High Post Delivery Support
Fixes failing verification Test coverage
Rework effort Test execution status Requirements delivery Aging ER Backlog
Survey of feature usage Practices:
Performance Testing
Practices: Practices:
Test-driven Dev. Requirements Mgnt.
Test management Practices:
Design-driven Implem. Shared Vision Shared Vision
Continuous integration
C&C management Use-case Driven Dev Risk-Value Lifecycle
Evolutionary Architecture
System Component Arch. Requirements Mgnt. Evolutionary Architecture
Component Architecture
Whole Team Test-Driven Development
Whole team Test-driven dev.
Pair Programming Iterative Dev. Iterative Development
Test practices Revenue / Cost
Functional Testing Evolutionary Design
Review/Inspection Iterative Dev.
C&C Management Component Architecture
Risk Value Lifecycle
Review/Inspection Continuous Integration
C&C Management Value (H,M or L)
Concurrent Testing
Review/Inspection Cost (H, M or L)
Whole Team
Review/Inspection Hot Component
Executive Summit 22
23. IBM Rational Software Conference 2009
Agenda
Transitioning to Agile Software Delivery
Economic governance
Steering and managing uncertainty
Metrics and Measures
Instrumentation for gaining control, improving efficiency and optimizing value
Improving Software Economics
A framework for improvement priorities
Executive Summit 23
24. IBM Rational Software Conference 2009
Four patterns of success in achieving Agility at Scale
1. Scope management Asset based development
Solutions evolve from requirements AND requirements evolve from available assets
As opposed to getting all the requirements right up front
2. Process management Rightsize the process
Process and instrumentation rigor evolves from light to heavy
As opposed to the entire project’s lifecycle process should be light
or heavy depending on the character of the project
3. Progress management Honest assessments
Healthy projects display a sequence of progressions and digressions
As opposed to progressing to 100% earned value with monotonically increasing
progress against a static plan
4. Quality management Incremental demonstrable results
Testing needs to be a 1st class, full lifecycle activity
As opposed to a subordinate, later lifecycle activity
Executive Summit 24
26. IBM Rational Software Conference 2009
Improving software economics
Empirical software cost estimation models for:
Enterprise modernization, software maintenance
New developments, new releases, early prototypes
Packaged applications, systems engineering
Time or Cost
To Build = (Complexity) (Process) * (Team) * (Tools)
Complexity Process Team Tools
Volume of human Methods Skills/Experience Automation
generated stuff Maturity Collaboration Process enactment
- KSLOC, FPs, UCs Agility Motivation
Quality/performance Precedence
Scope
Executive Summit 26
27. IBM Rational Software Conference 2009
Achieve continuous improvement by measuring cost
against business outcomes
Improve Improve Improve Increase Flexibility
Automation Collaboration Process & Investment Value
Implement tools Implement tools Implement Implement enterprise
to integrate & processes iterative / adaptive architecture & reuse
Business workflows & leverage skills processes web services
Value Productivity: Productivity: Productivity: Productivity:
5-25% 15-35% 25-100% 50-200+%
ECONOMIC IMPACTS
Timeframe = Days Timeframe = Weeks Timeframe = Months Timeframe = Years
Efficiency
Control
Implementation costs Individual Team Organization Business
are per person per year
Executive Summit 27
28. IBM Rational Software Conference 2009
Improve process to increase productivity by 25%-100%
Implement iterative / adaptive processes
Perceived Progress Against Plan
Demo Test
Releases Releases
Business Test and Release
Value Coding
Requirements/Design
True Technical Progress
Late scrap
and rework
Efficiency Cost of Change
Integration
Implementation
Maintenance Unit Test
Implementation Integration Maintenance
Unit Test
Control
Executive Summit 28
29. IBM Rational Software Conference 2009
Improving process and increasing flexibility Improve
Process
Reducing the significant uncertainties in quality and Increase
Flexibility
and performance
WATERFALL DEVELOPMENT ITERATIVE DEVELOPMENT
AND AGILE DELIVERY
Late quality and performance insight Continuous quality and performance insight allows
constrains flexibility to make tradeoffs flexibility in trading off cost, quality, and features
Measured First indications
of performance Demonstrated MTBF
progress challenges
and
quality
Indications
of other quality
challenges Software
MTBF allocation
Requirements/Design
Baseline and freeze Requirements/Design negotiation
Speculative quality requirements Release qualities that matter
Unpredictable cost/schedule performance Quality progressions/digressions
Late shoehorning of suboptimal changes Early requirement verification
that impact quality and/or negotiation
Executive Summit 29
30. Slide 29
A1 4) finally, on slide 4, i changed the heading that was iterative development to both iterative development and agile delivery. not sure if
that matters but it matches my paper and danny may prefer agile delivery rather than iterative development.
Author, 4/29/2009
31. IBM Rational Software Conference 2009
Measured Capability Improvement Framework (MCIF):
A systematic approach to software excellence
Determine
right solution Phase Elicit and set business value objectives
to deploy 1
Business objectives, contexts to operate
within, and development approaches
Phase Determine the solution components
2
Incremental improvement roadmap
Do Financial Analysis
Deploy solution
Phase Accelerate and monitor solution adoption using Rational’s
and measure 3
results preferred approach
Deploy practices and tools
Adopt proper usage model to maximize ROI
Measure and adapt
Phase
4 Review and communicate business results
Compile business value and results
Confirm results delivered
and start on next project Conduct formal review with stakeholders
Executive Summit 30
32. IBM Rational Software Conference 2009
Some final thoughts
Software delivery is a discipline of software economics
balancing risks and opportunities
Process enactment and measurement are imperatives
to achieving agility at scale
Software delivery requires a platform that is architected
for automation, collaboration and reporting
Executive Summit 31
