This document discusses quantitatively managing agile software development. It proposes using readily available data to benchmark agile team performance and optimize agile release contracts. The first paper explains how to benchmark agile releases. The second discusses optimizing agile release timeboxes based on team performance and constraints. The third explains managing dynamic changes to the agile "contract" using statistical process control. It advocates continuously updating benchmarks to help government agencies objectively evaluate supplier performance and negotiate contracts.

1. © 2017 QSMA AG 1
Manage Agile Software
Development
Quantitatively
Quantitative Software Management: QSM

2. (#2)
Agile
•Paper 1 Agile Release
Benchmark
Agile
•Paper 2 Agile Release
Time Box
Agile
•Paper 3 Managing
Agile Development
QSM: Manage Agile Quantitatively
Shows how readily available data is used to
benchmark Agile team performance and
reveal the cost of development due to high
time pressure. The QSM benchmark method
applies to all categories of software
development and development methods.
Sets out how informed cost and feature
content decisions are made to optimise the
Agile development plan consistent with the
team performance and the Time Box
constraints. This includes quantifying the
number of Sprints, their duration and Agile
Release feature content.
Explains how to use the formal Time Box
contract with statistical process control to
manage changes dynamically throughout
development. This supports visibility and
flexibility to control agreed changes to the
“contract” Agile Release content .
James Greene
QSM Ltd.
41A Aynhoe Road,
London W14 0QA
Tel: 44-20-7603-9009
E-mail : j.greene@qsmuk.com
Andrea Gelli
QSM Associates Switzerland
Fehrenstrasse 20
8032 Zürich
Tel: 41 76 4374545
E-mail: andrea.gelli@qsma.ch
For
copies
of
these
papers
please
contact
QSM

3. © 2017 QSMA AG 3
UK Institute for Government (IFG)
– “UK Government IT projects have become
notorious for running far behind schedule and failing to deliver the expected
benefits “
The report labels UK Government as
– “A less than intelligent customer. It has become unable to
judge objectively whether it is getting a good deal from suppliers”.
The report recommends:
• “Agile projects - Agile methods can deliver better outcomes at lower cost more
quickly. Agile focuses on delivering useable functionality quickly, rather than a
‘perfect solution’ late”.
Harvard Business Review
– “People always thought that the public sector was doing worse in IT than
private companies - our findings suggest they're just as bad.”
3
Is Agile Really the Solution?

4. © 2017 QSMA AG 4
Manage the Agile Road Blocks Quantitatively
IS AGILE
PRODUCTIVE?
WHAT AM I
BUYING?
CONTRACT
BASELINE
CONTROL?
BENCHMARK
AGILE!
OPTIMISE AGILE
RELEASE CONTRACT
BASELINE!
MANAGE THE AGILE
CONTRACT: RISK/
CHANGES/QUALITY
Benchmark
Data Base

5. © 2017 QSMA AG 5
VALUE FOR MONEY?
Contract
Agile Time Box: What Am I Buying?
5

6. (#6)
1
2
3
4
5
6
7
8
9
10
11
12
0
2
4
6
8
10
12
14
Agile Release Time Box Contract
QSM Quantifies Cost/Benefit Alternatives
6
€0
€500,000
€1,000,000
€1,500,000
€2,000,000
€2,500,000
€3,000,000
€3,500,000
€4,000,000
€4,500,000
€5,000,000
S1 S2 S3 S4 S5 S6 S7 S8 S9 S10 S11 S12
Sprint Benefits
Agile Release Time Box
Cost/Benefit Alternatives
• How many Months?
• What Team size?
• How much Software?
• How many Features?
• How many Sprints?
• How many Features per Sprint
• What Sprint Interval?
Agile Release Time/Sprint/Cost Alternatives
FEATURE BENEFIT
NUMBER ESTIMATE
Feature 1 € 200,000
Feature 2 € 180,000
Feature 3 € 160,000
Feature 4 € 160,000
Feature 5 € 145,000
Feature 6 € 140,000
Feature 7 € 140,000
Feature 8 € 140,000
∑
Compose
Features
in to
Sprints
C
O
S
T
?

7. (#7)
AGILE TIME BOX ALTERNATIVES (PERSON YEAR COST €100,000)
STAFF 40/30/20 RELEASE TIME 5/6/7 MONTHS
TEAM PERFORMANCE INDEX PI = 20
MONTHS MONTHS MONTHS
RELEASE TIME 5 6 7
40 STAFF FEATURES 23 34 40+
TIME PRESSURE 7.2 6.7 6.3
COST €Million 4.1M 5.0M 5.8M
30 STAFF FEATURES 20 31 40+
TIME PRESSURE 6.8 6.2 5.8
COST €Million 3.1M 3.8M 4.4M
20 STAFF FEATURES 18 26 36
TIME PRESSURE 6.2 5.7 5.2
COST €Million 2.0M 2.5M 2.9M
Quantify Agile Release Time Box Cost/Benefit
Alternatives

8. © 2017 QSMA AG 8
Agree Contract Baseline: Agile Release 30 Staff 6 Months
Work Breakdown Structure : 8 Sprints@3 weeks
AGILE RELEASE ~ 101K TIME BOX 6 MONTHS PI 20 STAFF 30
SPRINT WBS
1 2 3 4 5 6 7 8
Phase 3: AGILE SPRINTS
MANGEMENT & SUPPORT
MANAGEMENT
SUPPORT
SPRINTS 1- 8
SPRINT 1
SPRINT 2
SPRINT 3
SPRINT 4
SPRINT 5
SPRINT 6
SPRINT 7
SPRINT 8
BUILD TEAM
INTEGRATION
BUILD
PERFORMANCE
QUALITYASSURANCE
QUALITYASSURANCE
Tasks
271817161514131211109876543210
Milestones
0 - SP1S
1 - SP1E
2 - SP2S
3 - STINT
4 - SP2E
5 - SP3S
6 - STBLD
7 - SP3E
8 - SP4S
9 - SP4E
10 - SP5S
11 - STPER
12 - SP5E
13 - SP6S
14 - SP6E
15 - SP7S
16 - SP7E
17 - SP8S
18 - SP8E
27 - AgRel
Milestones
0 - SP1S
1 - SP1E
2 - SP2S
3 - STINT
4 - SP2E
5 - SP3S
6 - STBLD
7 - SP3E
8 - SP4S
9 - SP4E
10 - SP5S
11 - STPER
12 - SP5E
13 - SP6S
14 - SP6E
15 - SP7S
16 - SP7E
17 - SP8S
18 - SP8E
27 - AgRel
8
Agile Release
Baseline: Mandatory
Daily/Weekly
Sprint Progress
• Reporting
• Risk Assessment
• Change Control
Contract Use to approve & control expected changes
CUMULATIVE CUMULATIVE
RELEASED SPRINT STORY
FEATURES NUMBER POINTS
3 1 7
6 2 14
9 3 21
12 4 29
15 5 38
18 6 44
21 7 52
25 8 61

9. © 2017 QSMA AG 9
AGILE RELEASE 6 MONTH 8 SPRINTS 25 FEATURES FORECAST WEEK 9
25 FEATURES
TIME BOX 61 STORY POINTS
2 4 6 8 10 12 14 16 18 20 22 24 26 28 30
Agile Release Weeks
1
11
21
31
FEATURE
271817161514131211109876543210
FORECAST SPRINT 3
PLAN:ACTUAL:FORECAST
2 4 6 8 10 12 14 16 18 20 22 24 26 28 30
Agile Release Weeks
AGILE SPRINTS
Phases
271817161514131211109876543210
876543210 271817161514131211109
8 SPRINTS
TIME BOX 25 FEATURES
2 4 6 8 10 12 14 16 18 20 22 24 26 28 30
Agile Release Weeks
0
2
4
6
8
10
12
14
SPRINTS
271817161514131211109876543210
STORY POINTS
BURN DOWN
2 4 6 8 10 12 14 16 18 20 22 24 26 28 30
Agile Release Weeks
0
20
40
60
80
STORYPOINTSBURNDOWN
271817161514131211109876543210
Date 05/08/2010 (31.00 weeks)
FEATURES (FEATURE)
SPRINTS (SPRINTS)
BurnDown (BDown)
PI
MBI
Plan
25.0
8.0
0.0
20.2
6.3
Actual/
Forecast
25.0
8.0
0.0
19.1
5.8
Est. to
Complete
0.0
0.0
39.0
Current Plan Actuals Current Forecast Green Control Bound Yellow Control Bound Project: AGILE TIME BOX
Agile Release: Quantify and Track Weekly Progress
Forecast: Week 12 at Sprint 4 Slippage
Mandatory
Weekly
Progress
Reporting
Indicators
Overall Red
Slippage
Full Metric Assessment
Date
02/01/2010
09/01/2010
23/01/2010
30/01/2010
06/02/2010
13/02/2010
20/02/2010
27/02/2010
06/03/2010
13/03/2010
20/03/2010
Overall
Assessment
Cum
Effort
(Life
Cycle)
Cum
Cost
(Life
Cycle)
Cum Eff
SLOC
Cum Size
Assessment
CumEffort
Assessment
Cum
Cost
Assessment SPRINTS FEATURES
FEATURE
SIZE BurnDown
Forecast Slippage

10. © 2017 QSMA AG 10
AGILE RELEASE 6 MONTH 7 SPRINTS 23 FEATURES FORECAST WEEK (RECOVERY)
23 FEATURES
TIME BOX 54 STORY POINTS
2 4 6 8 10 12 14 16 18 20 22 24 26
Agile Release Weeks
1
11
21
31
FEATURE
271817161514131211109876543210
FORECAST SPRINT 4 RECOVERY DROP SPRINT 8
PLAN:ACTUAL:FORECAST
2 4 6 8 10 12 14 16 18 20 22 24 26
Agile Release Weeks
AGILE SPRINTS
Phases
271817161514131211109876543210
876543210 271817161514131211109
7 SPRINTS (RECOVERY)
TIME BOX 23 FEATURES
2 4 6 8 10 12 14 16 18 20 22 24 26
Agile Release Weeks
0
2
4
6
8
10
12
14
SPRINTS
271817161514131211109876543210
STORY POINTS
54 BURN DOWN
2 4 6 8 10 12 14 16 18 20 22 24 26
Agile Release Weeks
0
10
20
30
40
50
60
70
STORYPOINTSBURNDOWN
271817161514131211109876543210
Date 02/07/2010 (26.14 weeks)
FEATURES (FEATURE)
SPRINTS (SPRINTS)
BurnDown (BDown)
PI
MBI
Plan
23.0
7.0
0.0
20.2
6.3
Actual/
Forecast
23.0
7.0
0.0
20.0
6.1
Est. to
Complete
0.0
0.0
32.0
Current Plan Actuals Current Forecast Green Control Bound Yellow Control Bound Project: AGILE TIME BOX
Achieve Agile Release Time Box Contract:
Effort and Schedule Recovery Drop Sprint 8
Agree Agile Release Baseline now 7 Sprints
exclude Sprint 8- drop Features 24 & 25
Burn Down now 54 Story Points
Plan Team Performance
PI = 20
Current PI = 20
All Traffic Lights
Again Green

11. © 2017 QSMA AG 11
Agile Release Completion: Full History:
Update Agile Benchmark Database
Agile Release Final Completed/ Delivered Contract Baseline
Update Benchmark Data : Time 30 weeks Effort 175 Person Months
Software Size : 54 Story Points
Team Performance PI 19.5 Time Pressure 6
Full History
Agile Release
Weekly
Metrics
Update Team
Performance
Data Base
Full Metric Assessment
Date
02/01/2010
09/01/2010
23/01/2010
30/01/2010
06/02/2010
13/02/2010
20/02/2010
27/02/2010
06/03/2010
13/03/2010
20/03/2010
27/03/2010
03/04/2010
10/04/2010
17/04/2010
24/04/2010
01/05/2010
08/05/2010
15/05/2010
22/05/2010
29/05/2010
05/06/2010
12/06/2010
19/06/2010
26/06/2010
03/07/2010
Overall
Assessment
Cum
Effort
(Life
Cycle)
Cum
Cost
(Life
Cycle)
Cum Eff
SLOC
Cum Size
Assessment
CumEffort
Assessment
Cum
Cost
Assessment SPRINTS FEATURES
FEATURE
SIZE BurnDown
Update Team
Performance
Benchmark
Data Base

12. © 2017 QSMA AG 12
• IFG paper “It (UK Government) has become unable to judge objectively
whether it is getting a good deal from suppliers, especially as the siloed
nature of government make it difficult to obtain comparative figures for
reference.”
• Solution:
– Continuously update the Benchmark Database
– Consolidate across all UK Government departments
– Stratify by supplier/application complexity
– Use the results to negotiate new developments
– Set supplier team performance improvement targets
Add to the Team Performance Benchmark Data Base:
Solve the UK Government “Siloe” Problem
UK Govt. Supplier
Team Performance
Benchmark
Data Base

13. © 2017 QSMA AG 13
To Manage Agile
Quantitatively
Please Contact QSM
SLIM-MasterPlan
Manage
Portfolio
SLIM-Estimate
Estimate
Projects
SLIM-Control
Track &
Forecast
SLIM-Metrics
Analyze &
Benchmark
SLIM-DataManager
Manage
Historical Data
James Greene
Quantitative Software Management Ltd.
41A Aynhoe Road, London W14 0QA
Tel: 44-20-7603-9009
E-mail : j.greene@qsmuk.com
Andrea Gelli
QSM Associates Switzerland
Fehrenstrasse 20, 8032 Zürich
Tel: 41 76 4374545
E-mail: andrea.gelli@qsma.ch