Presentation given at Big Apple Scrum Day 2015 on advanced metrics for agile and scrum teams. It is recommended that teams track a few metrics for each of the 5 categories outlined in the presentation to be able to assess the impact of activities supportive of continuous improvement. The presentation includes over 30 metrics to give teams ideas on what they can measure. There isn’t a requirement to track 30 metrics on a scrum or agile team but rather teams should track just enough metrics to be able to understand their performance.

1. ©
2012,
Asynchrony
Solu2ons,
Inc.
All
rights
reserved.
ADVANCED
SCRUM
METRICS
Big
Apple
Scrum
Day
–
June
1,
2015
Jason
Tice
@theagilefactor
jason@theagilefactor.com

2. ©
2012,
Asynchrony
Solu2ons,
Inc.
All
rights
reserved.
Alterna4ve
Title:
As
many
as
30+
metrics
other
than
velocity
that
may
help
your
team
improve

3. ©
2012,
Asynchrony
Solu2ons,
Inc.
All
rights
reserved.
Hypotheses
Some
scrum
teams
do
not
have
easy
access
to
sufficient
data
to
assess
the
true
impact
of
process
changes
and
promote
effec2ve
self-­‐management.
Without
sufficient
measurements
encompassing
all
delivery
ac2vi2es,
scrum
teams
can
fall
vic2m
to
improving
one
aspect
of
their
process
while
other
aspects
of
their
process/product
suffer.
Lack
of
encompassing
quan2ta2ve
measurements
reduces
team
effec2veness
as
decisions
may
be
made
based
upon
the
beliefs,
theories
and/or
egos
of
team
members
without
scien2fic
data
to
assess
the
impact
and
effec2veness
of
decisions
made.

4. ©
2012,
Asynchrony
Solu2ons,
Inc.
All
rights
reserved.
5
Types
of
Scrum
Metrics
Process
Health
• Assess
day-­‐to-­‐day
delivery
team
ac2vi2es
&
evaluate
process
changes
Release
• Direct
focus
to
iden2fy
impediments
to
con2nuous
delivery
Product
Development
• Align
product
features
to
user
needs
Technical
/
Code
• Determine
quality
of
implementa2on
and
architecture
People
/
Team
• Promote
sustainable
pace
and
team
engagement
Recommenda4on
• Teams
track
several
metrics
from
each
type
to
have
a
well-­‐rounded
view
of
their
process
&
product
development
ac2vi2es

5. ©
2012,
Asynchrony
Solu2ons,
Inc.
All
rights
reserved.
Metric:
Velocity
Type
-­‐
Process
Health
What
• Number
of
work
items
completed
in
a
period
of
2me
Why
• Assess
the
impact
of
team
process
changes
Goal
• Velocity
is
intended
to
promote
effec2ve
self-­‐managing
teams

6. ©
2012,
Asynchrony
Solu2ons,
Inc.
All
rights
reserved.
Disclaimer
Teams
should
track
sufficient
metrics
to
guide
&
evaluate
improvements
• The
right
metrics
are
determined
by
a
team
and
its
sponsors
Adop4on
of
metrics
should
be
incremental
• Do
not
aVempt
to
start
gathering
all
30+
metrics
at
once
All
metrics
have
costs
to
gather
&
review
• A
team’s
metrics
should
evolve
based
on
observed
challenges,
risks,
and
needs
• Teams
should
track
enough
just
enough
metrics
to
understand
how
they
work
and
how
they
can
improve

7. ©
2012,
Asynchrony
Solu2ons,
Inc.
All
rights
reserved.
Why
Measure
• Assess
the
impact
/
value
of
process
improvements
using
scien2fic
methods:
• Problem
Statement
• Hypothesis
• Ac2on
Plan
• Gather
Data
• Conclusion
• Increase
effec2veness
of
self-­‐organiza2on
–
data
doesn’t
lie
and
creates
a
founda2on
for
team
members
to
make
cri2cal
decisions
• Collec2on
and
transparency
of
data
promotes
cross-­‐team
knowledge
sharing
–
data
enables
teams
to
have
engaging
conversa2ons
about
impediments
and
mi2ga2ons

8. ©
2012,
Asynchrony
Solu2ons,
Inc.
All
rights
reserved.
Best
Prac4ces
• Metrics
are
about
understanding
trends
–
not
hi`ng
magic
numbers
• Respect
that
desired
trends
and/or
quan2ta2ve
goals
are
unique
to
each
team
• Goals
should
be
arrived
at
by
team
consensus
–
not
mandated
by
management
• Use
metrics
to
encourage
team/value
contribu2on
vs.
establishing
baselines
across
all
teams
• Metrics
should
be
transparent
and
easily
accessible
to
all
teams
–
ideally
posted
as
informa2on
radiators
/
dashboards
• Use
metrics
to
direct
the
team’s
focus
to
agreed
upon
challenges
–
enable
team
members
to
self-­‐diagnosis
scenarios

9. ©
2012,
Asynchrony
Solu2ons,
Inc.
All
rights
reserved.
Metric:
Cycle
Time
Type
-­‐
Process
Health
What
• Amount
of
2me
needed
to
complete
a
work
item
Why
• Consistent
cycle
2me
increases
the
predictability
of
work
Goal
• Rela2vely
consistent
cycle
2me
for
each
work
item
size

10. ©
2012,
Asynchrony
Solu2ons,
Inc.
All
rights
reserved.
Metric:
Cycle
Time

11. ©
2012,
Asynchrony
Solu2ons,
Inc.
All
rights
reserved.
Metric:
Escaped
Defects
Type
-­‐
Release
What
• Count
of
defects
that
are
discovered
in
produc2on
Why
• Iden2fy
root
causes
as
to
why
defect
not
detected
prior
to
release
Goal
• Agreement
on
acceptable
level
of
escaped
defects
with
shared
understanding
of
mi2ga2ons
iden2fied
to
reduce
future
escaped
defects

12. ©
2012,
Asynchrony
Solu2ons,
Inc.
All
rights
reserved.
Metric:
Business
Value
Burnup
Type
-­‐
Product
Development
What
• The
amount
of
business
value
provided
by
each
completed
work
item
Why
• Allow
customers
&
stakeholders
to
manage
ROI
Goal
• Focused
on
highest
value
work
items
first;
consider
moving
on
as
value
per
work
item
decreases

13. ©
2012,
Asynchrony
Solu2ons,
Inc.
All
rights
reserved.
Metric:
Test
Coverage
Type
-­‐
Technical
/
Code
What
• Percentage
of
codebase
exercised
by
various
types
of
automated
tests
Why
• Guide
efforts
/
investments
to
improve
test
coverage
to
sufficient
levels
Goal
• Examine
trends
&
correla2ons
between
Defect
Density
and
Test
Coverage

14. ©
2012,
Asynchrony
Solu2ons,
Inc.
All
rights
reserved.
Metric:
Test
Coverage

15. ©
2012,
Asynchrony
Solu2ons,
Inc.
All
rights
reserved.
Metric:
Happiness
Metric
Type
-­‐
Team
/
People
What
• Team
member
sa2sfac2on
as
a
member
of
the
team
(5
point
scale)
Why
• Create
transparency
regarding
team
member
sa2sfac2on
Goal
• Enable
team
to
self-­‐manage
/
improve
team
morale

16. ©
2012,
Asynchrony
Solu2ons,
Inc.
All
rights
reserved.
Metric:
Happiness
Metric

17. ©
2012,
Asynchrony
Solu2ons,
Inc.
All
rights
reserved.
Metric:
Cumula4ve
Flow
Type
-­‐
Process
Health
What
• Amount
of
work-­‐in-­‐progress
at
any
state
at
any
2me
Why
• Iden2fy
boVlenecks
that
reduce
flow
and
assess
impact
of
process
changes
Goal
• Team
operates
within
capacity
as
incremental
progress
made
toward
goal
(narrow
bands
with
consistent
posi2ve
slope)

18. ©
2012,
Asynchrony
Solu2ons,
Inc.
All
rights
reserved.
Metric:
Cumula4ve
Flow

19. ©
2012,
Asynchrony
Solu2ons,
Inc.
All
rights
reserved.
Metric:
Escaped
Defect
Resolu4on
Time
Type
-­‐
Release
What
• Amount
of
2me
required
to
resolve
an
escaped
defect
Why
• Understand
the
unplanned
cost
of
resolving
escaped
defects
Goal
• Achieve
balance
between
proac2ve
tes2ng
and
sustainable
Service
Level
Agreements
(SLAs)

20. ©
2012,
Asynchrony
Solu2ons,
Inc.
All
rights
reserved.
Metric:
Risk
Burndown
Type
-­‐
Product
Development
What
• Amount
of
known
and
unmi2gated
risk
shown
across
a
period
of
2me
Why
• Encourage
self-­‐management
to
reduce
project
risk
Goal
• Product
risk
trends
down
to
acceptable
level
over
2me;
balance
investment
between
building
product
and
reducing
risk

21. ©
2012,
Asynchrony
Solu2ons,
Inc.
All
rights
reserved.
Metric:
Risk
Burndown

22. ©
2012,
Asynchrony
Solu2ons,
Inc.
All
rights
reserved.
Metric:
Build
Time
Type
-­‐
Technical
/
Code
What
• Execu2on
2me
to
run
build
and
tests
to
provide
developer
feedback
Why
• Guard
against
slow
builds
&
test
execu2on
that
reduce
frequency
of
feedback
Goal
• Build
provides
feedback
to
team
within
an
agreed
upon
amount
of
2me

23. ©
2012,
Asynchrony
Solu2ons,
Inc.
All
rights
reserved.
Metric:
Learning
Log
Type
-­‐
Team
/
People
What
• A
lis2ng
of
items
the
team
(or
team
members)
have
learned
Why
• Direct
focus
to
the
importance
of
learning
on
scrum
teams
/
projects
Goal
• Promote
learning
throughout
a
project;
suppor2ve
of
self-­‐
management
and
sustaining
team
morale
&
engagement

24. ©
2012,
Asynchrony
Solu2ons,
Inc.
All
rights
reserved.
Metric:
Percent
Complete
&
Accurate
Type
-­‐
Process
Health
What
• Number
of
completed
and
acceptable
work
items
Why
• Improve
delivery
by
measuring
comple2on
and
quality
Goal
• High
percentage
of
commiVed
work
is
complete
&
accurate
• Promote
clear
defini2on
and
consensus
on
done
criteria

25. ©
2012,
Asynchrony
Solu2ons,
Inc.
All
rights
reserved.
Metric:
Release
Success
Rate
Type
-­‐
Release
What
• Ra2o
of
accepted
vs.
rejected
releases
from
the
customer
Why
• Encourage
partnership
between
team
&
customer
Goal
• High
percentage
of
accepted
releases
due
to
effec2ve
process,
consensus
on
done
criteria
&
feedback
loops

26. ©
2012,
Asynchrony
Solu2ons,
Inc.
All
rights
reserved.
Metric:
Push
/
Pull
Type
-­‐
Product
Development
What
• The
ra2o
/
count
of
work
items
completed
vs.
work
items
added
Why
• Guard
against
team
being
overwhelmed
with
work
that
can
compromise
promise
Goal
• Push
/
Pull
is
balanced
–
new
work
is
added
when
work-­‐in-­‐
progress
is
completed

27. ©
2012,
Asynchrony
Solu2ons,
Inc.
All
rights
reserved.
Metric:
Push
/
Pull

28. ©
2012,
Asynchrony
Solu2ons,
Inc.
All
rights
reserved.
Metric:
Defect
Density
Type
-­‐
Technical
/
Code
What
• Percentage
of
defects
in
each
area
of
our
system
–
determined
by
func2onality
or
code
architecture
Why
• Iden2fy
parts
of
the
app/code
where
quality
can
be
improved
Goal
• Data
can
help
to
iden2fy
hidden
/
unknown
tech
debt,
and/or
unclear
development
steps
/
done
criteria

29. ©
2012,
Asynchrony
Solu2ons,
Inc.
All
rights
reserved.
Metric:
Team
Tenure
Type
-­‐
Team
/
People
What
• How
long
has
each
team
member
been
on
the
team
Why
• Encourage
ac2vi2es
reflec2ve
of
tenure
(mentoring
for
new
team
members,
job/knowledge
sharing
for
long-­‐standing
team
members)
Goal
• Promote
whole
team
approach
and
ability
to
shik
staff
between
teams
with
less
risk

30. ©
2012,
Asynchrony
Solu2ons,
Inc.
All
rights
reserved.
Metric:
Flow
Efficiency
Type
-­‐
Process
Health
What
• Ra2o
of
2me
spent
working
on
an
item
vs.
2me
the
item
waits
Why
• Calibrate
Work-­‐In-­‐Progress
(WIP)
limits
to
minimize
delay
and
promote
flow
Goal
• Consistent
wait
/
run
ra2o
for
each
work
item,
then
increase
efficiency

31. ©
2012,
Asynchrony
Solu2ons,
Inc.
All
rights
reserved.
Metric:
Release
Time
Type
-­‐
Release
What
• Amount
of
2me
required
to
release
the
product
to
a
produc2on-­‐like
environment
(or
produc2on
itself)
Why
• Establish
consensus
on
sustainable
cost/2me
for
a
produc2on
release
Goal
• Sufficient
automa2on
to
reduce
2me/cost
for
desired
level
of
business
agility

32. ©
2012,
Asynchrony
Solu2ons,
Inc.
All
rights
reserved.
Metric:
Product
Forecast
Type
-­‐
Product
Development
What
• Future
trend
lines
(best-­‐case,
worst-­‐case)
based
on
historical
performance
of
work
item
comple2on
Why
• Predict
when
future
work
will
be
completed
using
work
item
count
Goal
• Stable
performance
to
allow
for
sufficient
forecas2ng

33. ©
2012,
Asynchrony
Solu2ons,
Inc.
All
rights
reserved.
Metric:
Product
Forecast
(shown
as
burn-­‐up)

34. ©
2012,
Asynchrony
Solu2ons,
Inc.
All
rights
reserved.
Metric:
Product
Forecast
(shown
as
interval)

35. ©
2012,
Asynchrony
Solu2ons,
Inc.
All
rights
reserved.
Metric:
Code
Churn
Type
-­‐
Technical
/
Code
What
• Number
of
lines
of
code
changed
to
complete
a
work
item
Why
• Assess
if
the
amount
of
code
changed
is
reflec2ve
of
the
work
item
addressed
Goal
• Promote
whole-­‐team
understanding
of
the
code-­‐base
and
alignment
to
agreed
upon
design
/
code
paVerns
&
standards

36. ©
2012,
Asynchrony
Solu2ons,
Inc.
All
rights
reserved.
Metric:
Code
Churn

37. ©
2012,
Asynchrony
Solu2ons,
Inc.
All
rights
reserved.
Metric:
Phone-­‐A-­‐Friend
Stats
Type
-­‐
Team
/
People
What
• Number
of
2mes
a
former
team
member
needs
to
be
contacted
for
assistance
Why
• Assess
effec2veness
of
job
sharing
and
knowledge
transfer
ac2vi2es
when
team
composi2on
changes
Goal
• Encourage
whole-­‐team
approach
and
shared
work

38. ©
2012,
Asynchrony
Solu2ons,
Inc.
All
rights
reserved.
Metric:
Time
Blocked
per
work
item
Type
-­‐
Process
Health
What
• Amount
of
2me
that
a
work
item
was
blocked
during
its
comple2on
Why
• Determine
the
cost
of
delay
&
propose
proac2ve
mi2ga2ons
/
avoidances
Goal
• Reduc2on
in
2me
blocked
per
story
through
avoidance
and
faster
mi2ga2on

39. ©
2012,
Asynchrony
Solu2ons,
Inc.
All
rights
reserved.
Metric:
Time
Blocked
per
work
item

40. ©
2012,
Asynchrony
Solu2ons,
Inc.
All
rights
reserved.
Metric:
Time
Since
Last
Release
Type
-­‐
Release
What
• Amount
of
2me
since
the
team
last
released
their
product
to
“real”
users
Why
• Encourage
teams
to
integrate
more
user
feedback
into
development
ac2vi2es
Goal
• Team
is
able
to
get
sufficient
feedback
from
“real”
users
to
build
the
“right”
product

41. ©
2012,
Asynchrony
Solu2ons,
Inc.
All
rights
reserved.
Metric:
Product
Net
Promoter
Score
Type
-­‐
Product
Development
What
• Would
you
recommend
this
product
to
a
colleague?
Why
• Gather
simple
user
feedback
on
if
the
product
meets
user
needs
Goal
• Iden2fy
successful
elements
of
product
design
and
opportuni2es
/
ideas
to
increase
user
adop2on

42. ©
2012,
Asynchrony
Solu2ons,
Inc.
All
rights
reserved.
Metric:
Product
Net
Promoter
Score

43. ©
2012,
Asynchrony
Solu2ons,
Inc.
All
rights
reserved.
Metric:
Code
Ownership
Type
-­‐
Technical
/
Code
What
• Frequency
that
team
members
change
or
commit
to
each
area
of
the
code
base
Why
• Assess
and
promote
collec2ve
code
ownership
Goal
• No
no2ceable
areas
of
the
code
where
single
team
members
make
all
the
changes

44. ©
2012,
Asynchrony
Solu2ons,
Inc.
All
rights
reserved.
Metric:
Whole
Team
Contribu4on
Type
-­‐
Team
/
People
What
• Percentage
of
team
members
that
contribute
to
a
work
item
throughout
its
lifecycle
Why
• Quan2ta2ve
metric
to
assess
/
improve
whole-­‐team
approach
Goal
• Work
items
completed
with
at
least
the
agreed
upon
percentage
of
team
members
contribu2ng
to
them

45. ©
2012,
Asynchrony
Solu2ons,
Inc.
All
rights
reserved.
Metric:
Blocker
Clustering
Type
-­‐
Process
Health
What
• Frequency
and
grouping
of
items
that
block
work
items
Why
• Iden2fy
largest
sources
of
delay
and
propose
common
mi2ga2ons
Goal
• Reduc2on
in
occurrence
of
types
of
blockers
via
proac2ve
mi2ga2ons

46. ©
2012,
Asynchrony
Solu2ons,
Inc.
All
rights
reserved.
Metric:
Cost
Per
Release
Type
-­‐
Release
What
• The
cost
to
complete
a
sokware
release
(planned
and/or
unplanned)
Why
• Enable
considera2on
of
economic
factors
when
deciding
when/if
to
release
Goal
• Encourage
investments
to
reduce
release
cost
via
automa2on
to
an
agreed
upon
level

47. ©
2012,
Asynchrony
Solu2ons,
Inc.
All
rights
reserved.
Metric:
User
Analy4cs
Type
-­‐
Product
Development
What
• Iden2fy
usage
paVerns
within
the
product
Why
• Determine
effec2veness
of
design;
look
for
emergent
usage
paVerns
that
warrant
considera2on
for
future
investment
Goal
• Augment
business
analysis
and
product
design
ac2vi2es
with
actual
user
behavior

48. ©
2012,
Asynchrony
Solu2ons,
Inc.
All
rights
reserved.
Metric:
Code
Complexity
Type
-­‐
Technical
/
Code
What
• Cycloma2c
complexity
score
of
product
code
base
determined
by
a
tool
Why
• Promote
engineering
prac2ces
to
create
clean
code
using
quan2ta2ve
data
Goal
• Enable
team
to
assess
and
understand
upward/downward
trend
of
code
complexity

49. ©
2012,
Asynchrony
Solu2ons,
Inc.
All
rights
reserved.
Metric:
Code
Complexity

50. ©
2012,
Asynchrony
Solu2ons,
Inc.
All
rights
reserved.
Metric:
Release
Net
Promoter
Score
Type
-­‐
Release
What
• Would
you
recommend
this
product
based
on
the
new
features
included
with
this
release?
Why
• Determine
if
new
features
align
to
user
needs
Goal
• Integrate
user
feedback
from
prior
releases
into
future
enhancements;
ability
to
collect
feedback
to
improve
product
/
adop2on

51. ©
2012,
Asynchrony
Solu2ons,
Inc.
All
rights
reserved.
Metric:
Standards
Adherence
Type
-­‐
Technical
/
Code
What
• Assessment
score
of
code
alignment
to
architecture
standards
Why
• Promote
agreed
upon
coding
standards
to
create
clean
code
Goal
• Enable
team
learning
to
beVer
align
code
to
agreed
upon
design
paVerns;
iden2fy
code
that
can
be
improved
that
may
not
be
evident
by
code
review

52. ©
2012,
Asynchrony
Solu2ons,
Inc.
All
rights
reserved.
Metric:
Standards
Adherence

53. ©
2012,
Asynchrony
Solu2ons,
Inc.
All
rights
reserved.
Metric:
Release
adop4on
/
install
rate
Type
-­‐
Release
/
Product
Development
What
• Number
of
exis2ng
users
that
have
upgraded;
Number
of
new
users
gained
from
release
Why
• Assess
ROI
on
product
development
and
validate
business
/
market
assump2ons
Goal
• ROI
meets
or
exceeds
business
assump2ons;
iden2fy
users
that
have
not
upgraded
–
determine
why

54. ©
2012,
Asynchrony
Solu2ons,
Inc.
All
rights
reserved.
Metric:
Crash
Rate
Type
-­‐
Technical
/
Code
What
• Log
incidents
that
cause
our
applica2on
/
product
to
crash
Why
• Be
able
to
perform
root
cause
analysis
to
reduce
crashes
Goal
• Gain
insights
into
product
errors
in
released
code
that
have
significant
impacts
on
the
user
experience

55. ©
2012,
Asynchrony
Solu2ons,
Inc.
All
rights
reserved.
Your
Metric
Many
@mes
the
best
measurements
are
defined
by
team
members
who
have
the
most
knowledge
of
a
specific
problem
or
challenge
Type
-­‐
?
What
• ?
Why
• ?
Goal
• ?

56. ©
2012,
Asynchrony
Solu2ons,
Inc.
All
rights
reserved.
Conclusion
Next
Steps:
• Are
you
measuring
enough
to
assess
the
impact
of
changes
/
improvements
made?
• Have
a
team
discussion
and
decide
if
any
of
these
30+
metrics
(or
new
ones
you
think
up)
would
promote
beVer
self-­‐
management.
Ques4ons
&
Contact:
Jason
Tice
–
www.asynchrony.com
@theagilefactor
–
www.theagilefactor.com
jason@theagilefactor.com

57. ©
2012,
Asynchrony
Solu2ons,
Inc.
All
rights
reserved.
©
Copyright
2013
Asynchrony