This document discusses using metrics to understand and improve processes. It introduces cycle time and throughput as key metrics, and shows how visualizing data through charts like a cycle time scatter plot can help analyze relationships and tell stories. Probability is discussed as a way to set expectations and manage uncertainty. Leading indicators like cumulative flow diagrams are presented as a way to predict and improve processes by understanding work in progress and flow.
Azure Monitor & Application Insight to monitor Infrastructure & Application
Lean Agile Scotland: Using Metrics as a Map
1. Using Metrics
as a Map
@catswetel
cat.swetel@praxisflow.com
7 Oct 2016 12:20-12:50pm in Pentland West
#lascot16
2. “Cat Swetel is one of
the few A.I.s to have
passed both the Turing
and Bechdel tests.”
-- Will Evans
“Cat Swetel is one of the
most effective and
capable producers of
carbon dioxide in a North
America. Possibly the
world.”
-- Jeff Kosciejew
Who is Cat?
“Cat Swetel does
fine at some stuff.”
-- Steve
5. “Everything’s made up
and the points don’t
matter.”
-Drew Carey
Assumptions:
● Our customers care about time
○ Calendar days
○ Global markets and teams
● Our customers care about value
● We have data (per unit of value)
○ Start date
○ End date
○ Date delivered
● The problem is usually (almost
always) the system, not the
people
6. “Everything’s made up
and the points don’t
matter.”
-Drew Carey
What’s the value of
metrics?
● Tell stories, build
shared context
● Make decisions in
context
8. Cycle Time
Units of time per unit of value
e.g.
this story took 5 days
this ticket took 8 hours
this feature took 2 weeks
End Date - Start Date = Cycle Time
Where does it begin and end?
@CATSWETEL
12. Cycle time? Is there any old way?
Probably at project level or maybe in ops
there will be SLAs. This is a way under
utilized metric. IMO
Frequency
Cycle Time Distribution
@CATSWETEL
16. Cycle time new way….and with PROBABILITY
not a real control chart with control limits based
on sigmas
WTF?!
@CATSWETEL
Cycle Time Scatter Plot
17. Cycle time? Is there any old way?
Probably at project level or maybe in ops
there will be SLAs. This is a way under
utilized metric. IMO
Frequency
Cycle Time Distribution
@CATSWETEL
18. Cycle time new way….and with PROBABILITY
not a real control chart with control limits based
on sigmas
WTF?!
@CATSWETEL
Cycle Time Scatter Plot
19. Cycle time? Is there any old way?
Probably at project level or maybe in ops
there will be SLAs. This is a way under
utilized metric. IMO
Frequency
Cycle Time Distribution
@CATSWETEL
20. But what if it doesn’t look like that? What if
it looks like a camel?
1 2 3 4 5 6 7 8 9 10 11
Multimodal
@CATSWETEL
21. 1 2 3 4 5 6 7 8 9 10 11
Multimodal
Why?
● Different work item types
● Different work flows
● External dependency
● …etc
@CATSWETEL
24. Cycle time new way….and with PROBABILITY
not a real control chart with control limits based
on sigmas
Cycle Time Scatter Plot
probability??
95%
probability
50%
probability
@CATSWETEL
25. Cycle time new way….and with PROBABILITY
not a real control chart with control limits based
on sigmas
Cycle Time Scatter Plot
with throughput, max and min
@CATSWETEL
26. Throughput
Units of value per unit of time
e.g.
stories per sprint
tickets per day
features per month
@CATSWETEL
27. Cycle time new way….and with PROBABILITY
not a real control chart with control limits based
on sigmas
WTF?!
@CATSWETEL
Cycle Time Scatter Plot
28. Cycle time new way….and with PROBABILITY
not a real control chart with control limits based
on sigmas
WTF?!
@CATSWETEL
Cycle Time Scatter Plot
????????????
29. Cycle time new way….and with PROBABILITY
not a real control chart with control limits based
on sigmas
9 9
11
6
4
6 4
10
13
Cycle Time Scatter Plot
throughput
@CATSWETEL
30. Cycle time new way….and with PROBABILITY
not a real control chart with control limits based
on sigmas
9 9
11
6
4
6 4
10
13
Cycle Time Scatter Plot
throughput
@CATSWETEL
31. Cycle time new way….and with PROBABILITY
not a real control chart with control limits based
on sigmas
9 9
11
6
4
6 4
10
13
Cycle Time Scatter Plot
throughput + probability
95%
probability
50%
probability
@CATSWETEL
32. Cycle Time Scatter Plot
95%
probability
50%
probability
30
16
Good for managing customer expectations
Good for CCPM
@CATSWETEL
33. Cycle Time Scatter Plot
95%
probability
50%
probability
30
16
@CATSWETEL
“What’s the likelihood we’ll hit this date?”
“How quickly can you get this to me?”
34. Cycle Time Scatter Plot
95%
probability
50%
probability
30
16
What is the cost of more certainty?
@CATSWETEL
38. Predictive metrics!
Is there such a thing common in agile teams today?
Ummmm not really
DONE
REVIEW
DOING
TO DO
TUE WED THU FRI MON TUE WED
@CATSWETEL
Leading Indicator: Cumulative Flow Diagram
41. Little’s Law
average # of items in
a system
average
arrival rate
average time spent
in the system= *
average time spent
in the system =
average # of items in
a system /
average
throughput
@CATSWETEL
42. Little’s Law
average # of items in
a system
average
arrival rate
average time spent
in the system= *
average time spent
in the system =
average # of items in
a system /
average
throughput
@CATSWETEL
43. Little’s Law
average # of items in
a system
average
arrival rate
average time spent
in the system= *
average time spent
in the system =
average # of items in
a system /
average
throughput
@CATSWETEL
“Every time you violate an assumption of Little’s Law
your process becomes less predictable.”
--Dan Vacanti in Actionable Agile Metrics
44. Data is all about
stories and
relationships
@CATSWETEL
45. The (useful) Agile metrics you might not
know:
Cycle time, throughput, WIP
Know the bounds (time and space)
Different views change and often enrich
the story
The value of metrics is in the
relationships