http://sched.co/9hgs Metrics are useful tools, yet dangerous. Measuring the wrong thing can steer us in the wrong direction without us even noticing. And even the right metric can cause stagnation when we need to progress to the next challenge. Inspired by the Agile Fluency™ Model, in his talk we discuss how the stage of fluency determines what types of metrics you should focus on. And how we need to change that focus as we progress to a different stage. We’ll discuss how you can start with the basic metrics from Scrum or Kanban, such as velocity and cycle time. Then you can progress through to the type of delivery focused metrics from Continuous Delivery. Finally, we arrive at business focused metrics appropriate for product teams.

1. Metrics to Guide
changing measures along the way
Wouter Lagerweij
@wouterla
Karel Boekhout
@karelboekhout
#XP2017

2. Goodhart’s Law
https://en.wikipedia.org/wiki/Goodhart's_law
“When a measure becomes a target,
It ceases to be a good measure”

3. No metric without a goal
https://www.flickr.com/photos/rene-germany/2151312845/in/photostream/
https://www.flickr.com/photos/wwarby/3296379139

4. Do you want to be predictable?
Or fast?

5. Actionable
Metrics

6. Agile Fluency™ Model
● By Diane Larsen & James Shore (2012)
● Fluency is how a team develops software
when it’s under pressure
● NOT a maturity model!
● ★★★★ not relevant for every organization

7. Start: Building Code

9. Zone 1: Focus on Value

11. Zone 2: Deliver Value

13. Zone 3: Optimize Value

15. Zone 4: Optimize for Systems

17. Agile Fluency: Team Metrics
⭐ -> Focus: Working together, team and portfolio priorities
⭐⭐ -> Capability: Delivery speed and quality
⭐⭐⭐ -> Value: Product success
⭐⭐⭐⭐ -> Value: Organizational success

18. ⭐ ‘Basic Agile’ Goal: Focus
● WIP
● Cycle time

19. ⭐ Work in Progress
Metric: WIP
Distance: Direct
Scale: Individual, Team, Department, Organisation
Actionable: Yes
Feedback Cycle: From minutes to years
Team measurements:
● How many features is any
team working on?
● How many projects?
Practices:
● Scrum (Sprints)
● Kanban (WIP limits)
● King/Servant pattern
● Sprint Goal
● Velocity
● Pairing
● Mob programming
“Stop Starting, Start Finishing”

20. ⭐ Work in Progress
Metric: WIP
Distance: Direct
Scale: Individual, Team, Department, Organisation
Actionable: Yes
Feedback Cycle: From minutes to year
Organisational measurements:
● How many projects are active?
● How many are planned?
(portfolio)
● How long is the backlog?
Practices:
● Project WIP
● Iterative development
● Agile Portfolio Management
(⭐⭐)
● Product focus over project
focus (⭐⭐⭐)
● Set-based design (⭐⭐)
“Stop Starting, Start Finishing”

21. ⭐ Goal: Focus
Metric: Cycle time
Distance: (in)direct
Scale: Team, Department, Organisation
Actionable: Yes
Feedback Cycle: From days to years

22. ⭐ Cycle-time
Metric: Cycle-time
Distance: (In)direct
Scale: Team, Department, Organisation
Actionable: Yes
Feedback Cycle: From days to years
Team measurements:
● How long for one feature from
start of development to
production?
● How long for a project or
product?
Practices:
● Minimizing story size
● Iterative development
● Left-shifting QA
● Optimizing process (remove
handovers, waiting times)
● Increasing capability
(automation, quality, ⭐⭐)

23. ⭐ Cycle-time
Metric: Cycle-time
Distance: (In)direct
Scale: Team, Department, Organisation
Actionable: Yes
Feedback Cycle: From weeks to years
Organisational measurements:
● How long for one feature from
inception to production?
● How long for a project or
product?
Practices:
● Iterative development
● Left-shifting development
● Lean Startup / Lean UX (⭐⭐)
● Product vs project
development (⭐⭐)

24. ⭐⭐ ‘Continuous Delivery’: Capability
● Deployment frequency
“How often do we deploy to production?”
● Lead time for changes
“How much time from developer code check-in to running in production?”
● Mean time to recover
“How long between detection of production issue and resolving it”
● Change failure rate
“How often does a deployment fail?”

25. ⭐⭐ ‘Continuous Delivery’: Capability
https://leanpub.com/measuringcontinuousdeliveryContinuous Delivery book on Amazon

27. ⭐⭐ ‘Continuous Delivery’: Capability
● Deployment frequency
“Twice a year”
● Lead time for changes
● Mean time to recover
● Change failure rate

28. ⭐⭐ ‘Continuous Delivery’: Capability
● Deployment frequency
“Twice a year”
● Lead time for changes
“Three months”
● Mean time to recover
● Change failure rate

29. ⭐⭐ ‘Continuous Delivery’: Capability
● Deployment frequency
“Twice a year”
● Lead time for changes
“One day”
● Mean time to recover
● Change failure rate

31. ⭐⭐⭐ ‘Product Teams’: Value

32. ⭐⭐⭐ ‘Product Teams’: Value
http://ronjeffries.com/xprog/articles/value-is-what-you-like/
“Value is what you like”
-- Ron Jeffries

33. ⭐⭐⭐ ‘Product Teams’: Value
“What is your business model?”
-- Me, repeatedly

34. ⭐⭐⭐ ‘Product Teams’: Value
● External customers: Pirate metrics!
● Internal applications: UX analytics

35. ⭐⭐⭐ ‘Product Teams’: Value
http://500hats.typepad.com/500blogs/2007/06/internet-market.html | The Startup Owner’s Manual

37. ⭐⭐⭐ ‘Product Teams’: Value
http://startitup.co/guides/374/aarrr-startup-metrics

38. Thanks!
Get in touch...
Wouter Lagerweij
@wouterla
wouter@lagerweij.com
Karel Boekhout
@karelboekhout
karel@hedgefields.com

39. Results
Test:
Add two most common
filters
Test:
Offer list of ‘like this’ in detail
page
Test:
Offer extra search term
based on search history
1.5% more searches
1% more detail views
.5% more searches
4% more detail views
5% more searches
2% more detail views