AWB - 08 - Agile Metrics

273 Agile White Book – AXA Emerging Markets EMEA-LATAM
Chapter 8
AGILE METRICS
V1.0

Contents
DIAGRAM........................................................................................................................................................................................................ 276
THE METRICS DILEMMA ................................................................................................................................................................................... 277
PRINCIPLES OF AGILE METRICS ........................................................................................................................................................................... 279
THE METRICS ONION............................................................................................................................................................281
SCOPE DEFINITION ........................................................................................................................................................................................... 282
CREATE METRICS AT TEAM LEVEL ...........................................................................................................................................282
CREATE METRICS AT PRODUCT LEVEL ......................................................................................................................................286
GET BALANCED METRICS.......................................................................................................................................................289
HOW TO COLLECT INDICATORS ......................................................................................................................................................................... 290
MAKE THE BEST POSSIBLE DECISIONS ......................................................................................................................................291
GET THE RIGHT TOOLS TO MEASURE........................................................................................................................................292
THE EXPECTED OUTCOME ................................................................................................................................................................................ 294
TAKE AWAY.................................................................................................................................................................................................. 295
CHECKLIST 8.1.................................................................................................................................................................................................. 296

Agile metrics
Agile metrics focus on getting an understandable
set of indicators to support a clear business goal
while reinforcing a positive feedback loop towards
people´s behaviours and company processes.
The primary metric in Agile is piece of working
software that fulfils the client expectations.

Diagram
AGILE METRICS
I know when to use metrics.
I learnt some typical metrics in Agile.
BENEFITS
OF USING
METRICS
LEARN THE
METRICS
PRINCIPLES
I know the techniques to gather indicators.
KNOW HOW TO
CREATE METRICS
- At Team level
- At product level
- Balanced indicators
LEARN HOW TO
COLLECT
INDICATORS

People generally measure something with
the purpose of:
 Making sure they are on track.
 Keeping impediments and risks
under control.
 Checking that the current system is
not deteriorating.
THE METRICS dilemma
Metrics are generally used to make better decisions, and in that way, people tend to add new
indicators when they can´t figure out an answer. Following that way of thinking, more metrics
mean wiser decisions. The feeling that not measuring something leads to deterioration and lack
of insights is implicit in there.
While these common-sense assumptions are also valid in Agile, we place a different focus in
order to make sure that we avoid some problems.
These are some of the issues we try to avoid:
 Measuring things out of context.
 Evaluating indicators that are the result of an effect instead of analysing its root-
cause.
 Inciting inadequate behaviours or putting pressure on people.
 Making too many assumptions with little information or no facts.
 Analysing out-of-date data or not fully understanding the scope of the values presented.

Always use the rule of the “simplest set of
measures that can possibly work”, where no
other metric can satisfy the need.
THE METRICS dilemma
These five issues can add instability into the system and create a vicious circle that re-enforces the
need for new indicators. In order to break that, you need to make sure that all indicators
provide a clear value while keeping people away from adding processes which don’t
support value creation or continuous improvement.

PRINCIPLES of Agile metrics
The first objective of Agile metrics is to enable people to make better decisions. As Agile is
based on the fact that change is welcomed, such values should be adopted as a reflexion of the
reality instead of being an out-dated picture.
There are 3 areas that Agile focuses on:
1. Understanding why we need an indicator, which goal it supports and its benefits.
It should be clear for people what a specific indicator means and which assumptions
were made in order to elaborate it.
2. Supporting the continuous improvement.
They should support a continuous improvement process to help adapt practices and be a
topic of discussion during Retrospectives.
3. Reinforcing positive behaviours.
They should always reinforce positive behaviours in order not to create attitudes that
feed the wrong values into the system (i.e. “make up” some numbers to create a better
but unreal perception).

As you can see, adding an incorrect metric or in the wrong way can seriously impact Teams
and processes. For example, creating a metric that measures how many lines of code each
person writes per week could end up with disastrous effects.

THE METRICS ONION
We use the Metrics Onion to conceptualise all metrics as a system composed by interrelated
parts that cooperate with processes instead of being just an isolated group of indicators. The onion
has five-stages that help emphasize the right attributes at the time of defining the new metric.
An indicator should satisfy the metrics principles, have a defined and clear Scope, easily
identify a need, get an answer about how and what it can improve in the system and finally
establish the best way to get the data.
As a characteristic, good metrics always affirm and reinforce Agile Principles and are an
outcome from a Retrospective.

SCOPE definition
The first thing to be defined is the Scope for a particular indicator. There can be many areas to
measure but, in this chapter, we will group all of them into two main groups:
1. Related to the Team
2. The ones that make more business sense (product level).
CREATE METRICS AT TEAM LEVEL
When establishing metrics at Team level, it is important to have in mind the processes impacted
and to be improved. They are a main tool to help the Team learn and adapt their practices.
We, Agile Teams, typically need metrics to answer questions like these:
 How much value have we delivered to business?
 How much value can the Team deliver within the remaining time?
 How much work can we take?
 Are we on track to deliver what we forecasted for the Sprint?

SCOPE definition
The team Agile Star assessment can help analyse different aspects of the Team’s “health”.
This analysis can be also more specific and target different roles as follows:
Development Team Product Owner

SCOPE definition
There is also a second type of metrics that Team creates to make sure Business understands
the current situation and the way they work. These are some of the indicators we generally
recommend (but not limited to):
Benefit Expected result Metric used
Client
Expectation
Target tools and interactions to
improve people competences
(Skills, way they work, etc.).
Defects reported, Defects measure
Efficiency Smaller user stories and better
Acceptance User Criteria
User Stories started but not finished
Predictability Reduced variability, better
planning for Potential Product
Increments
Team velocity vs. Capacity measure
Product
experience
Better client experience,
Increased Quality
Support calls, defects measure
Stable product Better cadence, less pressure on
team, better cross-functional
team
% of tests automated, % test in Continuous Integration,
skills learnt and shared measure, Unit test coverage
Time to
market
Increase Business Value, Reduce
cost per feature
No. architectural refactors, Technical Debt Scoring measure
Value Delivery Focus on techniques to increase
requests (i.e. eXtreme
Programming techniques)
Requests per Sprint, Requests Resolved per Sprint measure
Effort Better time to market, improved
planning, less variability on
estimates
Extra work discover during the Sprint measure (i.e.
technical debt, documentation, database rollback scripts,
system testing, etc.)
Business
Alignment
Increase team understanding of
goals and vision, better visibility
and improved communication
with P.O.
% goal achieved, % User Stories accepted by product
Owner, Product Mapping Changes metric.

SCOPE definition
You should end up seeing some charts like the following ones:
Never forget that Business Value can be impacted by a large number of factors and any
approach to be taken has to be analysed in the context of the whole ecosystem.
Throughput
Average Lead Time

Typically people from business in an Agile environment
need metrics to answer questions like these:
 What is this really measuring?
 Is this supporting our goal?
 Who is the metric for?
 Who should see it?
 What behaviours risks will this drive?
 Are we measuring at the right level?
SCOPE definition
CREATE METRICS AT PRODUCT LEVEL
At business level, you still keep the focus on the continuous improvement but will need a more
comprehensive set of Key Progress indicators; they give everyone an idea of internal and
external progress and help align expectations and determine business outcomes.
You should see something like this:
Business Value Points
Contracts

SCOPE definition
This is a list of different indicators we recommend and see in the Agile world:
Goal/Business
Reason
Benefit Expected result Metric Used
I want to sell
more cars…
I want better
client retention
I want to
introduce new
products and be
faster in the
market
Agility
Continuous improvement in team and
program measures
Team/company assessment,
release predictability
measure
Value Delivery Target right features Accumulated ROI measure,
Features Scoring measure
(value points)
Client Expectations More satisfaction and understanding
of client needs
Features and level of
satisfaction measure, % goal
achieved measure
Customer Satisfaction Improve customer score and better
relationship
Client score Survey
Efficiency Reduce average requirement cycle
time
Reduce average requirement
cycle time
Predictability Inspect and adapt & learn more
about the client behaviour
Project Deviation measure
Productivity Better retention, more innovation Motivation measure
Time to Market More frequent releases Number of releases per
period of time
Innovation Lead innovation (really low scores per
release means low innovation)
Features Scoring measure
(value points)

SCOPE definition
And never forget that indicators should always support a strategic plan and goal. A variation in
an indicator is never isolated but a part of a whole. Business metrics should also follow the
metrics principles.
Remember that the most important concept in Agile is creating as much Business Value as
possible; therefore most of the indicators pivot around this idea. Agile Metrics are used to deliver
better software and help people know where they are and what else they need.

SCOPE definition
GET BALANCED METRICS
Having a balanced set of metrics is important so that actions to achieve/improve do not
impact other metrics. Note that when you choose to measure a person or team in a certain
way, their actions can be conditioned in excess to meet that goal and also neglect other
important aspects such as quality, costs, risks, sustainability, etc.
The following are some possible metrics grouped in order to help to create a balanced scorecard:
Customer Expectations
 User satisfaction
 Value or cumulative ROI
Cost
 Incurred cost and deviation
 Budget
 Etc.
Productivity
 Lead Time (from Idea to Market)
 Release cadence
 Completed requirements
 Flow efficiency (value time or points or
requirements/ elapsed time)
External Quality
 Hours corrective vs. Total
 Total of defects in production
 Defect density (by US, points, KLOC,
etc.)
Internal Quality
 Product Complexity
 Test coverage
Team
 Teamwork
 Motivation
 Agile Team Maturity
 Agile Management Maturity
 Attrition rate

HOW TO COLLECT indicators
We are always looking for metrics that are easy to collect and don't become an impediment to
the people gathering them or need to spend more time than the process itself.
These are some of my suggestions:
 Measure business outcomes instead of activity (e.g. value delivered vs. number of
tasks completed)
 Look at trends, not just a single point in time.
 Encourage whole team results over individual results.
 Think of how the measurement will improve the way to do things.
 Evaluate the impact in the company when adding new indicators.
Remember that an Agile metric is a real opportunity to give visibility, establish a
conversation and start a new improvement initiative.
Find below some advice you can follow when trying to measure things in Agile:
 Objectively evaluate the outcome of the project/service to make wiser decisions.
 Identify trends, to act on them and see if the actions produce an improvement and a
sustained impact.
 You can introduce a new metric at a given time if it is needed just for specific
situations, but remove it once the issue is fixed.
 Have a balanced set of metrics, so that actions to achieve/improve a metric do not
worsen the results in others.
 Establish a small set of metrics, so that everyone knows their current value, context
and purpose, and are non-intrusive.
 Perform cause-effect analysis of the metrics´ results and their relationships, so that
the actions taken truly help solve the problem.

MAKE THE BEST POSSIBLE DECISIONS
In Agile, we try to diminish assumptions as they produce a false sense of safety and
increase uncertainty/risk. For that reason, some indicators are not created to get a closer look
at reality but to validate an emerging hypothesis. Many Product Backlog Items are treated
as assumptions (or hypotheses) that must be validated by rapid experimentation in the
marketplace. The aim is to eliminate wasteful practices, increase value and diminish risk.
This process is generally performed during the Need stage in the Agile Onion. In these cases,
temporary indicators are created in order to make faster and wiser business decisions while
keeping assumptions and impact low.

GET THE RIGHT TOOLS TO MEASURE
Agile adds a broad number of tools that can be quantified and many relative metrics that can
help track and understand how much Business Value we deliver to our clients or company.
With this data we shape the whole vision.
These are some techniques that work for me in order to understand the ecosystem:
- Know how important a feature is for my client by using Business Story Points.
- Know how much a feature is worth for my client by using Kano Survey Model
(excitement, performance, basic needs).
- Establish how important a requirement is by applying MoSCow (Must, Should, Could,
Won’t).
- Know how motivated a Team is using Niko Niko Mood Tracking (Body metrics, mood
metrics, etc.), etc.
For example, you would like to know the amount of discovered work during a Sprint using Story
Points as it can affect your estimates, people’s morale, etc.

An additional example can be Business Value Points; BVP is a unit that allows companies to
measure what the Business Value of an expected set of features is. This scoring is typically
done by Stakeholders/Product Owner when they evaluate requirements. As a general rule, all
Stakeholders should be aware of the practice and participate in defining what is more valuable to
the client, without delegating the whole responsibility to Product Owner.
We suggest using values of 25, 50, 100 and 200 (higher number = higher importance) to score
it. You can think of this value as one or a mix of the followings:
 Client desire (the more clients ask for it, the higher the number)
 Role petition based scoring (CEO 200 points, Regional Manager 50, etc.)
 Client opinions (Kano Model or any other kind of survey)
 Revenue of the feature (the more ROI the higher)
 Etc.
Once BVP is in place, the company knows how much Business Value is achieved in each Sprint,
they can detect a lack of innovation, etc.
Open the checklist “Creating Metrics” to see how to create them step-by-step.

THE EXPECTED outcome
After using some of the techniques learnt, you should expect to get a metric that:
 Is understood by people and the goal it supports.
 Have clear stated benefits.
 Is easy to get.
 Does not impact negatively in other indicators.
 Does not have many assumptions (or a plan to diminish them).
 Clearly identifies which positive behaviours it reinforces.

TAKE AWAY
REMEMBER
 Everyone should fully understand the units to be used and indicators in order to be balanced.
 Don’t forget to extend the Team involvement in the agile process and always strengthen the Agile Principles.
 It is worth teaching Agile Product Management techniques to the Product Owner.
 Encourage automation to reduce variability and measure better.
 Refine metrics in order to identify if the business benefits of the continuous initiative are being achieved.
ADDITIONAL RESOURCES
Business Value Game
Measuring Success, Measuring Value
Moscow Method
Kano Model
Business Value Game
Optimising Value
BENEFITS
 Helps get insight on the changes needed.
 Keeps stakeholders informed and involved in a meaningful manner.
 Helps redirecting priorities in a continuous way.
 Builds a shared understanding and acceptance of what is measured.
 Establishes a discussion to make better decisions.

Creating Metrics
Checklist 8.1
Version 1.0
DATE: __________
Audience and Context
The checklist will be useful for anybody who thinks of establishing adequate metrics to measure
the efficiency and quality of the development team work.

__________________________
Task Comments
Metrics Creation

We made sure that the metric enables people
to make better decisions/better software (we
performed a cause-effect analysis if needed).

There was no other metric to serve the
purpose.

We checked and know a clear answer of why
we need it.

We clearly know its benefits.
 We made sure that the metric supports
continuous improvement.

We made sure that the metric supports
positive behaviour.

It does not negatively impact other
indicators.

Everyone understood the scope of this
metric.

We diminished assumptions when creating
the metric.
 We clearly knew how to get it.

AWB - 08 - Agile Metrics

Empfohlen

Empfohlen

Weitere ähnliche Inhalte

Was ist angesagt?

Was ist angesagt? (20)

Andere mochten auch

Andere mochten auch (20)

Ähnlich wie AWB - 08 - Agile Metrics

Ähnlich wie AWB - 08 - Agile Metrics (20)

Kürzlich hochgeladen

Kürzlich hochgeladen (20)

AWB - 08 - Agile Metrics