Michael S. McCalla presents an outcome measurement model for assessing agile adoption. The model focuses on measuring strategic objectives and tactical goals at the enterprise, unit, program, team and individual levels. Self-assessments are an important part of the model to gather feedback on agile practices and identify areas for improvement. The desired outcome of agile adoption should be clear and measurable. Both leading and lagging metrics should be tracked to evaluate progress towards outcomes. The model is demonstrated through a case study of a financial services company aiming to improve customer satisfaction through agile.

1. Michael S. McCalla
An Outcome Measurement Model:
Is your Agile Adoption Moving the Needle?
Measure
Strategic
Objectives
Self-
Assess
Tactical
Goals

2. Michael S. McCalla CSP, PSM II, SA, PMI-ACP
• Enterprise Agile Coach
• 10 Years of Experience Assisting Organizations with Agile
Transformation
• Activist in the Agile Community:
• One of the original board members of the Agile Uprising, online
community dedicated to advancing the Agile Mindset
• Data Nerd
• Problem Solving and Product Development is My Passion, Coaching is a
Close Second
• President of Lean Agile Intelligence, an Agile Assessment, Improvement,
and Measurement Platform Assisting Organizations with Agile Adoption
given their Desired Outcomes

3. Poll #1
What are the biggest challenges of adopting agile across your enterprise(aka scaling)?
A. Company philosophy or culture at odds with core agile values
B. Lack of experience with agile methods
C. Lack of management support
D. General resistance to change
E. Inconsistent processes across teams
F. Lack of business/customer/product owner availability
G. Regulatory compliance issue
H. We don’t have any challenges

4. Fundamentals
Source: Ahmed-Sidky

5. Traditional Transformation Approach
Source: Ahmed-Sidky

6. Frustration

7. What We Need to Consider….
Enterprise Culture
Command & Control
Leadership

8. Agile Adoption Challenges
Source: Version One State of Agile 2017
Challenges to Adopting Agile Across the Enterprise/Scaling
Source: 2017 Cprime Scaling Agile Survey

9. Traditional Transformation Approach
Source: Ahmed-Sidky
Implies “agile” is the goal…….

10. Agile is simply “a means to an end”
This movement is about delivering value and creating better business outcomes
Agile is Not the Goal

11. Value in the Form of Outcomes
Value
Customer
Satisfaction
Time to Market
Employee
Satisfaction
ResponsivenessInnovation
Reliability
Predictability

12. Fundamentals
Source: Ahmed-Sidky

13. Taking It Further

14. Taking It Further
Considering Context

15. A Different Approach
Considering Context
The Outcome is the Goal…….
Agree and Align on the Outcome......

16. Measure and Adapt
Considering Context
Is the Agile Practice Adoption helping us achieve the desired outcome?
Measure
Leading Lagging

17. Agile Transformation on the Side
Vision
Mission
Organizational
Strategy &
Objectives
Initiative Portfolio
Programs
IT Agile Transformation

18. Agile Transformation as a Program
Vision
Mission
Organizational
Strategy & Objectives
Initiative Portfolio
Programs
Program:
IT Agile Transformation
Objective:
Improve Productivity

19. Agile Transformation as Part of the Strategy
Vision
Mission
Organizational
Strategy &
Objectives
Agile Transformation
Initiative Portfolio
Programs

20. Poll #2
Can your organization clearly articulate the
desired outcome(s) of its Agile
Transformation and how it is being
measured?
A. Yes & Yes
B. Yes & No
C. No & No
D. No & Yes
Value
Customer
Satisfaction
Time to Market
Employee
Satisfaction
ResponsivenessInnovation
Reliability
Predictability

21. Measure
Strategic
Objectives
Self-Assess
Tactical
Goals
Outcome Driven Improvement & Measurement Model

22. Measure
Strategic
Objectives
Self-Assess
Tactical
Goals
Outcome Driven Improvement & Measurement Model
Enterprise
Unit
Program
Team
Self
For Every Level of Today’s Agile Organization……

23. Measure
Strategic
Objectives
Self-Assess
Tactical
Goals
Enterprise
Unit
Program
Team
Self
Outcome Driven Improvement & Measurement Model: Enterprise Level

24. Real Life Case Study
Just Do It Advisors is mid-tier (5000 people) asset management boutique that has been in
existence for 85 years. They pride themselves on establishing personal relationships with
their clients and utilizing an active money management approach. This strategy has enabled
them to charge higher fees. However, over the past 6 years, market share and revenue have
declined significantly because many of their long standing clients are now deceased and their
children have chosen to go elsewhere. Feedback has been competitors offer a comparable
service at a cheaper price, and provide more transparency of funds and reporting via online
portals.
They have a traditional silo-ed organization structure in which the IT organization has always
been seen as a cost center who keeps the lights on, and serves multiple business units.
Recently, they have been asked to deliver new applications, but feedback was the deliveries
did not meet the need or it took to long. They have experienced with Scrum, but it is the
typical “Scrum-But.”

25. Measure
Strategic
Objectives
Self-
Assess
Tactical
Goals
• What outcome are we looking to achieve given our context?
Increased Customer Satisfaction with Innovative Products
• What do we need to focus on to get us there?
Tight feedback loops to ensure we’re delivering the right things
• What are some of the challenges?
Siloed Environment & Lack of Collaboration
• Do we have clear priorities?
New Product Development Trump Everything Else
• What metrics can we use to measure our progress?
Customer Attrition, Referrals, Revenue
Enterprise
Unit
Program
Team
Self

26. • What practices will help us achieve our desired outcomes?
1. Dedicated “True” Product Owners
2. Stable Cross Functional Teams
3. Customer Engagement
4. Experiments and Prototyping
5. Lean Budgets
• How are we performing in the practices that will lead to better
outcomes?
1. Dedicated “True” Product Owners - Dev Managers
2. Stable Cross Functional Teams - Temp Project Teams
3. Customer Engagement - None
4. Experiments and Prototyping - Governance Constraints
5. Lean Budgets - Annual
• Who is best to tell us this information?
The people doing the work in the form of self-assessments
Measure
Strategic
Objectives
Self-Assess
Tactical
Goals
Enterprise
Unit
Program
Team
Self

27. Why Self-Assessments?
• Easy and Tangible Improvement Model for Teams to Grasp
• Self Sustaining – Something that can be done by themselves when coaches
aren’t around or consultants walk out the door
• Self-Assessments
• Invitation to Improve!
• Everyone has a voice, and stake in the game
• Empowerment to self organize and get to the next level of team
performance
• Provides Holistic View of Agility Across the Enterprise
• Aggregated Results all the way up to the Enterprise Level
• Data Analytics to identify patterns preventing Business Agility such as
coaching needs, investment opportunities, and organizations constraints

28. • What short term tactical goals can we set to help achieve our
outcome?
1. Identify Product Owners from the Business
2. Get New Product Owners Training
3. Create 2 Cross Functional Teams for New Prod Dev
• Do we have any impediments that are out of control preventing us
from improving in the practice?
1. Product Owner Empowerment
• What metrics can we use to measure our progress?
1. Leading: # of CF Teams, % of Trained POs, New vs.
Existing Prod Dev %, Net Promoter Score (NPS)
2. Lagging: Customer Attrition, Referral, Revenue
Measure
Strategic
Objectives
Self-Assess
Tactical
Goals
Enterprise
Unit
Program
Team
Self

29. • Did we meet our tactical goals and do what we said we were
going to do?
Look at Leading Indicators
• Is it helping us move the needle?
Look at Lagging Indicators
• Do we have organization constraints that are slowing us down?
This is an opportunity to gather insights and determine
new objectives for another cycle of improvement
• Do we need to continue improving in these practices, or should
we try some others?
Has our hypothesis proved to be true?
Measure
Strategic
Objectives
Self-Assess
Tactical
Goals
Enterprise
Unit
Program
Team
Self

30. Measure: Tracking Progress of Leading Indicators Measure
Strategic
Objectives
Self-Assess
Tactical
Goals
Track leading indicators to see if the team is progressing towards the goal

31. Measure: Track Progress of Lagging Indicators Measure
Strategic
Objectives
Self-Assess
Tactical
Goals
Track lagging indicators to see if the tactical improvement is leading to desired outcomes

32. Measure: Track Organization Patterns Measure
Strategic
Objectives
Self-Assess
Tactical
Goals
Look at patterns across the organization to identify possible organization constraints

33. Measure: Putting it All Together
Measure
Strategic
Objectives
Self-Assess
Tactical
Goals
Practice Improvement

34. Did we meet our tactical goals and do what we said?
Is it helping us achieve the desired outcome?
Do we have organization constraints slowing
us down?
Outcome Driven Improvement and Measurement Model Canvas
What is important to this team right now?
What outcome are we looking to achieve?
What are some of the teams' challenges?
Measure
Strategic
Objectives
Self-
Assess
Tactical
Goals
What short term tactical goals can we set in
those practices to help us achieve our desired
outcomes?
Do we have any impediments out of our control
preventing us from achieving those tactical
goals?
What metrics, both leading and lagging,
can we use to measure our progress?
What Agile and Lean practices will help us
achieve those outcomes?
Where could we go to better understand the practices
and if we are doing them correctly?
Who is best to assess our performance on them?

35. Takeaways
• Agile Transformations should not be looked at as just another program,
rather the catalyst to achieve the organization’s strategy and objectives
• Agile is not the goal; start with outcomes and context and gain
alignment
• Define the form of value you are looking to achieve, and measure it.
• Utilize self-assessments to drive improvement in the practices that will
lead to your desired business outcomes
Vision
Mission
Organizational Strategy &
Objectives
Agile Transformation
Initiative Portfolio
Programs

36. Contact
Website: www.leanagileintelligence.com
Email:
mmccalla@leanagileintelligence.com
Twitter: @MSMcCalla_1
LinkedIn:
https://www.linkedin.com/in/michaelsm
ccalla/
Website: www.cprime.com
Email: marketing@cprime.com
Twitter: @cPrimeInc
LinkedIn:
https://www.linkedin.com/in/cprime-inc

37. Appendix

38. Tactical Goals: Determine Leading Indicators
• Team and groups should proactively pick the metrics that will help them track their progress towards
their desired outcomes so they can baseline and incrementally measure progress
• Identify Leading Indicators that will help measure if you are on track to achieve your tactical goal
Measure
Strategic
Objectives
Self-
Assess
Tactical
Goals
Time to Market Customer
Satisfaction
Employee
Satisfaction
Innovation Responsiveness Reliability Predictability
Throughput
(Velocity)
NPS Happiness Index # of New Products
Released
Product/ Project
Lead Time
Test Coverage Story Point
Attainment
Build Time Value Points Team Health
Assessment
Bug Fixing vs New
Product
Development %
Feature Lead Time Builds Per Day Velocity Variation
Deployment Lead
Time
Cost of Delay Code Complexity
New Environment
Automation
Failed
Deployments

39. Measure: Determine Lagging Indicators
• Team and groups should proactively pick the metrics that will help them track their progress towards
their desired outcomes so they can baseline and incrementally measure progress
• Identify Lagging Indicators to see if your tactical improvements are moving the needle
Measure
Strategic
Objectives
Self-
Assess
Tactical
Goals
Time to Market Customer
Satisfaction
Employee
Satisfaction
Innovation Responsiveness Reliability Predictability
Number of
Releases
ROI Attrition New Customer % New
Project/Product
Cycle Time
Escaped Defects in
Prod
On Time Delivery
Rate
Project/Product
Cycle Time
Revenue Yearly Surveys New Markets
Entered
New Feature Cycle
Time
Availability
Feature Cycle Time Profit Number of
Production
Deployments
# of Prod
Deployments