Presentation from Scrum Gathering Orlando (April 2016) about value mapping and prioritizing from a customer and business perspective using both Kano and ROI analysis using Agile and Scrum principles - Natalie Warnert
www.nataliewarnert.com
TrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
Houston - We Have a Priority - Natalie Warnert Scrum Gathering Orlando 2016
1. Houston, we have a priority!
Natalie Warnert
Scrum Gathering Orlando #sgfla
April 18, 2016
2. Agenda
• Natalie’s background
• The fight for priority – org levels
• Traditional value measurement
• Activity 1 & debrief
• Customer value measurement
• Activity 2 & debrief
• Bringing it all together
• Questions
5. The fight for priority
• This is more valuable than that!
• Priority can look very different at different
levels of the organization
6. The fight for priority - Portfolio
• Long term roadmaps
• Strategy
• Performance objectives
• Funding allocation
7. The fight for priority - Portfolio
• Long term roadmaps
• Strategy
• Performance objectives
• Funding allocation
• Strength: ROI
• Deficiency: being laser focused
8. The fight for priority - Program
• Roadmap alignment
• Features
– Top down
– Laterally
– Bottom up
• Dependency management
• Funding project/teams
9. The fight for priority - Program
• Roadmap alignment
• Features
– Top down
– Laterally
– Bottom up
• Dependency management
• Funding projects/teams
• Strength: value stream
• Deficiency: trying to find the right balance
10. The fight for priority - Team
• Team roadmap
– Enhancements
– New functionality
– Tech debt
– Defects
– Dependencies
• Constraints
11. The fight for priority - Team
• Team roadmap
– Enhancements
– New functionality
– Tech debt
– Defects
– Dependencies
• Constraints
• Strength: customer factors
• Deficiency: the bigger picture
12. Brainstorming Activity: 5 min
• What issues have you noticed between
priority valuation at different levels of the
organization?
• Who usually ”wins” and “loses” these
discussions?
14. Traditional Value Measurement –
Return on Investment
• ROI
– Net profit/cost of investment = ROI
– OR… how much can we profit from this
investment (percentage)?
– Tangible
15. Traditional Value Measurement –Cost of
Delay
• Cost of Delay – what is the cost of having
NOT done this? [value + urgency + risk
reduction & opportunity enablement]
16. Traditional Value Measurement –Cost of
Delay
• Cost of Delay – what is the cost of having
NOT done this? [value + urgency + risk
reduction & opportunity enablement]
• If we have an equal cost of delay pick the
shortest job first
Feature Cost of Delay Duration
A $2 2
B $2 1
17. Traditional Value Measurement –Cost of
Delay
• Cost of Delay – what is the cost of having NOT
done this? [value + urgency + risk reduction &
opportunity enablement]
• If we have an equal cost of delay pick the
shortest job first
• If we have an equal duration pick highest CoD
Feature Cost of Delay Duration
A $2 2
C $1 2
18. Traditional Value Measurement –Cost of
Delay
• Cost of Delay – what is the cost of having
NOT done this? [value + urgency + risk
reduction/opportunity enablement]
• What if it’s not one or the other?
Feature Cost of Delay Duration
A $2 2
B $2 1
C $1 2
19. Traditional Value Measurement –Cost of
Delay
• Cost of Delay – what is the cost of having
NOT done this?
• Weighted shortest job first (WSJF)
• Implementation and decision to use CD3 =
CoD/Duration
Feature Cost of Delay Duration CD3 = CoD/Duration
A $2 2 1
B $2 1 2
C $1 2 1/2
20. Traditional Value Measurement –Cost of
Delay
• Cost of Delay – what is the cost of having
NOT done this?
• Weighted shortest job first (WSJF)
• Implementation and decision to use CD3 =
CoD/Duration
Feature Cost of Delay Duration CD3 = CoD/Duration
A $2 2 1
B $2 1 2
C $1 2 1/2
21. Activity – 10 minutes
• Prioritize the work items into a program
roadmap based on monetary measures only
• Incorporate Cost of Delay, Duration,
Weighted Shortest Job First, and ROI
• Left = lower value
• Right = higher value
Lower Value Higher Value
22. Activity Debrief
• What was difficult about the activity?
• What were the first few items in your
prioritization order? Last few?
• How realistic is it to get everything done?
• Which scope would likely get cut?
• What was missing from the conversation
about priority and valuation?
• Take a picture of your priority arrangement.
24. Traditional Value Management
• WSJF is important, but not as important as
how we calculate it
• Cost of Delay [value + urgency + risk
reduction/opportunity enablement]
25. Traditional Value Management
• WSJF is important, but not as important as
how we calculate it
• Cost of Delay [value + urgency + risk
reduction/opportunity enablement]
• And duration…
26. Traditional Value Management
• WSJF is important, but not as important as
how we calculate it
• Cost of Delay [value + urgency + risk
reduction/opportunity enablement]
• And duration…
• But it’s an estimation at best
27. Traditional Value Management
• WSJF is important, but not as important as how
we calculate it
• Cost of Delay [value + urgency + risk
reduction/opportunity enablement]
• And duration…
• But it’s an estimation at best
• And when we estimate in a box (especially with
money), we are more often than not WRONG
29. Customer Value Measurement
• But the customer does not care about value as
a measure
• Customer thinks of value as:
val·ue
/ˈvalyo͞o/
Noun
1. the regard that something is held to deserve; the importance, worth, or
usefulness of something.
“this product is of great value”
Synonyms: worth, usefulness, advantage, benefit, gain, good, help, merit
www.google.com
31. Customer Value Measurement
• As a…I want…so that…
• As a customer/user/subscriber…
• I want [certain functionality]
• So that I can do something that is valuable
to me and gain satisfaction by doing it
32. Customer Value Measurement
• As a…I want…so that…
• As a customer/user/subscriber…
• I want [certain functionality]
• So that I can do something that is valuable to me and
gain satisfaction by doing it
– Ease, time-saving
– Unique/new/differentiating
– NOT so that the company can make money off of me
• Intangible
33. Customer Value Measurement
• Customer cost of delay
– What I NEED to do (basic/threshold)
– What I WANT to do (performance)
– What I haven’t thought to do yet
(delight/excite)
36. Customer Value Measurement - Kano
Very Satisfied
Very Dissatisfied
Need FulfilledNeed Unfulfilled
Basic
Performance
37. Customer Value Measurement - Kano
Very Satisfied
Very Dissatisfied
Need FulfilledNeed Unfulfilled
Basic
Performance
Delight
38. Customer Value Measurement - Kano
Very Satisfied
Very Dissatisfied
Need FulfilledNeed Unfulfilled
Time
Basic
Performance
Delight
39. Activity 2 – 10 minutes
• Move feature cards ONLY vertically based on customer value factors
• Discuss which buckets each feature could fit into based on what the customer
values
Lower Value Higher Value
Delight: Don’t know I want
Performance: I WANT
Basic: I NEED
40. Activity 2 – Debrief
• Which features are now the most important
customer value factored in?
• Which features are more likely to be lower
in priority?
43. ROI vs. Kano
(Money vs. Feeling)
Very
Satisfied
Very
Dissatisfied
Need
Unfulfilled Time
Basic
Performance
Delight
ROI (Valuation =
Money)
Kano (Value = Feeling)Vs.
Value
Cost
44. ROI vs. Kano
(Money vs. Feeling)
Very
Satisfied
Very
Dissatisfied
Need
Unfulfilled Time
Basic
Performance
Delight
ROI (Valuation =
Money)
Kano (Value = Feeling)Vs.
Value
Cost
45. ROI vs. Kano
(Money vs. Feeling)
Very
Satisfied
Very
Dissatisfied
Need
Unfulfilled Time
Basic
Performance
Delight
ROI (Valuation =
Money)
Kano (Value = Feeling)Vs.
Value
Cost
46. ROI vs. Kano
(Money vs. Feeling)
Very
Satisfied
Very
Dissatisfied
Need
Unfulfilled Time
Basic
Performance
Delight
ROI (Valuation =
Money)
Kano (Value = Feeling)Vs.
Program (value stream)
Value
Cost
48. Wrap Up
• Value is both a verb and a noun (like Agile)
• Think about making money and the
customer – balance between importance
• Development tier (level) can help influence
decisions
Prevent crashes
Limited resources of time and money.
-Has anyone experienced this type of thing
Stability, security, and flashiness. We work the entire year to build up the site and then at the busiest time turn off a lot of the new functionality to improve performance and prevent issues.
Ended up with a list of 30 critical defects and had to choose which ones we would be allowed to fix and get into the final builds of the freeze (which were pushed off a few extra days). At that point,
how do we justify the value?
defects differently than we looked at new functionality in general.
So we looked at what was important for the freeze – security, stability, executive investment/interest, moron factor (spelling error on cart page)…this was a new way to look at what NEEDS to be done – why can’t it expand to program level roadmaps? Is it always about dollars earned? Dollars saved? No – there are many other things to consider…
We will always have conflict on what is the most important as long as we have finite resources. How often have you had a sprint or PI planned and then something “happens” or someone gets an idea or another business does something we need to keep up with?
There are different goals and motivations driving what should be “one” strategy, but it never is a single strategy.
Product owner, other product owners, tech leads, product manager, architects, executives
Chicken and egg question? Do we make money and invest it to make the customer happy or do we make the customer happy and therefore we make money? Different areas in the development stack look at value differently.
Value return to the company
Where is the money coming from and going to--- disconnect with what is actually being built on the ground/team level
Where should I invest this money? What features should I buy from which teams? What product line is doing well and should we invest in? Those lines that are performing the best and are probably focusing on their users so that makes for a difficult decision – feed those teams or invest in teams that are not doing as well? Finite resources drive the need to make these decisions.
Laser focus and understanding the lower level issues that are contributing to some of the priority discussion we see at a higher level. Constraints that teams feel are not felt at the portfolio level – they are usually extrapolated and are harder to get to the root issue (example? Dependencies between teams, arch/infra constraints, devops and RM issues)
They do not understand how their actions impact and disrupt at the program and team level – reprioritizing of features,
pet projects,
needing to “see” to believe, not attending demos and then later trying to incorporate feedback – all of this delays value delivery.
Level does not exist in most companies – can utilize collective personas (JTBD and user journeys). They can see the picture at multiple levels and understand what is driving value and what is not but often get stuck resolving most disagreements and managing expectations
Discuss – ORV wants to know where their money will be spent and how much of a feature they will be funding and when they will see it
-PG & A does not put in enough money but their features need to be done first from a program and product dependency perspective.
This is often shown at the program level and directly affects team and portfolio, program is stuck in the middle
Budgeting and funding decisions are leading to poor directional decisions based on short term returns and a less than ideal user experience
Poor budgeting models based on funding projects v. teams can contribute to this issue of building the wrong thing and not managing expectations
Can understand and see how things are implemented
the value they could bring as well as
managing the expectations of many different levels in the organization.
It is difficult to balance expectations, budget, and value, though as things are constantly changing.
Teams are at the close level. They know what is being built and how it will affect the customer (or how it will not). They know how it will affect themselves.
Often teams feel the constraints very closely because they feel the need to deliver everything because priority decisions cannot happen. Or when priorities change there is a lot of throw away work.
How many times is the solution “hacked” together? How many times does it “work on my local machine?” Does it work for one area and not integrate with other solutions or with other things? How many firedrills when another team did something similar or is thinking about doing something similar?
Can translate the value to persona – sometimes cannot see the entire picture and how it relates to the business
Are they discussions at all?
3:55
Noun: regard that something is held to deserve, worth, usefulness
Verb: valued at – estimate the monetary worth
Run the business
Tangible makes us feel safe – we can assign a number to it, even if it is a wrong number
How often have you heard “just put a number on it?” How often have you said it? How often have you needed to make some numbers match or look profitable, successful, efficient, effective
How much will we lose if we wait (opportunity cost)
Value – cust and business
Urgency – is there a deadline? Regulation? Event?
RR – is this something that is risky? Competition? Some event (e.g. Holiday)
OE – competition? Is there a window? (First to have something, need to match or beat someone else – also urgency).
B
A
WSJF is the implementation of CD3 and the decision to use
SAFe and Lean implementation
Assuming WIP = 1
Pick highest CD3 or WSJF
4:05 activity start
4:15 activity complete
4:20
Should there be one “right answer”? Probably not. Should make some of the features the same or close WSJF and CoD? – check on this?
Gives us the most value for the least effort to:
Gives our customer the most value for the least effort…
Tangible = blame, we feel wrong, are we punished? What about how the customer feels?
Often the components are forgotten about for CoD
Look to do something else with the greater than/less than sign – maybe add descriptor in front of/behind value and then effort?
Tangible = blame, we feel wrong, are we punished? What about how the customer feels?
What is lost…? The actual reality of what could happen
Gives us the most value for the least effort to:
Gives our customer the most value for the least effort…
Look to do something else with the greater than/less than sign – maybe add descriptor in front of/behind value and then effort?
Tangible = blame, we feel wrong, are we punished? What about how the customer feels?
Time
Risk
Scope
Value
What is lost…?
Gives us the most value for the least effort to:
Gives our customer the most value for the least effort…
Look to do something else with the greater than/less than sign – maybe add descriptor in front of/behind value and then effort?
Tangible = blame, we feel wrong, are we punished? What about how the customer feels?
But it feels good because it’s tangible = blame, we feel wrong, are we punished?
This estimate was wrong, we were wrong, we spent too much, took too much time, didn’t get the results.
What about how the customer feels?
Value to the company as a measure. In most cases…
The ”so that” is not usually so that the company I’m paying can make money. It’s so I can do something. The value is me doing something – the value is a thing, not an amount/measure.
Though that is usually what happens because that is how the company stays in business. But it is not the first thought of the customer.
Intangible and hard to assign a number to…
Retail example…
Here is where I tell the story about the auto save…Moron factor
Tech debt also fits here – the customer doesn’t notice if we do, but they notice if we don’t
Needs to balance because we can only do so much before it becomes a loss and levels out
What is a performance feature? Filter, ISPU
Apple Pay – maybe this doesn’t make us a ton of money yet, but if differentiates us
Time: it becomes commonplace after time and delights/performance turn into base expectations…search, save example, pay with something other than a credit card (before paypal you needed to have a credit card – teenager story when I had to beg to use a credit card to buy things online)
Add diagram of where factors are here…
Explain why I valuated these in this way. Delights – can be a differentiator or can flop, but if you don’t have basic needs met they mean nothing.
Start at 4:30
End at 4:40
4:45 move on
What order were they actually implemented in?
ISPU, Save, Save for Later, filters
Required Accessories
ApplePay
Gift packaging
Responsive
What happens when we only look at one? We miss the whole picture (portfolio and team)
How do I talk about ROI graph? Is it necessary? Should probably label axis…
But what about those damn executive pet projects?? How do we show this to folks that are not into cards and valuation in this way?
Where do I add relative estimation into this?
Add image here – label with what the lines are called… Eric
But what about those damn executive pet projects?? How do we show this to folks that are not into cards and valuation in this way?
Where do I add relative estimation into this?
Add image here – label with what the lines are called… Eric
But what about those damn executive pet projects?? How do we show this to folks that are not into cards and valuation in this way?
Where do I add relative estimation into this?
Add image here – label with what the lines are called… Eric
But what about those damn executive pet projects?? How do we show this to folks that are not into cards and valuation in this way?
Where do I add relative estimation into this?
Bring this back around – needed to look at not just what is going to make us money, but also what customers want/need and what will keep the site up and stable. It needs to balance out and by admitting we cannot do everything we are making one step toward progress.
Fixed about the top 8 defects and site did not go down.