A snapshot of important actionable metrics to be employed in full life cycle of lean product development to ensure the "right" product being developed.

1. Huong Ngo 2016
Actionable Metrics in
Lean Product Development

2. Agenda
Introduction
• Motivation
• Objectives
• Background
Actionable Metrics Framework
• Lean Product Development Life
Cycle
• Idea Selection
• Business Model
• Discovery- Problem/Solution Fit
• MVP- Product/Market Fit
• Scale

3. High failure rate in software product development
• 30% projects failed
• 92% start-ups failed within the first three years
Main Reasons
• Wrong products having been developed
• Failed to realise benefits
Required Solutions
• Validate business viability of the product early in the process
• Deliver the product early and continuously to enable quick learning loops
• Employ actionable metrics for validation and learning purpose
Motivation

4. Metrics are required for measurement and management
• If you can’t measure it, you can’t manage it – Peter Ducker
Wrong metrics might cause severe negative impact
• Give false motivation
• Lost focus
• Reduce quality
• Reduce staff morale
Important to employ “right” metrics having 3 “A” attributes:
• Actionable
• Auditable
• Accessible
Why Actionable Metrics?

5. • Invest to “right” product
• Optimise the Products
• marketability
• profitability
• scalability
• development efficiency
Design actionable
metrics
framework for
full life cycles of
Lean product
development
Objectives

6. Background
Lean Metrics Frameworks
• Dave McClure’s AARRR
• Eric Rie’s Three Engines of Growth
• Alistair Croll & Benjamin Yoskovitz’s Lean Analytics Stages And Gates
Analysis Techniques
• Cohort Analysis
• A/B Testing
Web Analytics

7. Ideas
Business
Model
Discovery
MVP
Scale
Lean Product Development Life Cycles
Includes stages in the whole life cycle of a product development, from cradle to grave

8. Fuel for innovation and trigger for product development/ improvement
Management framework for collection, handling, selection and distribution of ideas
Selection Metrics
•Idea Selection Criteria
•Align to core business?
•Align to business strategy?
•Create new cient value?
•Marketplace win?
•Technically and/or financially feasible?
•Weighted Score Model:
•Give a weight (0-1) to each criteria based on their importance
•Define Minimum Success Score
Idea Management and Selection
Criteria Weight Score
Alignment to core business 0.2 5
Alignment to business strategy 0.2 5
Create new values to clients 0.2 3
High possibility to win
marketplace
0.2 3
Technically possible 0.1 4
Financially possible 0.1 4
Total Score 4
Minimum Success Score 4.5
Shortlisted No

9. Business modelling is critical for product development
• A product is not only a solution to resolve a problem but also encapsulates the whole business model
to deliver and capture values
Traditional business case and financial metrics rely on assumptions and estimation.
• Thus they are expensive to develop but not suitable for products with high uncertainties
Lean Canvas provides full picture of business model and quick to develop
• Allowing time and resources to experiment
Business Model
“A business model is a story about how an organisation creates, delivers and captures value” – Saul Kaplan.

10. Valid
Problem
• Enough people care? How many % respondents have the problem?
• Painful enough? How many % respondents have tried or want to resolve the problem?
Valid
Solution
• How many % respondents are willing to sign up?
• How many % respondents show interests to the product by asking more details and
suggest features?
Valid Value
• How many % respondents are willing to pay at targeted price and above for the solution?
• How many % respondents are willing to pay some amount for the solution?
Discovery- Problem/Solution Fit

11. Discovery- Problem/Solution Fit (con’t)
Respondents- 100%
% Have Problem – 70%
% Try/Want to Resolve Problem- 60%
% Want to sign up to the solution-
40%
% Willing to pay at
minimum price – 30%

12. MVP- Minimum Viable Product
 A slice of the system which has just enough features to make the
product feasible, valuable, usable and delightful to its customers

13.  Validate business viability of the product before committing to scale.
 Validate MVP in 3 areas: Stickiness, Virality and Revenue
MVP Validation- Product/Market Fit
MVP Scale
Stickiness
Virality
Revenue
Known customer segment

14. Clearly define who is considered as customer, returning customer, churning customer.
Choose a period for your customer stickiness measurement
Define which metrics to measure and its secondary metrics
• Retention Rate/Churn Rate
• 𝑐ℎ𝑢𝑟𝑛 𝑟𝑎𝑡𝑒 =
𝑛𝑢𝑚𝑏𝑒𝑟 𝑜𝑓 𝑐ℎ𝑢𝑟𝑛𝑖𝑛𝑔 𝑐𝑢𝑠𝑡𝑜𝑚𝑒𝑟𝑠 𝑑𝑢𝑟𝑖𝑛𝑔 𝑝𝑒𝑟𝑖𝑜𝑑
𝑛𝑢𝑚𝑏𝑒𝑟 𝑜𝑓 𝑐𝑢𝑠𝑡𝑜𝑚𝑒𝑟𝑠 𝑎𝑡 𝑡ℎ𝑒 𝑏𝑒𝑔𝑖𝑛𝑖𝑛𝑔 𝑜𝑓 𝑝𝑒𝑟𝑖𝑜𝑑
𝑥100(%)
• Drill down report to each individual
• Usage patterns in site/app
• Returning channel frequency
• Identify customer retention patterns, customer churn patterns
MVP- Stickiness
The goal is to prove (or disprove) that the product is interesting and value enough to keep customers
coming back and what you can do differently to improve customer engagement.

15. Traditional financial statements - do not give indicators on achievability of sustainable
growth revenue model
Monetisation model consists of three elements
•From whom
•How
•How much
Define what is a customer in your business
Funnel Analysis
MVP- Revenue
The goal is to validate and achieve a profitable, sustainable monetisation model before committing to Scale

16. Customer Acquisition Cost
• 𝐶𝐴𝐶 =
𝑇𝑜𝑡𝑎𝑙 𝑠𝑎𝑙𝑒𝑠 𝑎𝑛𝑑 𝑚𝑎𝑟𝑘𝑒𝑡𝑖𝑛𝑔 𝑐𝑜𝑠𝑡 𝑑𝑢𝑟𝑖𝑛𝑔 𝑝𝑒𝑟𝑖𝑜𝑑
𝑇𝑜𝑡𝑎𝑙 𝑛𝑢𝑚𝑏𝑒𝑟 𝑜𝑓 𝑛𝑒𝑤 𝑐𝑢𝑠𝑡𝑜𝑚𝑒𝑟𝑠 𝑎𝑐𝑞𝑢𝑖𝑟𝑒𝑑 𝑑𝑢𝑟𝑖𝑛𝑔 𝑝𝑒𝑟𝑖𝑜𝑑
Customer Life Time Value
•LTV = ∑ revenue per customer - ∑ service cost per customer
CAC: LTV ratio
•CAC: LTV metric is the proportion of cost to acquire a new customer in total net value brought by a customer.
Time to Customer Breakeven
•BEV (months) = CAC ÷ ARR
•ARR: Average recurring revenue per month.
•ARR (£) = LTV ÷ customer life time (month)
MVP Revenue (cont’t)

17.  Viral Cycle Time
Viral Cycle Time = t1 + t2+ t3
MVP- Virality
Virality is the ability of having new customers by referrals from existing customers. Virality helps to boost
profitability and scalability
Sign up
Use the product
Send Invitation to
others
Invitee receives
invitation
Invitee reviews and
decides to sign up
t1
t2
Almost no delay
t3
0
50000
100000
150000
200000
250000
m1 m2 m3 m4 m5 m6 m7 m8 m9 m10 m11 m12
Impact of Viral Cycle Time to Growth
cycle time = 2 months cycle time = 1 month

18.  Viral Coefficient
Viral coefficient = invitation rate x acceptance rate
MVP- Virality (Con’t)

19.  Exploit validated business model,
product and captured market to
maximise benefits.
 Build quick momentum in product
development and market scale to
outperform competitors and fast
followers
 Focus on product/service quality and
customer satisfaction
 Be aware of crossing the chasm
Scale
Technology Adoption Cycle - from 'Crossing the Chasm’ by Geoffrey Moore

20.  Continuous validation via metrics of
stickiness, revenue and virality
 Measure business performance
against targets
 Watch the market of the product
and identify its market maturity
stage to have relevant strategy
Scale- Market metrics
Market maturity life cycle, from Dealing with Darwin by Geoffrey A. Moore, 2006

21. Feature priotisation
• Differential Values to increase customer satisfaction
• Cost of Delay
• Return on Investment
• Others
Development efficiency (using flow-based approach)
• Little’s Law extention
• Cycle Time =
𝑊𝑜𝑟𝑘 𝑖𝑛 𝑃𝑟𝑜𝑔𝑟𝑒𝑠𝑠
𝑇ℎ𝑟𝑜𝑢𝑔ℎ𝑝𝑢𝑡
 Throughput =
𝑊𝑜𝑟𝑘 𝑖𝑛 𝑃𝑟𝑜𝑔𝑟𝑒𝑠𝑠
𝐶𝑦𝑐𝑙𝑒 𝑇𝑖𝑚𝑒
• Limit Work in Progress, tracking Cycle time and Throughput
Development Efficiency
“Deliver highest values within the shortest time”

22.  The Lean Startup Eric Ries
 Lean Analytics Croll & Yoskovitz
 AARRR framework Dave McClure
 Dealing with Darwin Geoffrey A. Moore
 Lean Enterprise Jez Humble, Joanne Molesky, Barry O’Reilly
References