BoSUSA20 Online | Matt Lerner | Metrics that Matter

W W W . S T A R T U P C O R E S T R E N G T H S . C O M

M E T R I C S T H A T M A T T E R
22 September, 2020
You get what you measure. Choose carefully.
Matt Lerner

J A N U A R Y, 2 0 0 9
S A N J O S E , C A L I F O R N I A

S Q U A R E : $ 3 6 B S T R I P E : $ 3 0 B

Y O U G E T W H A T Y O U M E A S U R E
A L I G N A N D F O C U S T H E T E A M
A D O U B L E - E D G E D S W O R D
I F Y O U M E A S U R E T H E W R O N G T H I N G
M E T R I C S

Silicon Valley veteran (4 startups: ! ,", #, and $)
Growth team @ PayPal
Partner @ 500 Startups VC
Lectures at Stanford Business School, Imperial
Founder of Startup Core Strengths – Coaching Program
StartupCoreStrengths.com/playbook (it’s free)
M A T T L E R N E R – F O U N D E R & C E O

H O W D O T A R G E T S C A U S E S U C C E S S ?
G O O D M E T R I C S

9 0 % O F Y O U R
R E S U L T S
C O M E
F R O M 1 0 % O F
Y O U R W O R K
0%
20%
40%
60%
80%
100%
Results Work
T H E C R I T I C A L I N S I G H T

H E L P P E O P L E M A K E B E T T E R D E C I S I O N S A B O U T
W H I C H W O R K T O D O

Measure customer value in the
first instance, not profit

D O N O T C H A S E R E V E N U E I N T H E F I R S T I N S T A N C E

D E L I V E R V A L U E A N D R E V E N U E W I L L C O M E

H O W D O Y O U M E A S S U R E V A L U E D E L I V E R E D ?

W H A T I S C U S T O M E R V A L U E D E L I V E R Y ?
If customers absolutely loved your product or service, how
would they naturally behave?
You deliver value when you help customers make progress
in their lives.
What measurable customer behavior is the closest proxy for
that “value delivery”?
How many of them are behaving like this?
(Track in cohorts, improve over time.)

Y O U R M E T R I C S

T H R E E K E Y T Y P E S O F M E T R I C S
1. North Star Metric – Customer behavior that
encompasses the whole funnel or marketplace, and
increments when value is delivered to both customer
and the company.
2. Key Drivers – 1-3 Measured first-order impacts of the
work you do each day to maximise the NSM.
(One of these is your “rate limiting step”)
3. Nuance Metrics – Important numbers to watch,
maintain in a range, but not maximise.

W H A T I S Y O U R N O R T H S T A R M E T R I C ?
North Star Metric Increments when we deliver value to both
customer and company in a natural way where customers
and company behave as expected.
1. Simple & memorable for the whole company
2. Represents the full funnel including new, engaged and
churned users.
3. Does not change often (e.g. every few years max)
4. Everyone can connect their work to this number

A S I M P L E E X A M P L E

Subscribers * Avg Plan Cost
WAUs (Retained + New Actives) + Lapsed
Make First Wireframe
Trial Account
Qualified Traffic
Monthly Recurring Revenue (MRR)
Y O U R N O R T H S T A R
Make More Wireframes, Share, etc.
Convert to Paid Plan

B2C
AirBnB – Nights booked
Facebook – DAUs
Quora – Number of answered
Medium – Minutes Read
B2B
Slack – Daily Active Users (DAUs)
Amplitude – Weekly querying users
PayPal – Total Payment Volume (TPV)
Hubspot - # of weekly active teams
If customers absolutely loved your product or service, how would they naturally
behave?
E X A M P L E N O R T H S T A R M E T R I C S
MARKETPLACES
Your NSM is typically a transaction where two sides come together.

K E Y D R I V E R S
Key Drivers are the customer behavior “levers” you pull each day
to maximize the North Star. They:
1. Are controllable customer behaviours.
2. Select only a few (1-3)
3. Often, it’s a ratio (e.g. activation rate, signup rate).
4. Revisit every 3 months.
5. Each one should have an owner and target.

“To give people the power
to build community and
bring the world closer
together.”
“Provide simple, affordable,
secure and reliable digital
payments to…millions of
people around the world.”
“To offer our customers the
lowest possible prices, the
best available selection, and
the utmost convenience”
Mission
North Star
E X A M P L E C U S T O M E R V A L U E K E Y D R I V E R S
Total Payment Volume Repeat Purchases Daily Active Users
Ubiquity
(be available at every
transaction point)
Preference
(be every consumer’s
preferred payment method)
Price
Selection
Convenience
DAUs by product, number
of products
Registered users, user
frequency
Invites, follows, post count,
post engagement, etc.
Key Drivers

Which of these metrics, if improved, makes the whole system
run better?
Find it. Focus on it.
Y O U R R A T E L I M I T I N G S T E P

Trial Account
Qualified Traffic
Y O U R K E Y D R I V E R S
Make More Wireframes, Share, etc.

W H A T A R E Y O U R K E Y D R I V E R S ?
Which main levers cause your “value delivery metric” to go up?
Which customer behavior best tracks those levers?

R E T E N T I O N K E Y D R I V E R S
Absolute numbers matter less than the shape of the curve
Key Driver:
How would people naturally behave if they loved your product?
What early life behavior pattern predicts retention? That’s the
habituation threshold
What % of new signups cross that line?
Starts with finding your habituation threshold
How much of your revenue is coming from habituated customers?
(Happy MRR) vs. New customers vs. At-Risk ones?
CumulativeWAUs
Monthly Cohorts
CumulativeWAUs
Monthly Cohorts

Trial Account
Qualified Traffic
Y O U R H A B I T U A T I O N T H R E S H O L D
åMake More Wireframes, Share, etc.

W H A T A R E Y O U R K E Y D R I V E R S ?
Which main levers cause your “value delivery metric” to go up?
Which customer behavior best tracks those levers?
Which one is your rate limiting step?
Who “owns” each of these key metrics?

W H A T A R E Y O U R N U A N C E M E T R I C S ?
Nuance Metrics are important “read only” numbers (watch but
not maximise). Make sure they stay within an acceptable range.
Many are drivers, but not “key” drivers
Sometimes they are not directly controllable
Helpful if they explain variations in the North Star
Often a “check” number that should stay in a certain range

D O E S T H I S C H A N G E
I F Y O U D O N O T H A V E P R O D U C T - M A R K E T F I T ?

“If your product is broken and people are still using it, if you have high retention with
a broken product, that’s a clear sign you have PMF.”
Elad Gil

100%
Find
Product-Market
Fit
60%
Maximise
Proven Vectors
40%
Seek Adjacent
Vectors
80%
Optimise
Core Business
20%
Find New Business Lines
Post-PMF Post-PMF
SuperImportant
Decision
Yourtime,energyandresources
B E H O N E S T W I T H Y O U R S E L F
Pre-PMF
“The Hunt” “The Sprint” “The Marathon”

Deepen fit through iterative learning loops.
Key Drivers incent deepening fit + validated
customer learning:
1. Strength of fit – e.g. conversion rates,
activation rates, easy to sell, referral rates,
retention
2. Pace of learning – experiments run per week
Tips: First validate the need. Look for
“language-market fit”
F I N D I N G P M F K E Y D R I V E R S

B R I N G I N G Y O U R M E T R I C S T O L I F E

THE PROCESS
The leverage is in which work you choose to do.
Your goal is to get everyone doing the most impactful work, making
good tradeoffs about how to spend their time and budgets.
Each person must understand the North Star, and be able to connect
their work to it. (OKRs)
And that should empower them to make good “resource allocation”
decisions for how to move that metric, as the expert in their domain.
Keep asking “Which is the most important work, and how is it going?”
H E R E I S T H E M A G I C

C O M M O N
P I T F A L L S
Data should help us focus, make good decisions
quickly and keep executing.
X Focusing on revenue, not value delivery
X Chasing too many metrics
X Analysis paralysis (e.g. attribution, retention)
X Obsessed with a local maximum

North Star Metric of Value Delivery, endures over time
Focus on 1-3 Key Drivers, including your rate limiting step, revisit
focus every 3 months.
If pre-PMF, key drivers focus you on fast learning and fit-deepening
For retention, figure out your habituation threshold & work backwards
Each key driver gets a single accountable person.
Questions for each employee:
Do they understand how their work affects the NSM?
Do they understand their key driver?
Can they make good tradeoffs re. investments of time and budget, and
focus them on the most impactful work?
R E C A P

Y O U R H O M E W O R K : 5 Q U E S T I O N S
1. A conversation with your CEO, Head of Product, Head
of Marketing, Finance + Analytics. Figure out:
1. What’s your North Star Metric?
2. What are your 1-3 biggest drivers of that metric,
company-wide?
3. What’s your “rate-limiting step”?
2. Ask each employee: “How does your work each day
impact our North Star?” (Check for alignment &
comprehension)
3. Regularly ask each employee or team: “What’s the
most important work, and how’s it going?”

M E A S U R E T W I C E
C U T O N C E
@MattHLerner
StartupCoreStrengths.com/subscribe
matt@startupcorestrengths.com

BoSUSA20 Online | Matt Lerner | Metrics that Matter

Empfohlen

Empfohlen

Weitere ähnliche Inhalte

Was ist angesagt?

Was ist angesagt? (20)

Ähnlich wie BoSUSA20 Online | Matt Lerner | Metrics that Matter

Ähnlich wie BoSUSA20 Online | Matt Lerner | Metrics that Matter (20)

Mehr von Business of Software Conference

Mehr von Business of Software Conference (20)

Kürzlich hochgeladen

Kürzlich hochgeladen (20)

BoSUSA20 Online | Matt Lerner | Metrics that Matter