Customer Health Scoring Best Practices

How to evaluate customer engagement and likelihood of churn.
CUSTOMER HEALTH
SCORING BEST PRACTICES
Sam Richard | Director of Growth at OpenView

Proprietary and Confidential ©2020 OpenView Advisors, LLC. All Rights Reserved 2
WE CAN ALL AGREE:

IT’S NEVER BEEN MORE
IMPERATIVE TO UNDERSTAND
IF YOU’RE PROVIDING
ENOUGH VALUE TO
YOUR CUSTOMERS

IT MAY NOT BE
POSSIBLE TO SPEAK TO
AND UNDERSTAND HOW
THEY THINK ABOUT
YOUR PRODUCT
EVERY SINGLE CUSTOMER

SOMETIMES YOU NEED
YOUR PRODUCT TO TELL YOU
IF YOUR CUSTOMERS ARE
OR IF THEY WANT TO LEAVE
TRULY SEEING VALUE

HERE’S THE KICKER:
SOME CHURN
ISN’T PREVENTABLE

11%
89%
Not preventable
Preventable
KEEP IN MIND PRODUCT CAN GET IN FRONT OF
89% OF CHURN
Some churn simply
can’t be prevented:
• Companies going out of
business
• Companies being acquired
• New admins used to their
own tools
• New processes put in place
Churn Reasons by Account

SO HOW CAN YOU
SAVE THAT 89%?

9Proprietary and Confidential ©2020 OpenView Advisors, LLC. All Rights Reserved
NOT TO BE ALARMIST, BUT IT
COULD BE LESS THAN 89%
DEPENDING ON YOUR
TARGET MARKET
*

TIME IS OF THE ESSENCE;
YOU DON’T WANT SOME
ANALYST GOING ON A
PRODUCT ANALYTICS
WILD GOOSE CHASE

PRO TIP: TALK TO CUSTOMER
FACING EMPLOYEES TO
DEVELOP HYPOTHESES FIRST

User
Offboarding
Bad Support
Cases
Quiet Account
Disengaged
Admin
Product Usage
Theme Respondents felt
that the user
count of an
account
decreasing over
time was a bad
sign.
Respondents
thought that bad
support
experiences for
the account could
lead to churn.
An account
stakeholder not
asking questions
or engaging with
CSX was
considered
negative.
An admin not
regularly in the
application or
taking action
The breath and
depth of feature
usage in the
product
Examples An account goes
from 200 users to
15 over a month.
A support case
with a longer-
than-average
time to solve, or
where the admin
seems upset
(tone).
When CSX sends
their update
emails, an admin
doesn’t engage.
An admin not
reaching out to
support.
An admin who
only adds users
An admin who
doesn’t take
many actions in
the app.
Only uses one or
two pages of the
product, spends
limited time.
COMMON THEMES IN YOUR BUSINESS CAN
EVALUATE CUSTOMER HEALTH. FOR EXAMPLE:

SO HOW DO YOU
THE THEMES?
GO ABOUT VALIDATING

TAKE EACH THEME
INDIVIDUALLY AND
EVALUATE HOW CLOSELY IT
CORRELATES TO CHURN

Recommended Action
FOR EXAMPLE:
What we learned from our research
User counts aren’t that indicative of engagement. User
counts rarely changed prior to churn or expansion. Churners just
paid until they called to cancel.
Difficult account questions or longer ticket wait times
didn’t correlate to churn. Churners are speaking with support
more often than healthy accounts, not going quiet.
Engagement with and usage of the product is the key
indicator of customer health. Speed to set up an account, as
well as the number of features and products used, are the best
indicators of an account’s commitment to using the app.
Pay attention to account admin behavior. This gives apps
that have not yet driven user reviews the ability to receive
placement at the top of the app store.
Find another metric to measure
customer health. While pricing is tied to
user count, organizations that are going to
churn or expand don’t give indicators using
this measure.
Monitor and leverage customer success
feedback to understand the personal and
circumstantial side of how accounts use or
don’t use the product.
Use usage-based metrics to understand
account health for customer success, sales
and support. Create an engagement score
powered by the usage indicators that matter
most.
Add admin engagement to your score
and build playbooks around outreach to
expand revenue or prevent contraction.
1
2
3
4

YOU DON’T SAY?
Engagement with and usage of the product is the key
indicator of customer health. Speed to set up an account, as
well as the number of features and products used, are the best
indicators of an account’s commitment to using the app.
Use usage-based metrics to understand
account health for customer success, sales
and support. Create an engagement score
powered by the usage indicators that matter
most.
3

SO HOW DO YOU
UNDERSTAND THE DEPTH
OF EACH ACCOUNT’S
ENGAGEMENT WITH
YOUR PRODUCT?

WHILE THE WHOLE WORLD IS
CONCERNED WITH STAYING
HEALTHY, YOU SHOULD BE
MEASURING YOUR
ACCOUNTS’ HEALTH, TOO.

AND BY HEALTH WE DON’T
JUST MEAN LIKELIHOOD OF
CHURN

DRILL DOWN INTO
THE USAGE THAT MATTERS

IT COULD BE A SPECIFIC
FEATURE USED WITH
REGULARITY (DEPTH OF USE)

OR IT COULD BE MULTIPLE
FEATURES USED BY AN
ACCOUNT (BREADTH OF USE)

10%
5% 5% 5% 4% 4%
18%
15%
33%
64%
13%
7%
4% 3% 2%
5%
2% 2%
0 1 2 3 4 5 6-10 11-15 15+
Healthy Churned
EXAMPLE: COUNT OF FEATURES USED CORRELATES
TO LIKELIHOOD OF CHURN
Number of features used by healthy and churned accounts
Insights:
• Accounts that
convert use features
90% of the time,
and they’re using 3
or more features
79% of the time.
• Meanwhile, only
16% of non-
converters add 3+
resources.

OR IT COULD BE A MIXTURE
OF THE TWO

FIND THOSE 2-3 HIGHEST
VALUE ACTIONS

NEXT COMBINE THOSE LEVERS
INTO A HEALTH SCORE FOR
EACH ACCOUNT

AND DON’T OVERTHINK IT!

IT CAN BE THIS SIMPLE
50%
25%
25% Product Usage
CSX Feedback
Admin
Engagement
The data team should:
• Establish a dataset of all inputs that would
be part of the score
• Weight those inputs
• Use a random forest model to predict each
account’s likelihood to churn each day
• Use that percent likelihood as a health score
• Note: add alerts based on the findings of
biggest drivers of churn
Weighting of score by Category

EXPERTS’ TAKES ON HEALTH SCORING
“Common ingredients
for a customer health
score are:
• Business
outcomes
• Usage and
engagement
• Utilization
These are all
indicators that help
you make a decision
of what to do with
each customer next”
-Guy Nirpaz, CEO,
Totango
Best Practices
Takeaways“Outside of CSX, little
was known about the
score. We built a score
that allowed more
people to use it
without a lot of time
learning how to
interpret it…we
provided info about
why a score was low,
since teams felt lost
when reaching out to
unhealthy customers.”
-Erin Akinci, Data Science,
Asana
“We used health
scores to see whether
cohorts are doing
better or worse than
baseline. We use this
to see whether
experiments we each
month are working. It
gives us a valuable
leading indicator
to see whether
changes are having
positive effects.”
-Jonah Lopin, Founder,
Crayon
According to experts,
the ideal health score
should:
• Be a leading indicator
of desired business
outcomes
• Consider usage and
engagement as inputs
• Be easy to understand
and socialize internally
• Be tracked on a cohort
basis

EXAMPLE SCORE: ASANA
“We chose to map the
number of trees (from a
randforest) predicting churn
to an actual probability. That
score between 0-100 is the
health score of each account.
We used product inputs, but
also CS provided us with
qualitative indicators of
account health and we
would look for a way to
quantify them given the
available data.”
-Erin Akinci, Data Science, Asana
Read full piece here
This is a real-life health dashboard for the Asana CS team

EXAMPLE SCORE: USERIQ
“UserIQ’s customer health
dashboard monitors five key
variables called churn
indicators that are used to
calculate Health IQ.
Monitoring indicators of
potential churn is a
necessary function of
making sure customers
become (and remain)
successful.”
-Erika Childers, Marketing, UserIQ
The 6 inputs that UserIQ uses to understand customer health

EXAMPLE SCORE: RAPID7
“Rapid7’s first customer
health score was an
amalgam of the following:
product adoption (40%),
support experience (35%),
and purchasing behavior
(25%). Each of these 3
components was a
mashup of many data
points before a score
was created. Plans of
action would be placed
according to that score.”
-Kalley Collete, BI, Rapid 7
The evolution of datapoints that Rapid7 Used

ONCE YOU HAVE A SCORE,
YOU NEED TO GET IT IN THE
HANDS OF TEAMMATES WHO
CAN FIGHT CHURN ASAP

BUT DON’T FORGET TO
CONFIRM WITH THOSE
STAKEHOLDERS THAT THEY
SEE VALUE IN IT, TOO

WHAT ARE THE BEST WAYS TO IMPLEMENT SCORES?
“Focus on getting
yellow accounts to
green. A healthy
account is 6x more
likely to upsell than
one that is “okay.”
The effort you put
into getting an
unhealthy red
account to an
“okay” yellow
account may not
be worth it”
-Jeff Johnson, Dir. CSX,
Splunk
“If anything is a fire
alarm, build alerts
to get your team on
it. A Customer
Scorecard won’t give
you everything, you
have to have an
escalation process in
place.”
-Jon Turri, VP CSX,
Raise.me
Focus Rally your teams Takeaways
“Aligned CS and
product
management
teams benefit from
a common vision
of customer health
and feature usage.
Understanding churn
warning signs or
power-user behavior
helps both teams
deliver the right
responses or change.
-Nicole Smith, CMO,
Tackle.io
The best ways to get
your organization
focused around a
health score:
• Don’t try to save
everyone but do use
the health score to pull
accounts back from the
edge.
• Build alerts and
workflows for the team,
expand further with
playbooks
• Get all teams on the
same page
Takeaways

STEPS TO TAKE: IMPLEMENTATION
• Decide on an
approach
• Create a dataset
that refreshes at
least weekly
• Cross check with
historic data
• Consult with CSX
on how the
score should be
surfaced on
account level
• Build
dashboards for
each level of
employee
• Socialize scoring
definitions
Creation Implementation Playbooks
Data Team Data Team CSX CSX
• Begin training
the team on
what health
score is
• Work to create
“rules” on the
CSX-based
inputs
• Start to utilize in
account
planning and
outreach flow
• Experiment with
workflows based
on scoring and
alerts
• Consult with
data team on
what’s working
and what’s not
• Socialize across
organization
(especially
support, sales)
ImplementationCreation

Sam Richard
Questions? Email me at Sam@ov.vc
THANK YOU

Customer Health Scoring Best Practices

Empfohlen

Empfohlen

Weitere ähnliche Inhalte

Was ist angesagt?

Was ist angesagt? (20)

Ähnlich wie Customer Health Scoring Best Practices

Ähnlich wie Customer Health Scoring Best Practices (20)

Mehr von OpenView

Mehr von OpenView (20)

Kürzlich hochgeladen

Kürzlich hochgeladen (20)

Customer Health Scoring Best Practices