Making Sense of the Numbers (Lean Analytics)

MAKING SENSE
OF THE NUMBERS
Ben Yoskovitz
@byosko
#leananalytics

Monday, March 18, 13

I’M NOT AN
ANALYTICS EXPERT


I’M AN
ENTREPRENEUR.
INVESTOR.
PRODUCT GUY.


I BELIEVE IN
GUTS + DATA


I BELIEVE IN
BRIDGING THE GAP
BETWEEN
ANALYTICS
AND
BUSINESS

Order today!
(please)

http://leananalyticsbook.com
h"p://oreilly.com
(LEAN13)


NOT JUST FOR STARTUPS

http://www.ﬂickr.com/photos/98909113@N00/308041146/sizes/z/in/photostream/

More wins than losses

GoInstant
1st startup Year One Labs
Started blogging
Standout Jobs
Big pivot

1996 1998 2001 2006 2007 2010 2011

The “I got too comfy” years Failed $0


WHAT DATA LOOKS LIKE
TO MOST PEOPLE


WE’RE DETECTIVES +
PROBLEM SOLVERS

http://www.ﬂickr.com/photos/ollieolarte/3028314931/sizes/z/in/photostream/


BUT WHAT PROBLEMS
MATTER
AND WHAT
SHOULD WE BE
LOOKING FOR?


Case study: From friends to moms

• Started as Circle of Friends
• Grew to 10M users

BUT ENGAGEMENT SUCKED


Case study: Moms are crazy (in a good way!)

• Messages to one another were on average 50% longer.
• 115% more likely to attach a picture to a post they wrote.
• 110% more likely to engage in a threaded (i.e. deep) conversation.
• Friends, once invited, were 50% more likely to become engaged users.
• 180% more likely to click on Facebook news feed items.
• 60% more likely to accept invitations to the app.

ENGAGEMENT WAS GREAT


WE TURN DATA
INTO DOLLARS

http://www.ﬂickr.com/photos/jfolsom/5931303869/sizes/l/in/photostream/


HOW TO SIMPLIFY
ANALYTICS
(so you can explain it to others)


Analytics is the measurement of
movement towards your
business goals.


Six things you can explain to
others about metrics

* Simple vs. Complex
* Qualitative vs. Quantitative
* Ratios vs. Numbers
* Exploratory vs. Reporting
* Vanity vs. Actionable
* Leading vs. Lagging


simplify.

http://www.ﬂickr.com/photos/josefeliciano/3849557951/sizes/l/in/photostream/


Qualitative Quantitative
Unstructured, Numbers and stats;
anecdotal, revealing, hard facts but less
hard to aggregate. insight.

Warm and fuzzy. Cold and hard.


DISCOVER QUALITATIVELY
AND

PROVE QUANTITATIVELY


Number Ratio
Absolute, difﬁcult Comparative,
to analyze or easier to analyze,
compare to particularly vs.
anything. other cohorts


Exploratory Reporting
Speculative, trying to Predictable, keeping
find unexpected or you abreast of
interesting insights. normal, managerial
operations.

http://www.flickr.com/photos/50755773@N06/5415295449/ http://www.flickr.com/photos/elwillo/4737933662/

Vanity Actionable
Picks a
direction.

Makes you feel
good, but doesn’t
change how you’ll
act.

VANITY METRICS ARE BAD

A metric from the early, foolish days of the Web.
Hits
Count people instead.
Marginally better than hits. Unless you’re displaying
Page views
ad inventory, count people.
Is this one person visiting a hundred times, or are a
Visits
hundred people visiting once? Fail.
This tells you nothing about what they did, why they
Unique visitors
stuck around, or if they left.

Followers/ Count actions instead. Find out how many followers
friends/likes will do your bidding.


http://www.ﬂickr.com/photos/circasassy/7858155676/

If it won’t change
how you behave,
it’s a

bad
metric.


LEADING LAGGING
Number today Historical metric
that shows metric that shows how
tomorrow-makes you’re doing-
the news reports the news

http://www.ﬂickr.com/photos/vegaseddie/3310041214/sizes/l/in/photostream/


What mode of e-commerce are you?
How many of
your customers Then you are in this Your customers will You are just
Focus on
buy a second time mode buy from you like
in 90 days?

Low CAC,
1-15% Acquisition Once 70% high
of retailers checkout

15-30% Hybrid 2-2.5 20% Increasing
per year of retailers returns

Loyalty,
>30% Loyalty >2.5 10% inventory
per year of retailers expansion

(Thanks to Kevin Hillstrom for this.)

The power of leading indicators
• A Facebook user reaching 7 friends within 10 days of signing up
(Chamath Palihapitiya)
• If someone comes back to Zynga a day after signing up for a game,
they’ll probably become an engaged, paying user (Nabeel Hyatt)
• A Dropbox user who puts at least one ﬁle in one folder on one device
(ChenLi Wang)
• Twitter user following a certain number of people, and a certain
percentage of those people following the user back (Josh Elman)
• A LinkedIn user getting to X connections in Y days (Elliot Schmukler)

(from the 2012 Growth Hacking conference)

ANALYTICS SUPERPOWERS
(or what the heck is growth hacking?)

http://www.ﬂickr.com/photos/bloke_with_camera/401812833/sizes/o/in/photostream/


10000

1000

100

10

1
Jan Feb Mar Apr May Jun Jul Aug Sept Oct Nov Dec

Ice cream consumption Drownings

Correlated Causal
Two variables that An independent
change in similar ways, factor that directly
perhaps because they impacts a dependent
are linked to something one.
else.
Summer
al

Ca
us

us
Ca

al
Correlated
Ice cream Drowning
consumption

Causality is a superpower, because it lets you change
the future.

Correlation lets you Causality lets you
predict the future change the future

“I will have 420 engaged “If I can make more ﬁrst-
users and 75 paying time visitors stay on for
customers next month.” 17 minutes I will increase
sales in 90 days.”

Optimize the
Find correlation Test causality
causal factor

LEAN ANALYTICS
FRAMEWORK


Basics of Lean Startup

Eric Ries
http://theleanstartup.com


SO WHAT
METRICS
SHOULD YOU TRACK?


YOUR BASIC
THE STAGE OF
BUSINESS
YOUR STARTUP
MODEL
How you make $$ Lifecycle

• E-commerce • Empathy
• SaaS • Stickiness
• Free mobile app • Virality
• Media site • Revenue
• Collaborative content site • Scale
• Two-sided marketplace


LEAN ANALYTICS “GATE” NEEDED TO
STAGES MOVE FORWARD
I’ve found a real, poorly-met need
EMPATHY that a reachable market faces.

I’ve ﬁgured out how to solve the
problem in a way they will adopt
STICKINESS and pay for.

I’ve built the right product/features/
GROWTH RATE

functionality that keeps users
VIRALITY
around.

The users and features fuel growth
REVENUE organically and artiﬁcially.

I’ve found a sustainable, scalable
business with the right margins in a
SCALE
healthy ecosystem.


Case study: Buffer goes from
Stickiness to Scale (through Revenue)

• Stage: Scale
• Model: SaaS (consumer)
• Popular social sharing application.
• Focused primarily on customer
acquisition
• Charged from day one


Buffer charges early to prove people
want the problem solved

20% of visitors create an account
(acquisition / Empathy)
of sign-ups return in the 1st month
60% (engagement / Stickiness)

of sign-ups are active after 6 months
20% (engagement / Stickiness)

convert from free to paid
2% (Revenue)


SKIP STEPS
AT YOUR
OWN RISK


Case study: WineExpress increases
revenues

• Stage: Revenue
• Model: E-commerce
• Exclusive wine shop partner of the
Wine Enthusiast catalog and website
• “Wine of the day” page is highly
trafﬁcked, needed optimization


Case study: Before and after

41%
increase in
revenue per
visitor


How it all comes together

The business you’re in

E- 2-sided Mobile User-gen
SaaS Media
commerce market app content
Empathy
The stage you’re at

Stickiness

Virality

Revenue

Scale


How it all comes together

The business you’re in

E- 2-sided Mobile User-gen
SaaS Media
commerce market app content
Empathy
The stage you’re at

One Metric
Stickiness

Virality

Revenue That Matters.
Scale


CHOOSE
ONLY ONE
METRIC
AND
DRAW A LINE
IN THE SAND

Case study: Timehop aims for
virality through content sharing

• Stage: Virality
• Model: Mobile app
• Social network around the past
• Focused on virality (but not the viral
coefﬁcient)


The One Metric That Matters:
Content sharing

• Focused on % of daily active users
that share content

• Aiming for 20-30% of daily active users to
share content

“All that matters now is virality. Everything else—be it
press, publicity stunts or something else—is like
pushing a rock up a mountain: it will never scale. But
being viral will.” - Jonathan Wegener, co-founder


Case study: SEOmoz reduces the
KPIs it tracks

• Stage: Scale
• Model: SaaS
• SEO toolkit (product suite)
• Reduced KPIs to focus on Net Adds


Net Adds = “health of the business”
indicator

If Net Adds: Why & Next Steps:


indicator

• Was a marketing campaign successful?
• Was churn lowered?
• Were customer complaints lowered?
• Was a product upgrade valuable?


indicator


• How can we acquire more valuable customers?
• Can we increase site conversion?
• How can we lower churn?
• What product features can increase engagement?


indicator


• How can we acquire more valuable customers?
• Can we increase site conversion?
• How can we lower churn?
• What product features can increase engagement?

• Are the new customers not the right segment?
• Did a marketing campaign fail?
• Are too many customers churning?
• Did a product upgrade fail to impress or cause issues?


METRICS
ARE LIKE
SQUEEZE
TOYS


YOUR GOAL IS TO MAKE
FASTER,
MORE INTELLECTUALLY
HONEST DECISIONS
AND

EMPOWER YOUR
ORGANIZATION
TO DO THE SAME


TURN DATA INTO DOLLARS

http://www.ﬂickr.com/photos/68751915@N05/6551534889/sizes/l/in/photostream/


HOW TO
BECOME
DATA-DRIVEN


The Lean Canvas (leancanvas.com)


AVOID THE
PITFALLS
@mrogati
• Assuming the data is clean
• Ignoring seasonality
• Data vomit
• Metrics that cry wolf
• The “not collected here”
syndrome
• Focusing on noise (look at
the big picture)
http://www.ﬂickr.com/photos/shefﬁeldmickey/504565957/


DATA DRIVEN CULTURE
• Start small, pick one thing and show value
• Make sure goals are clearly understood
• Get executive buy-in
• Make things simple to digest
• Ensure transparency
• Don’t eliminate your gut
• Get closer to the customer
• Ask good questions

http://www.ﬂickr.com/photos/jonolave/3619431413/sizes/o/in/photostream/


Once, a leader convinced others in
the absence of data.


Now, a leader knows what
questions to ask.


Thank you.
follow me. email me.
@byosko

byosko@gmail.com
instigatorblog.com

subscribe.
ORDER!

leananalyticsbook.com
oreilly.com (LEAN13)


Making Sense of the Numbers (Lean Analytics)

Empfohlen

Empfohlen

Weitere ähnliche Inhalte

Andere mochten auch

Andere mochten auch (11)

Ähnlich wie Making Sense of the Numbers (Lean Analytics)

Ähnlich wie Making Sense of the Numbers (Lean Analytics) (10)

Mehr von Lean Analytics

Mehr von Lean Analytics (15)

Kürzlich hochgeladen

Kürzlich hochgeladen (20)

Making Sense of the Numbers (Lean Analytics)