21. Â Metrics
Â
 Figure
 out
 what
 to
 improve
Â
 Hypothesis
Â
 Make
 an
 educated
 guess
Â
Â
Â
 Experiment
Â
 Test
 your
 guess
Â
 Act
Â
 Decide
 whatâs
 next
22. Our funnel showed a low installation rate
Hello Bar Funnel Metrics
Steps Visitors Homepage
URL Entries
Created
First Bar
Completed
Registration
Installed
Totals 15068 1868 716 688 398
% prev step 12.40% 38.33% 96.09% 57.85%
% drop off -87.60% -61.67% -3.91% -42.15%
% of visitors 100% 12.40% 4.75% 4.57% 2.64%
23. We
 achieved
 a
 40%
 increase
 for
 our
Â
installa2on
 rate
 with
 just
 this
 one
Â
ïŹrst
 experiment.
The Lean Analytics Cycle At Work
25. Â Metrics
Â
 Figure
 out
 what
 to
 improve
Â
 Hypothesis
Â
 Make
 an
 educated
 guess
Â
Â
Â
 Experiment
Â
 Test
 your
 guess
Â
 Act
Â
 Decide
 whatâs
 next
32. Hello Bar Experiment Hypothesis
If
 more
 installa2on
 op2ons
 are
 given,
Â
then
 more
 people
 will
 add
 Hello
 Bar
Â
to
 their
 website
 due
 to
 the
 need
 for
 a
Â
WordPress
 plugin
 and
 ability
 to
 email
Â
the
 code
 to
 someone
 else.
33. Â Metrics
Â
 Figure
 out
 what
 to
 improve
Â
 Hypothesis
Â
 Make
 an
 educated
 guess
Â
Â
Â
 Experiment
Â
 Test
 your
 guess
Â
 Act
Â
 Decide
 whatâs
 next
39. Â Metrics
Â
 Figure
 out
 what
 to
 improve
Â
 Hypothesis
Â
 Make
 an
 educated
 guess
Â
Â
Â
 Experiment
Â
 Test
 your
 guess
Â
 Act
Â
 Decide
 whatâs
 next
40. Full funnel metrics for the experiment
Variation
Steps Visitors Homepage
URL Entries
Created
First Bar
Completed
Registration
Installed
Totals 7456 906 342 329 276
% prev step 12.15% 37.75% 96.20% 83.89%
% drop off -87.85% -62.25% -3.80% -16.11%
% of visitors 100% 12.15% 4.59% 4.41% 3.70%
Control
Steps Visitors Homepage
URL Entries
Created
First Bar
Completed
Registration
Installed
Totals 7469 925 353 341 197
% prev step 12.38% 38.16% 96.60% 57.77%
% drop off -87.62% -61.84% -3.40% -42.23%
% of visitors 100% 12.38% 4.73% 4.57% 2.64%
41. Resulted in a 30% installation rate increase
HTTP://ISVALID.ORG
Experiment resulted in a 40% improvement
42. ACer
 a
 dozen
 more
 experiments,
 our
Â
installa2on
 rate
 is
 now
 hovering
Â
between
 5%
 and
 7%
Â
Thatâs
 an
 addi;onal
 89%
 increase!!!
The Lean Analytics Cycle At Work
53. âA solid test hypothesis is an informed solution to
a real problem â not an arbitrary guess. The more
research and data you have to base your
hypothesis on, the be er it will be.ââš
â Michael Aagaard, h p://kiss.ly/abthypo
74. Only
 1
 out
 of
 5
 tests
 win.
4
 out
 of
 5
 tests
Â
cost
 you
 money.
$
75. 1)
 Which
 top
 metric
 are
 we
 trying
 to
 improve?
Â
2)
 What
 can
 you
 anchor
 your
 metric
 against?
Â
3)
 What
 is
 your
 hypothesis
 about
 why
 this
 metric
 is
 at
 x%?
Â
4)
 What
 do
 we
 expect
 from
 this
 experiment?
Â
5)
 What
 was
 the
 result
 and
 what
 did
 we
 learn
 from
 it?
HTTP://KISS.LY/6QSAB
Answer key questions for each experiment
84. KISSmetrics Experiment Hypothesis
If
 people
 authen2cate
 with
 Google,
Â
then
 more
 people
 will
 sign
 up
 due
 to
Â
their
 exis2ng
 familiarity
 with
 Google
Â
Analy2cs
 and
 alignment
 with
 our
Â
winning
 headline.
95. Â Metrics
Â
 Figure
 out
 what
 to
 improve
Â
 Hypothesis
Â
 Make
 an
 educated
 guess
Â
Â
Â
 Experiment
Â
 Test
 your
 guess
Â
 Act
Â
 Decide
 whatâs
 next
96. Hiten Shah
hnshah@gmail.com
@hnshah
You will get all you want in life if you help enough
other people get what they want.
Zig Ziglar
Download the slides: h p://kiss.ly/leanprod2014