Experiment your way out of Beta!

1. Experiment Your Way Out of Beta!
Or be stuck in a perpetual cycle of ignorance & low revenue
Rick Perreault
Co-Founder & CEO
Unbounce.com
Find me on twitter
@rickperreault
Slide #1 of 24

2. What is an experiment?
"scientific procedure undertaken to make a
discovery, test a hypothesis, or demonstrate a
known fact."
Important because they test your assumptions &
get you closer to the truth
Iʼm going to show you 3 types of experiments that
you should use to get out of beta

3. How do I start an experiment?

4. How do I start an experiment?
I heard about
you from...
I would definitely
pay if... I tell my
friends...
I hate you
because... I love you
because...
Customer feedback will give you insights & help
ask good questions

5. How do I start an experiment?
Surveys quick & easy way to get lots of customer
feedback

6. How do I start an experiment?
www.survey.io
Step 1: fill out these 5 form fields

7. How do I start an experiment?
www.survey.io
How did you discover sample app?
How would you feel if you could no longer use sample app?
What would you likely use as an alternative if sample app were no longer available?
-> What is the primary benefit that you have received from
sample app?
-> Have you recommended sample app to anyone? (If Yes,
please explain how you describe it)
What type of person do you think would benefit most from sample app?
How can we improve sample app to better meet your needs?
Would it be okay if we followed up by email to request a clarification to one or more of
your responses?
Step 2: email these 8 questions your customers,
done!

8. Experiment Type #1

9. Experiment #1
I love surveys! Not only for discovery, test
hypothesis as well

10. Experiment #1
My Hypothesis:
Customers that pay us the most & stay with us the longest
have a different product wish list than those who pay us the
least (but make the most noise)
My Test Method:
Survey our customers asking them to priorities feature
request & segment to track responses by pricing tiers

11. Experiment #1
Customers paying...
$25 per $125 + per
month want month want
features A, B, features F, G,
C H

12. Experiment #1
Action Item
Use a survey to learn what your customers find
most import (to get out of Beta)
If you have different customer types (Free, Paid,
Paid Tiers), segment
I used

13. Experiment Type #2

14. Experiment #2
A/B Test
Version A (control)
50%
Traffic
100%
Traffic Version B (test group)
50%
Traffic
Running two or more variations to determine
which one converts better

15. Experiment #2
Test ad copy, email (subject/body), website & landing pages elements,
example:
1. Value Proposition (your headline)
2. Image/Video of product or context of use
3. Core Benefits
4. Request for data (form size)
5. Call-to-action (CTA)
6. Trust elements (Testimonials)
7. Secondary CTAs (social sharing)
Your conversion rate WILL improve if you test

16. Experiment #2
Version A (control) (Variant - new headline) (Variant - new headline + smaller page)
Conversion 10%
Conversion 14%
Conversion 17.5%

17. Experiment #2
Action Item
Use Survey.io question #4 & #5 results to develop
new ways of describing your startup and test it
against what you currently have.
I used

18. Experiment Type #3

19. Experiment #3
“Fake It” Test
Developing features are
expensive so you should test
interest in the feature before
you spend time developing it.

20. Experiment #3
I Wanted to Know:
Would customers be interested in benchmarking their
conversion against their industry average?
Nothing says ʻinterestedʼ like clicking a “Buy Button”
My Test Method:
Fake Landing page describing the feature with CTA
tracking

21. Experiment #3
Landing Page
Link from App
CTA, buy for $5/mo

22. Experiment #3
Landing Page
15% Clicked Buy

23. Experiment #3
Action Item
I havenʼt done many ʻfake itʼ
experiments
Best advice read “The Lean Startup”
This action doesnʼt require any
software or dev time

24. Thank You!
Any questions, find
me on twitter
@rickperreault