In this workshop we'll show you how to: - Set the right testing goals that drive results - Create a test hypothesis using our experiment hypothesis framework - Analyze your results and communicate the value of data to key stakeholders

1. Conversion Optimization
Workshop

2. Lev Tatarov
Strategy Consultant

3. Agenda
Setting the right goals
Hypothesis building
workshop

4. Agenda
Setting the right goals
Hypothesis building
workshop

5. Company
Metric
Revenue
Business
Unit
Subscriptions
Optimization
Goals Visits to
subscription page
Experiment
Goals
Clicks on “subscribe”
button

7. Step
45%
30%
10%
65%
90%
70%
40%
40%
40%
20%
25%
15%
30%
50%
15%
Homepage
Category
Page
Product
Page
Shopping
Cart Page
Checkout
Page
Payment
Page

8. Step
45%
30%
10%
65%
90%
70%
Completion
Homepage
Category
Page
Product
Page
Shopping
Cart Page
Checkout
Page
Payment
Page

9. Step
40%
40%
40%
20%
5%
25%
Divergence
Homepage
Category
Page
Product
Page
Shopping
Cart Page
Checkout
Page
Payment
Page

10. Step
15%
30%
50%
15%Exit Rate
Homepage
Category
Page
Product
Page
Shopping
Cart Page
Checkout
Page
Payment
Page

11. Step
45%
30%
10%
65%
90%
70%
40%
40%
40%
20%
25%
15%
30%
50%
15%Exit Rate
Divergence
Completion
Homepage
Category
Page
Product
Page
Shopping
Cart Page
Checkout
Page
Payment
Page

12. Step
Primary /
Secondary
Goals
Monitoring
Goals
Homepage
Category
Page
Product
Page
Shopping
Cart Page
Checkout
Page
Payment
Page
 Product Page
Views
 Category page
views
 Searches
submitted
 Product views
 Subcategory
page views
 Filter use
 Searches
 Add-to-Cart
 Quantity
selection
 More info
 Tabs
 Continue
clicks
 Checkout
page views
 Payment type
chosen
 All Checkout
Funnel Steps
 Payment page
views
 Conversion
rate
 Conversion
rate
 Payment type
 Search bar
opened
 Top menu
clicks
 Home page
clicks
 Menu
interactions
 Searches
submitted
 Home page
clicks
 Menu
interactions
 Searches
submitted
 Home page
clicks
 Back button
clicks
 Shopping cart
page views
 Home page
clicks
 Return to
previous page
 Edit address,
information
 Error rate

13. Agenda
Setting the right goals
Hypothesis building
workshop

14. Explorative
“We have a few images we would
like to test for our new campaign
banner to see which one drives the
most clicks”
Declarative
“We notice that only 37% of users
who add an item to the shopping-cart,
actually checks the cart out. We
believe that if we decrease the
number of steps, we can drive move
cart checkouts”
Explorative vs Declarative Testing

15. Problem Solution Result
A great hypothesis begins with the problem,
NOT the solution!

16. • Enables a strong ideation process
• Enforces meaningful reasoning
• Drives focus
• Frames your ideas in a repeatable way
Why is a strong hypothesis important?

17. Problem
Company goal
Test Test Test
Problem
Test Test Test

18.  Discover 5-10 problems
for your users
 Group problems by
themes
 Prioritize most important
problems
Problem Solution Result
 Select 1-2 themes
 Design 5-10 solutions
per theme
 Select 2-3 preferred
solutions
 Primary Goals:
What do you seek to
impact immediately
 Secondary Goals:
What would you like to
impact, but do not have
to
 Monitoring Goals:
What are metrics you
want to track
Exercise: Build Powerful Hypotheses