How do you build an e-commerce experimentation program that results in a real sales lift? Widerfunnel Senior Experimentation Strategist, James Flory, tackled this question in a recent webinar. Access the slides to learn: - How to avoid common optimization mistakes and deliver an impactful program that is underpinned by framework thinking - How leading e-commerce companies are using advanced experimentation to drive conversion rate lift and increased customer lifetime value - How to use losing experiments to gain valuable insights and further drive growth

1. James Flory, Sr. Experimentation Strategist at Widerfunnel

2. James Flory is a Senior Experimentation Strategist
at WiderFunnel, working directly with leading
brands like H&R Block, Direct Energy, Intercom,
and The Motley Fool to help them achieve tangible
growth through digital experimentation.

3. “It should come as no shock that the age of
customer experience is already upon us.
Being competitive in the retail market today
means being able to learn and adapt more
quickly to the experiences that
customers want.”
MIT Sloan Management Review, 2018

4. © Copyright 2007-2019 Widerfunnel. All rights reserved.
“...by 2023, Alibaba and Amazon will
have captured 40 percent market
share of global online retail, up
from 33 percent in 2017.”

6. © Copyright 2007-2019 Widerfunnel. All rights reserved.
“The most important thing is to be
customer obsessed. Don’t satisfy
your customers, figure out how to
absolutely delight them. That is the #1
thing. ”

7. © Copyright 2007-2019 Widerfunnel. All rights reserved.
“Our success at Amazon is a function
of how many experiments we do
per year, per month, per week, per
day.”

8. © Copyright 2007-2019 Widerfunnel. All rights reserved.
Customer Obsession ExperimentationX

9. © Copyright 2007-2019 Widerfunnel. All rights reserved.

10. © Copyright 2007-2019 Widerfunnel. All rights reserved.

11. © Copyright 2007-2019 Widerfunnel. All rights reserved.
✓ How we collect data around shopper
behaviour, motivations, and other inputs.
✓ How we translate that data into experiment
hypotheses and prioritize those ideas.
✓ How we design, measure, and report on
experiments to maximize growth and
insights.
✓ How we scale these programs across
organizations.

13. © Copyright 2007-2019 Widerfunnel. All rights reserved.

14. © Copyright 2007-2019 Widerfunnel. All rights reserved.

15. © Copyright 2007-2019 Widerfunnel. All rights reserved.

16. © Copyright 2007-2019 Widerfunnel. All rights reserved.

18. © Copyright 2007-2019 Widerfunnel. All rights reserved.
TM
Does your web page
deliver on what your
prospects expect?
Does it match their
needs and feelings?This is the vehicle that
provides the potential for
your conversion rate, it is
the most important of the
six factors.
How easily is your
value proposition
communicated? This
includes eyeflow,
imagery, copywriting
and calls-to-action.
How are you matching
the visitors’ internal
urgency and creating
external incentives?
Is your website
content diverting
visitors away from
your primary goal?
What elements on
the page could be
creating uncertainty
for your prospects
about taking action?

19. ● Click maps
● Scrollmaps
● Session Recordings
● Polls & Surveys
● Interviews &
Intercepts

20. © Copyright 2007-2019 Widerfunnel. All rights reserved.
Insight:
Shoppers want to refine their results
by color.
Inconsistent use of terminology and
long list of options is increasing the
cognitive load for users
(reducing ability)

21. © Copyright 2007-2019 Widerfunnel. All rights reserved.
If we reduce the length of the filter options and clarify
the features by which users can filter we will increase
engagement with the filters, increasing the likelihood
users find a product they want, ultimately increasing
the likelihood they convert.

22. © Copyright 2007-2019 Widerfunnel. All rights reserved.
↑ 15.6% Sales ↑ 23.6% Sales ↑ 9.8% Sales

23. © Copyright 2007-2019 Widerfunnel. All rights reserved.

24. © Copyright 2007-2019 Widerfunnel. All rights reserved.
Over 9000 pixels tall
Low value information
mid-page.
Key product information
at the bottom of the page.

25. © Copyright 2007-2019 Widerfunnel. All rights reserved.
Variation A Variation BControl
↑15.8%
Sales ↑41.4%
Sales

26. © Copyright 2007-2019 Widerfunnel. All rights reserved.

27. © Copyright 2007-2019 Widerfunnel. All rights reserved.
Homepage Product Page

28. © Copyright 2007-2019 Widerfunnel. All rights reserved.
↑ 23% Sales
Hypothesis:
Adding the carrier
information to the product
page will increase clarity
about product functionality
and reduce anxiety around
purchase, ultimately
increasing conversions.

31. © Copyright 2007-2019 Widerfunnel. All rights reserved.
How much improvement
can be made?
How much impact can these
improvements have on our
primary business goal?
How easy would it be
to actually make these
improvements?

32. © Copyright 2007-2019 Widerfunnel. All rights reserved.
Increases Score Decreases Score
Potential
Importance
Ease
● Supporting Evidence
● Transferability of Insight
● High Exit / Abandon Rates
● No Supporting Evidence
● No Transferability
● Many Stakeholders
● High Tech Complexity
● Many Dynamic Elements
● Multiple Pages Affected
● Paid Traffic
● High Traffic Volume
● High Conversion Rate
● Business Context
● Low Value Traffic
● Low Traffic Volume
● Low Conversion Rate
● Few Stakeholders
● Single, Static Page

33. © Copyright 2007-2019 Widerfunnel. All rights reserved.
Hypothesis 1:
Social proof on the Homepage
Hypothesis 2:
Form fields on the checkout
Potential
Importance
Ease
+ Supporting Evidence
+ Transferability of Insight
+ Supporting Evidence
- No Transferability
+ No Stakeholders
- High Tech Complexity
- Many Dynamic Elements
- Multiple Pages Affected
+ High Traffic Volume
- Moderate Conversion
Rate
- Low Value Traffic
+ High Conversion Rate
+ High Revenue Impact
- Many Stakeholders
+ Single, Static Page

34. © Copyright 2007-2019 Widerfunnel. All rights reserved.
Hypothesis 1 8 5 5
Hypothesis 2 5 7 3 5
6

37. © Copyright 2007-2019 Widerfunnel. All rights reserved.

38. © Copyright 2007-2019 Widerfunnel. All rights reserved.
Growth Insights
Exploit
(A/B)
Explore
(Multivariate)Factorial

39. © Copyright 2007-2019 Widerfunnel. All rights reserved.
↑ 15.6% Sales ↑ 23.6% Sales ↑ 9.8% Sales

40. © Copyright 2007-2019 Widerfunnel. All rights reserved.
• Vertical product layout
• “View Details” is the primary call to
action

41. © Copyright 2007-2019 Widerfunnel. All rights reserved.
• Horizontal layout
↓ 9.6% Sales

42. © Copyright 2007-2019 Widerfunnel. All rights reserved.
• “Add to Cart” is the primary call to
action
↑ 16.4% Sales

43. © Copyright 2007-2019 Widerfunnel. All rights reserved.
↓ 9.6% Sales ↑ 16.4% Sales ↑ 29% Estimated
↑ 38.1% Actual
Control Variation A Variation B Control + B
Experiment 1
Explore & Exploit (Factorial)
Experiment 2
Exploit (A/B)

45. © Copyright 2007-2019 Widerfunnel. All rights reserved.
Variation AControl
↑153.6%
Video views
↓10.6%
Sales

46. © Copyright 2007-2019 Widerfunnel. All rights reserved.
🤦
💡
🙋‍♂

47. © Copyright 2007-2019 Widerfunnel. All rights reserved.
Product Page

48. © Copyright 2007-2019 Widerfunnel. All rights reserved.
↓ 29.4%
Video views
Control Variation A
↑ 33%
Sales

54. https://www.widerfunnel.com/delight/
iwant@widerfunnel.com