Why A/B Testing Scrambles Your Funnel’s Data, And How To Fix It presented by Evan Shellshear, Chief Commercial Officer, Ebex & Van Nguyen, Tech Lead and Full Stack Node.js Developer, Ebex Single page application architectures like ReactJS and AngularJS have created a revolution in user experiences that are fast, easy, and more dynamic. But they've also posed challenges for experimenters, often requiring extensive workarounds and development to play nicely with Optimizely. Until now. We're bringing easy and fast experimentation to single page apps in Optimizely--the way it was meant to be. This session will cover best practices for experimenting in SPAs, product updates related to single page apps, and provide you with tips to easily implement SPAs. ABOUT MODERNIZE 2018: Be a part of the future. Shape the customer experience with Accenture, Forrester, Qantas, REA, Intrepid and more -- and learn how leading organisations are harnessing their people, processes, and technologies to exceed customer’s expectations. Visit: https://www.optimizely.com/anz/modernize2018

1. Why A/B Testing Scrambles Your Funnel’s
Data, and How to Fix It
EVAN SHELLSHEAR
Chief Commercial Director
EBEX
VAN NGUYEN
Tech Lead
EBEX
experience

2. experience
Why A/B testing scrambles your
funnel’s data, and how to fix it
Evan Shellshear - Chief commercial officer
Van Nguyen - Tech lead
experience

3. experience
SPA?
Singaporean Pale Ale?

4. experience
1. Interactive experiences without page loads
2. Dynamic content based on javascript
3. Higher converting funnels
SPA - Single Page
Application

5. experience
The problem with A/B
testing SPAs
1. Missing URL changes
• A/B testing code does not reload after user goes
to a new step in the funnel.
2. No page loading
• Does not track page elements that haven’t
displayed yet.
3. Your data can get messed up
• Incorrectly configuring A/B tests will lead to
events being associated with the wrong funnel
pages

6. experience
1. Platforms began supporting SPAs in 2017
2. Risky using untested platforms
3. Which platform is embracing SPAs?
What are our options?

7. experience
1. Developer and non-developer support
2. Page-level conditional activation
3. Navigation-based activation
4. Trialed and tested solution
Optimizely

8. experience
How did it go?
The shocking truth that has these
conversion rate optimisers mad

9. experience
What happened?
The data got messed up...

10. experience
Lessons Learned
What should you avoid? What should you do?

11. experience
Hacking it with the data layer...
Lesson 1/3
www.eshealth.com.au/sign-up/#/step1
event: “trackStep”
funnel: “regular”
gtm.uniqueEventId: 13
step: 1
_page: “signup_form”
www.eshealth.com.au/sign-up/#/step2
event: “trackStep”
funnel: “regular”
gtm.uniqueEventId: 89
step: 2
_page: “signup_form”

12. experience
1. Don’t wire the testing logic directly in the
application.
2. Make sure all the variations including the
original one fire the same events
consistently.
3. And always: Remember to test the test!
Lesson 2/3

13. experience
Lesson 3/3
1. SPAs are future of high converting
websites.
2. They raise conversion rates by
providing a better user experience
3. Any company with a funnel will
migrate to this format at some point

14. experience
What’s the secret?
Enter Convincely

15. experience

16. experience
Evan Shellshear
evan@ebex.com.au
We’re Hiring!
THANK YOU
Yours?
Battle-hardened funnels