Learn why research is important for successful CRO programs and how it should be activated to ensure that testing is fueled with actionable insights. In this webinar, Daniel Gold, Head of Research at House of Kaizen, a digital agency, shares top tips and recommendations on how behavioral and attitudinal research methods can be deployed to see measurable business growth.

1. Accelerate Your Conversion Optimization Program
With Attitudinal & Behavioral Research
Vipul Bansal
Co- Marketing Manager,
VWO
Daniel Gold
Head of Research,
House of Kaizen

2. Agenda
Speaker
Our clients acquire and retain more
revenue from customers through
Digital Experience Optimization
Daniel Gold -
Head of Research
/gold-daniel
Test and learn to delight and grow.
We design, deliver and optimize incredible experiences for
your customers.
● Conversion rate optimization
● Usability
● Customer retention
● User & customer insight
© House of Kaizen 2019

3. ● Why CRO research is important
● How to do successful CRO research
● Which CRO research methods
● Takeaways
Agenda
© House of Kaizen 2019

4. CRO research
is important

5. Simple things can be learnt by trial & error
Nail: One element
Options: A couple ways to try nailing it in
© House of Kaizen 2019

6. Website optimization has too many elements/options
for trial & error
Imagine...
A small website can have 100 elements (sentences, images, icons). Randomly changing
elements to discover which are the 20% that a typical user will see.
5 e+20 variations 4 e+11 stars in milkyway
(400 billion)
© House of Kaizen 2019

7. NEVER
User interface
optimization
© House of Kaizen 2019

8. Goals &
Attitudes
Product
knowledge
Identity
Decision
process
Users’ Bubbles
Users and marketers live in their own worlds
User
Behaviour
Bubble
Marketers
Experiences
© House of Kaizen 2019

9. Example - Usability test on a cruise
website
● Feel the website looks ‘too conservative’.
● Can’t understand basic terminologies.
(‘flight cruises,’ ‘excursions’).
● Can’t see any benefits.
● High stress levels when deciding.
Users new to cruise holidays (novices):
Marketers who designed the website (experts on cruises)
think the website is appealing, clear and full of USPs!
© House of Kaizen 2019

10. How research increases test success
Our agency’s A/B test performance since focus on research
16%
Average conversion uplift per test
+100%
83%
Percentage of green tests
+65%
Year on year comparison
2015/16 vs 2017/18
© House of Kaizen 2019

11. Successful
CRO research

12. Test
Ideas/hypotheses
1
2
3
4
5
Ideas
prioritization
Testing
Strategy
Blueprint &
Wireframes
Design &
Copywriting
Code, QA
& Launch
$
Test
Results
2. Test performance research
When to research in a testing process
1. Ad-hoc
research
© House of Kaizen 2019

13. Making research more actionable
Delivery: Joint test ideation workshops
between research, CRO, design &
development.
Planning: Designing every piece of research
with potential test applications in mind.
Multi-Method: Using both behavioral and
attitudinal methodologies.
What
users do
What
users
think &
feel
Behavioral
research
Attitudinal
research
© House of Kaizen 2019

14. User research
Actionable insights
Behavioral economics
User behavior theory
Test strategy
Combining two sources of knowledge
Actionable Insights
Attitudinal research
Behavioral research
Research becomes more actionable when it’s merged with user behavior theory
© House of Kaizen 2019

15. 3
What does this pattern mean?
Which actions to take?
Example: Eye tracking data
Users landing on an antivirus shop (first 5 seconds on page)

16. 3
Masthead Eye Tracking Data-compatibility to system
-store match to personality
1.
2.
3.
Three cognitive steps:
Identified with eye tracking (behavioral) + theory + association tests (attitudinal)
Example: Landing on an antivirus shop
© House of Kaizen 2019

17. ● Thought/reasoning
● Slow
● Rational
● Rich details
System 2System 1
Behavioral economics: Two decision making systems
● Intuition
● Fast
● Emotional
● Easy & quick cues
5 seconds on page = system 1 (fast)
© House of Kaizen 2019

18. Personalized device
compatibility cues
Lifestyle cue
Brand cue
Strategy: Use System 1 and make desired information easily accessible
Affordance
Masthead test addressing the 3 cognitive stages
Windows 10:
+26.61% CR
© House of Kaizen 2019

19. CRO research methods

20. Quantitative & Qualitative
On-site & Offsite
Classification of
research methods
Attitudinal &
Behavioral
© House of Kaizen 2019

21. What
users do,
think &
feel
What
users do
What
users
think &
feel
Attitudinal research
• Pop-up surveys
• Passive feedback
• On-site questionnaires
• Targeted social surveys
• Interviews
(emphatic/in-depth)
• Focus groups
• Association tests
• Social media listening
Mixed research
• Customer journey
mapping
• Usability testing
• Diary studies
• Customer journey
mapping
• Facial coding
• Galvanic skin response
• Pupil dilation
Behavioral research
• Traffic analytics
• Heatmapping
• Session recordings
• Eye tracking
• Big data
CRO research methodologies
Recommended
minimum
Our agency uses all
methods to get
bigger uplifts, as
methods have
different strengths
and weaknesses
© House of Kaizen 2019

22. Backbone behavioral research methods
● Conversion rates
● Paths to conversion
● Drop off points
● Level of engagement
which page elements
(scrolling, hovering,
clicks)
● Order of page element
engagements
● Engagement durations
Session recordingsHeatmappingTraffic analytics
Custom set up to track scrolling and
clicks, Time on page unreliable
Poor tracking of engagement
durations and order
Small samples
© House of Kaizen 2019

23. Main conversion path Pathological path
Convert
Drop Off
Finding: Most visitors convert via a
100 days trial content page.
1. 2.
Hypothesis: Driving more traffic to
the trial page will increase
conversion.
+ 7.4% CR
3. Solution: Adding links to the trial page.
Traffic analytics example - mattress shop
Home
page
Trial page
© House of Kaizen 2019

24. Finding: 30% of visitors interact
with the magazine image (click
and/or mouse hovering).
1.
2.
Clicks
Mouse movements
Hypothesis: Visitors interact with the
magazine image because of a preconception
learned from website convention, to expect
additional content on such interaction.
3.
+ 10.8% CR
Solution: Magazine preview in
a light box, triggered on
magazine image click.
Heatmapping example - magazine shop
© House of Kaizen 2019

25. Multiple choice
Open questions
Rating scales
Attitudinal research: Pop-up surveys
● Content gaps
● Reasons to visit a site
● Needs/ interests
Certain traffic volume are required, as response
rates are typically between 0.1% and 1%
Measuring thoughts and feelings which are accessible
while on a page/app
© House of Kaizen 2019
vs passive feedback which
find mainly website bugs

26. Pop-up finding: 25% of visitors read
business pages to aid decision
making.
1. 2.
3.
Design Principles Different delivery
mechanisms of the value proposition
(discrete, contrasting or intrusive)
are possible.
Solution: Pop-up
With relevant value
proposition and
image.
+ 112.4%
visits to the
store
Pop-up survey example - newspaper editorial
© House of Kaizen 2019

27. ● Research is very important, as trial & error/intuition
do not work for optimizing websites.
● Successful CRO research needs to be actionable,
especially through combining behavioral & attitudinal
methods and connecting to consumer behavior theory.
● There is a tremendous amount of research methods
which work well for CRO. We recommend to use as a
minimum: traffic analytic, heatmapping, session
recordings, and pop-up surveys
Takeaways
© House of Kaizen 2019

28. Thank You!
Q&A