Your analytics will give you the precise details of what is happening on your site, where the problems are, and when they are most likely to happen, but analytics can’t tell you “why”. You have drop-off after testing several checkout flows, how do you diagnose the problem? Real-time A|B testing on the home page reveals lower conversion, but you don’t know why. Your landing pages have high bounce rates after version 6. This presentation outlines some of the key analytic findings that require user experience research to deliver the insights necessary for resolution. Analysts will take away specific understanding of how user research can be used to provide solutions to key problems of engagement, adoption, and retention. Key Points: + Diagnostic research + Bounce investigation + Driving engagement + User flow optimization

1. Uncovering the Why Behind Web Analytics
The Bi Design Conference
Th Big D i C f May
M 2010

2. What do I do?
learn
l optimize
ti i validate
lid t
CONFIDENTIAL 2

3. CONFIDENTIAL 3

4. Who Are You?
CONFIDENTIAL 4

5. CONFIDENTIAL 5

6. What’s in Store . . .
Triangulating the U
Ti l i h User E
Experience
i
How to Diagnose 3 Common Symptoms
Group Exercise: The 4th Symptom
CONFIDENTIAL 6

7. Meet Peter Griffin
Your Website
© Fox
CONFIDENTIAL 7

8. Peter as a Statistic . . .
Data Type Peter’s Experience
Referring URL: Google
First Time or Repeat Visitor: Repeat
Total Page Views in Session: 20
Time Spent: 10 minutes
Types of Pages Visited: ‘Confabulator’ Category Pages
# of Searches in Session: 3
Exited to: Google
CONFIDENTIAL 8

9. How Peter Really Feels . . . Wounded by Your Site
y y
© Fox
CONFIDENTIAL 9

10. Peter’s Actual Experience
p
Never found the confabulator
50% of the page views were to the
SAME page
p g
Exhaustive review, gave up, and
finally searched
Confabulator was actually on 5 of the
pages he visited
© Fox
Swears he’ll never return
CONFIDENTIAL 10

11. What You Don’t Know About Peter . . .
How did he
Did h
he
interpret what
accomplish what
he saw?
he came to do?
What did he
notice?
Was he satisfied?
What is he
looking for?
Will he come
Why is he back?
visiting?
© Fox
CONFIDENTIAL 11

12. Triangulating the User Experience
g g p
Do
Context
Think See
CONFIDENTIAL 12

13. Tools to Diagnose Symptoms
CONFIDENTIAL 13

14. Symptom #1: High Drop-off in Process Flow
y p g p
Step 1 Step 2 Step 3 Step 4 Step 5
100% 80% 40% 35% 30%
CONFIDENTIAL 14

15. Example: Automotive Website
p
Recent redesign of 40% of site visitors
quote request dropped off the flow
p
process had at dealer section
unintended outcome page
CONFIDENTIAL 15

16. What Do You Need to Learn?
How did Why did What else
they they leave can be
interpret it? improved?
the dealer
selection
page?
CONFIDENTIAL 16

17. Panel-Recruited Quantitative Benchmark
They complete
tasks on . . . We track their
Your Site behavior and
trigger
200+ panelists
q
questions
representative of
your target market
along the way
are recruited from
GMI, SSI, or Toluna
CONFIDENTIAL 17

18. How Does It Work?
• Behavioral tracking via ActiveX download (Keynote / UserZoom)
• 20-30 minutes with 30 to 50 questions
• Quantitative user experience metrics
• Qualitative insight through comments and open-ends
• Triggered questions determine the ‘why’ for drop-off
• 4-6 weeks total
CONFIDENTIAL 18

19. Drop-Off on Task Mirrored Analytics
p y
1 2 3 4 5
Contact Purchasing Review and Thank You
Information Select Dealer Information submit
337 312 195 192 190
7% 38%
drop-off drop-off
CONFIDENTIAL 19

20. Why did users end the p
y process p
prematurely?
y
Customer Information Page
7% Drop-Off
11% of users did not want t provide
f t t to id
phone and address (not clear to all
that information was optional)
“Too much info. email, phone, when I'm
looking to purchase. . . then they want
my home phone so it makes me feel
like they will start hassling me at home
to purchase a car. With this kind of ‘feel’
from a web site. I would normal stay away
and purchase/ research my car a different
way.”
CONFIDENTIAL 20

21. Why did users end the p
y process p
prematurely?
y
25% thought Step 3 was
the last step of the
process
Page gave users
typical confirmation
message too soon in
the process
Select Dealer Page: 38% Drop-off
Drop off
No clear call to action
or next step highlighted
in the main content
zone of the page;
select dealer button not
strong enough
CONFIDENTIAL 21

22. Qualitative Insights re: Look & Feel Confusion
g
1 in 5 participants expressed frustration around the change in look
and feel of the site
“When I selected that button I was taken to a completely different looking page. The page
to enter personal information looked too different from the customizing page. At first I thought I
page
did something wrong when an entirely different page popped up.”
“The look and feel of the two sites are very disjointed I felt as if I was being redirected and
t o er disjointed. as
that I may be in the wrong area. A more seamless transition between the sites would be
recommended.”
CONFIDENTIAL 22

23. Key Recommendations
y
ontent
Trust
gation
Visual Design
Make Eliminate Explain why Ensure
progress misleading personal seamless
Navig
Co
indicator
i di t confirmation
fi ti information
i f ti transition
t iti
D
more messages is required; between
prominent clarify applications
optional info
CONFIDENTIAL 23

24. Symptom #2: Low Click-thru on Promo / Low
Usage of Feat re
Feature
Only 2% of all visitors to this
page visit promo / use feature
It’s the least visited area of the
? page
CONFIDENTIAL 24

25. Why Low Usage?
y g
They don’t notice it.
They don’t need it.
They don’t like it.
CONFIDENTIAL 25

26. Example: Peripheral Manufacturer
p p
Unsure how to
Data revealed low
prioritize content
consumption of
development for
video content on
third-party
third party shopping
partner sites
sites
CONFIDENTIAL 26

27. What Do You Need to Learn?
Do they Do they How can
notice value the video
video video content be
content? content? improved?
CONFIDENTIAL 27

28. One-on-One Lab Study with Eye Tracking
y y g
Visit your site
via Tobii We track
their
th i eye
movements
Target users come and
to the lab and . . . impressions
.
CONFIDENTIAL 28

29. How Does It Work?
Eye-Mind Hypothesis
ye d ypot es s
where people look is
where they focus their
attention
Saccades
rapid eye movements
p y
Fixations
gaze focused on one
point
CONFIDENTIAL 29

30. Eye Tracking Metrics
y g
% Participants # of users who fixated on the AOI at least once
Fixation Count # of fixations within an AOI
Observation
# of times users looked at the AOI
Count
Time to First
Amount of time it took from initial exposure to see the AOI
Fixation
Fixation Length Length in seconds of fixations within AOI
Observation
Total amount of time users looked at the AOI
Length
Fixations Before # of fixations that occurred before a users fixates on AOI
CONFIDENTIAL 30

31. Do they notice video content?
y
Overwhelmingly - NO
CONFIDENTIAL 31

32. Gaze Plots
# of items shoppers look at on a product page before they start reading
manufacturer-provided video product content
59 70 290!
CONFIDENTIAL 32

33. The Experience Gap
p p
“The reviews on it were really
great, the looks of it is good… Plus
it has a touch screen… I just hope
that this is user friendly… and
that’s where I d probably go to the
h h I’d b bl h
store and actually demo it myself
and operate it to see how well it
d t t h ll
functions.”
CONFIDENTIAL 33

34. Remaining Questions. . .
g
“What is it like to hold the product?”
“What’s the installation or setup process?”
“What happens when I plug it in?”
“What is it like to use the product?”
“Will this work in the way that I want to use it?”
CONFIDENTIAL 34

35. Key Recommendations
y
Volume
cement
Content
Make video Develop State
content more demos benefits
more and videos early and
V
Plac
C
discoverable that often
illustrate the
experience
with the
p
product
CONFIDENTIAL 35

36. The Symptom #3: High Bounce Rates
y p g
LEAD
PRODUCT GENERATION
HOME PAGE
DETAIL PAGE PAGE
65% Bounce 55% Bounce 85% Bounce
CONFIDENTIAL 36

37. Case Study: Higher Education Microsite
y g
Bounce rates from
Invested millions
landing pages and
of dollars in online
lead pages were
ad campaign
VERY hi h
high
CONFIDENTIAL 37

38. What Do You Need to Learn?
Who is What Why did
bouncing? were they they
seeking? leave?
CONFIDENTIAL 38

39. Tool A: Post-Site Visit Intercept Survey
p y
Your Site Other Site
Visitor Survey
Survey
leaves your Invitation
Invitation
Pop Under site
• Brief survey; no more than 10 questions
• Reliable occurs at point of bounce
• Small line of Java Script added to page
• Requires large site traffic numbers / longer field time
• Sweepstakes recommended
p
CONFIDENTIAL 39

40. Composition of Prospect Microsite Bouncers
p p
Prospective student 62%
Current professionals,
p ,
Other 23%
students on the cusp of
enrolling, and friends of
Current student -- other
institution 13% prospective students
Faculty member -- other
institution 1%
Parent or legal guardian 1%
CONFIDENTIAL 40

41. Reasons for Prospects’ Visit to Microsite
p
To learn more about higher education
30%
programs
To receive a free learning assessment 21%
Other 20% Showing the site
to a friend,
looking for
To request information about enrolling in financial aid
fi i l id
19%
higher education programs
information,
finding
To learn more about advancements in information
16%
higher education
about a specific
b t ifi
program, and
To learn about receiving educational credit
9% learning about
for previous work / life experiences
fees.
CONFIDENTIAL 41

42. Prospects’ Reasons for Bounce from Microsite
p
20% of users were unable to find 15% said the site didn’t match
enough course information on what they initially clicked
the microsite
“I did not intend to visit; it was an
I
“I couldn't find what I was looking accident.”
for.”
“It’s not the site I want to be at ”
It s at.
“Cannot see details at all. Vague info
available.”
“I thought it was a site that was
geared towards employees directly ”
directly.
“I am looking for a number.”
CONFIDENTIAL 42

43. Prospects’ Reasons for Bouncing from Lead Page
p g g
Not enough info received to warrant submitting a lead
g g
“I'm in no hurry... will browse catalogue offline and see if there are any degree
courses appropriate for myself and re-visit if appropriate.”
“Need more information, confirm that a degree from this institution is going to get
Need
me where I want to be.”
Many felt too much personal info was required up front
“Caution about leaving any information. I have worked in IT for 10 years and know
that nothing is 100% secure, so as little information as possible should be sent.”
“I don't want to give out info without having gained something first.”
“I don’t want to give out my info just yet.”
CONFIDENTIAL 43

44. Key Recommendations
y
Microsite
Lead Page
Lead Page
Provide Develop Give
more hooks prominent consumers
to site paths to control over
content on more how they’re
all landing information contacted
pages from lead
pages
CONFIDENTIAL 44

45. Tool B: One-on-One Usability Testing
y g
User visits
your site / Your Site
landing page
Online after initial
Ads exposure to
online ad
• Best for testing hypotheses
• Focus on page content and expectations
• Blind, experimental lab-based study
• Used when time is of the essence
CONFIDENTIAL 45

46. Example: Pharmaceutical Microsite
p
Most participants had trouble
understanding the value
proposition of the microsite Your Online Ad
Y O li
“I expected to see a introductory
offer based on the ad I clicked but
this page also just seems like
another ad. Where are the special
offers?”
“I don’t understand what this site is. .
. It has nothing to do with what I
clicked.
clicked ” Your Website / Microsite
CONFIDENTIAL 46

47. Summary of Tools to Diagnose Analytics
y g y
Site Traffic
Research Tools Who?
Required
Quantitative Panel-Based Study N/A Target Audience
Eye Tracking Study N/A Target Audience
Intercept Survey Large Actual Site Visitor
1:1 Usability Testing N/A Target Audience
CONFIDENTIAL 47

48. Symptom #4: A/B Testing Reveals Lower
Conversion
Con ersion
OLD NEW
HOME PAGE HOME PAGE
A B
25% Click the 2% Click Calls
Calls to Action to Action
CONFIDENTIAL 48

49. How would you diagnose Symptom 4?
y g y p
Site Traffic
Research Tools Type
Required
Quantitative Panel-Based Study N/A Experiment
Eye Tracking Study N/A Experiment
Intercept Survey Large Real-Time
1:1 Usability Testing N/A Experiment
CONFIDENTIAL 49

50. CONFIDENTIAL 50

51. CONFIDENTIAL 51

52. Triangulating the User Experience
g g p
Do
Context
Think See
CONFIDENTIAL 52

53. Questions?
Amy Buckner
Managing Partner & Co-Founder
415-977-0303
amy@answerlab.com
amy@answerlab com
CONFIDENTIAL 53