This webinar discusses how user experience (UX) research and usability testing can positively impact brands. It advocates for mixing quantitative and qualitative methods to understand both what customers think and do. The speaker recommends identifying key performance indicators, building customer personas, measuring UX quality, analyzing top tasks, benchmarking experiences, and linking activities to metrics. Mixing methods like surveys, usability testing, and analytics can help measure improvements. Attendees learn how data-driven UX approaches can benefit marketing by linking the user experience to brand attitudes and business outcomes.

1. Empowering Data-Driven Marketers:
How UX Research & Usability Testing Can Positively
Impact Your Overall Brand Experience
Featuring Jeff Sauro

2. Quick Housekeeping
• Control panel on the side of your screen if you have any
comments during the presentation
• Time at the end for Q&A
• Today’s webinar will be recorded for future viewing
• All attendees will receive a copy of the slides/recording
• Continue the discussion using #uzwebinar

3. 2016 goals
1. Provide more UX resources to UX
professionals.
2. Improve brand consistency.
3. Improve “International” UXPA.
User Experience Professionals
Association (UXPA)
Annual Conference in Seattle, WA
May 31 – June 3
uxpa2016.org
Monthly webinars
User Experience Magazine
The Journal of Usability Studies
Visit www.uxpa.org for more info about
the conference, webinars, and other UX
resources!

4. Measuring U 2016 4
UserZoom at a Glance
• All-in-one solution to rapidly test Usability & Measure
UX
• Remote Unmoderated Usability Test (Qualitative or Think-
out-loud)
• Usability Benchmarking
• Card Sorting
• Tree Testing
• Screenshot Click Testing
• Screenshot Timeout Testing
• Online Surveys
• Test any digital UI at anytime
during the product life cycle
The All-in-One UX Research Platform for the Data-Driven Enterprise

5. Measuring U 2016 5
Meet the Speaker
Jeff Sauro PhD
Founding Principal | MeasuringU
Measuringu.com
@MeasuringU
BS Syracuse University
MA Stanford University
PhD University of Denver
IT & TV/Radio Film
Learning Design & Technology
Research Methods & Statistics

6. Measuring U 2016 6
That’s Not What UX does…That’s Not Marketing’s Job!
MarketingUX David Ricardo
Law of Comparative
Advantage

7. Measuring U 2016 7
Division of Labor
Marketing UX
Quantitative Qualitative
Measures branding and satisfaction Measures behavior (if they measure anything)
Segmentation Personas
Considers what customers say Considers what customers do
Quantitative Qualitative

8. Measuring U 2016 8
Mixing Methods & Mindsets
• It’s OK to mix quantitative and qualitative
• Don’t be afraid to use surveys and observation
• Understand how the user experience affects brand attitudes
and vice versa.
• Measure what people think and what they do, often in the same
study.

9. Measuring U 2016 9
4.7 5.35.3 4.5
1
2
3
4
5
6
7
Budget Enterprise
Pre
Post
Website Experiences Affect Brand Attitudes
Enterprise Brand favorability
ratings dropped by 15%
after users attempted to rent
a car.
VeryFavorableVeryUnfavorable
12% 15%

10. Measuring U 2016 10
Mix Methods and Mindset
Steps
1. Identify and collect Key Performance Indicator (KPIs)
2. Uncover your customer segments and build personas
3. Collect a measure of UX Quality
4. Understand customers’ top tasks
5. Benchmark the Experience and link to KPIs
6. Mix quant and qual. methods to improve
7. Measure again
8. Link to changes in KPIs
Traditional “Owner”MarketResearchUXResearch

11. Measuring U 2016 11
1. Identify & Collect the Key Performance Indicators (KPIs)
Branding Questions
• Hotels.com is a Brand I can trust.
• When thinking about renting a car, which company is your first choice?
Satisfaction (By Touchpoint)
• How satisfied were you with installing the product?
• How satisfied were you with recent call to customer support ?
Loyalty
• How likely are you to recommend to a colleague or friend?
Future Intent
• How likely are you to return and book a hotel on this website?
Look to leverage existing company KPIs
These are what you want to
IMPROVE

12. Measuring U 2016 12
2. Uncover customer segments and build personas
Key Differentiators
Age
Online Comfort
Income
Zillow Trust
Realtor Trust
Low Hig
h
Low Hig
h
Low Hig
h
Low Hig
h
Low Hig
h
“I don’t believe in it. I’d rather
make large monetary exchanges
face-to-face.”
Demographics
Age: 57 years old
Income: $50K-$100K
Ethnicity: Caucasian
Children: No children living in the home
Marital Status: Divorced
Next home purchase: 2+ years
First home purchase: 20 years ago
23%
60%
18%
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
N = 1200
Personas should come from large scale surveys and in depth interviews.

13. Measuring U 2016 13
3. Collect a measure of Website UX Quality : SUPR-Q
USABILITY CREDIBILITY (TRUST, VALUE & COMFORT)
LOYALTY APPEARANCE
• It is easy to navigate with this website
• This website is easy to use
• I will likely visit the website in the future
• How likely are you to recommend this website to
a friend or colleague? [NPS]
• The information on this website is trustworthy.
• The information on this website is credible.
• The website has a clean and simple presentation
• I found the website to be attractive
Standardized User Experience Percentile Rank Questionnaire
More at suprq.com

14. Measuring U 2016 14
4. Find what customers want to do: Top Task Analysis
Separate the trivial many from the vital few
TOP TASK ANALYSIS
Vital few tasks
Trivial Tasks

15. Measuring U 2016 15

16. Measuring U 2016 16
Automotive Websites Top Tasks
Pick Only Five

17. Measuring U 2016 17
Automotive Website: 94 Tasks
0%
5%
10%
15%
20%
25%
30%
Trivial “Long Tail” Tasks
Long Neck
How many miles per gallon will a car get?

18. Measuring U 2016 18
Top Tasks: Health Insurance Website
0%
1%
2%
3%
4%
5%
6%
7%
“Find someone close to my home/office”
“See if my healthcare professional will accept my insurance”
“See if a particular doctor is in my plan”

19. Measuring U 2016 19
5. Benchmark The Experience: SUPRQ
Error bars represent 90% confidence intervals | N: 394
81% 76% 79% 75% 75% 35%
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
SUPR-Q Usability Trust, Value & Comfort Loyalty Appearance NPS (Raw %)

20. Measuring U 2016 20
Task Data
Error bars represent 90% confidence intervals | N: 394
70% 61% 62% 63%
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Search Browse Checkout Fulfillment
These are your top-tasks per user
segment.

21. Measuring U 2016 21
Link activities to KPIs
How likely are you to book a hotel on your next visit?
0
0.05
0.1
0.15
0.2
0.25
0.3
0.35
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Importance(BetaCoefficient)
Percent of Maximum Score
Brand Awareness
Trust in Price
Website Usability

22. Measuring U 2016 22
Link activities to KPIs
How likely are you to recommend this website to a friend? [Categorized Verbatim Responses]
NPS %
34% 18% 10% 7% 7% 6% 5% 4% 4% 3% 3%
0%
5%
10%
15%
20%
25%
30%
35%
40% Simplify/Ease/Nav
SpecificSuggestions
Design
Ads
Conditions&Opinions
Search
Speed
Readability
Comparisons
ZipCodes
SurveyComments
-19% 0% 45% 100% -25% -29% 80% 50% 25% 0% 0%

23. Measuring U 2016 23
6. Mix Quant & Qual. Methods to Improve
CARD SORTING
HEURISTIC
EVALUATIONS
FIRST CLICK
TESTING
KEYSTROKE
LEVEL MODELING
TREE TESTING
AnalyticEmpirical
USABILITY TESTING

24. Measuring U 2016 24
6. Compare After Changes: SUPR-Q Year-Over-Year
Error bars represent 90% confidence intervals | N: 394, 148
Statistical improvement in UX Quality Scores
81% 76% 79% 75% 75% 35%95% 91% 98% 87% 94% 62%
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
SUPR-Q Usability Trust, Value & Comfort Loyalty Appearance NPS (Raw %)
Desktop 12/13
Desktop 15/16

25. DATA COLLECTION

26. Measuring U 2016 26
Data Collection Methods
WEBSITE INTERCEPTS
TARGETED SURVEYS
Leverage existing survey efforts where possible.

27. Measuring U 2016 27
Typical Combined Study Flow in UserZoom
Welcome Page Segmentation & Branding Questions Task Description
Tasks Post Task Questions Post Study Questions

28. Measuring U 2016 28
Integrate Survey Data with Website Behaviors
UserZoom Google Analytics

29. Measuring U 2016 29
Mix Methods and Mindset
Steps
1. Identify and collect Key Performance Indicator (KPIs)
2. Uncover your customer segments and build personas
3. Collect a measure of UX Quality
4. Understand customers’ top tasks
5. Benchmark the Experience and link to KPIs
6. Mix quant and qual. methods to improve
7. Measure again
8. Link to changes in KPIs
Traditional “Owner”MarketResearchUXResearch

30. Learn More
London June 16th
Creating a Plan to Measure the User Experience
Hosted By UserZoom @ Altitude 360 London
Rome June 20th-22nd
UX Boot Camp
Measuringu.com/italyux.php
Chicago July 15th
Quantifying the User Experience (Half Day Workshop)
Measuringu.com/chicago.php
Denver Aug 17th-19th
UX Boot Camp
denverux.com

31. Q&A

32. Need more info?
Get answers about UserZoom. We are here to help.
Talk with one of our experts.
Contact us
UserZoom
10 Almaden Blvd. #250
San Jose, CA 95113
Toll free number: (866) 599 1550
www.linkedin.com/company/userzoom@UserZoomFollow us