This presentation was a part of the 2016 Cleveland AMA Market Research Conference.

1. Market Research and UX:
Birds of A Feather or
A Cat Among the Pigeons?
Tom Bubnick, Manager User Experience and User Interface
Kristen Deevers, Senior Manager Consumer Insights
Cleveland AMA Market Research Conference
April 21, 2016

2. “If market research tells us
what consumers want....
usability helps us learn
how to deliver on that
promise and when we have
succeeded.”
Source:WQusability

3. Market research, which includes social and opinion
research, is the systematic gathering and interpretation of
information about individuals or organizations using
statistical and analytical methods and techniques of the
applied social sciences to gain insight or support decision
making.
Source: ESOMAR
User experience (UX) is an approach to product
development that incorporates direct user feedback
throughout the development cycle (human-centered
design) in order to reduce costs and create products and
tools that meet user needs and have a high level of usability
(are easy to use).
Source: User Experience Professionals Association (UXPA)

5. Birds of A Feather…
Both MR and UX are concerned with the unbiased,
systematic collection of data to answer business questions
An innate curiosity about people, what they do, their
motivations and how they do things is helpful to success in
both roles, though these skills may be applied differently
Observation is a key methodology used by both MR and UX
But key differences also play important
roles in understanding both disciplines

7. Or, A Cat Among the Pigeons…
Market Research User Experience
Primary focus on what people SAY Primary focus on what people DO
Uses QUANTITATIVE measurement methods
(i.e. surveys) with large sample sizes to
project findings to larger overall populations
Uses QUALITATIVE methods (i.e., focus
groups or one on ones) to dive deeper with
a smaller number of respondents to probe
underlying perceptions, attitudes and
beliefs; research may be conducted in
homes, stores, focus group facilities, etc
Uses EXPERIENCE methods to focus on
practical, measurable actions and responses
that occur around specific experiences and
what people actually DO when trying to
accomplish certain tasks
Often used to help validate or set strategy,
validate concepts/designs, get input on
pricing, identify areas for focus or growth
Focuses on the design of an experience,
product or website and involves speaking to
a much smaller number of users

8. Or, A Cat Among the Pigeons…
Market Research User Experience
Primary focus on what people SAY Primary focus on what people DO
Uses QUANTITATIVE measurement methods
(i.e. surveys) with large sample sizes to
project findings to larger overall populations
Uses QUALITATIVE methods (i.e., focus
groups or one on ones) to dive deeper with
a smaller number of respondents to probe
underlying perceptions, attitudes and
beliefs; research may be conducted in
homes, stores, focus group facilities, etc
Uses EXPERIENCE methods to focus on
practical, measurable actions and responses
that occur around specific experiences and
what people actually DO when trying to
accomplish certain tasks
Often used to help validate or set strategy,
validate concepts/designs, get input on
pricing, identify areas for focus or growth
Focuses on the design of an experience,
product or website and involves speaking to
a much smaller number of users

9. Or, A Cat Among the Pigeons…
Market Research User Experience
Primary focus on what people SAY Primary focus on what people DO
Uses QUANTITATIVE measurement methods
(i.e. surveys) with large sample sizes to
project findings to larger overall populations
Uses QUALITATIVE methods (i.e., focus
groups or one on ones) to dive deeper with
a smaller number of respondents to probe
underlying perceptions, attitudes and
beliefs; research may be conducted in
homes, stores, focus group facilities, etc
Uses EXPERIENCE methods to focus on
practical, measurable actions and responses
that occur around specific experiences and
what people actually DO when trying to
accomplish certain tasks
Often used to help validate or set strategy,
validate concepts/designs, get input on
pricing, identify areas for focus or growth
Focuses on the design of an experience,
product or website and involves speaking to
a much smaller number of users

10. Or, A Cat Among the Pigeons…
Market Research User Experience
Primary focus on what people SAY Primary focus on what people DO
Uses QUANTITATIVE measurement methods
(i.e. surveys) with large sample sizes to
project findings to larger overall populations
Uses QUALITATIVE methods (i.e., focus
groups or one on ones) to dive deeper with
a smaller number of respondents to probe
underlying perceptions, attitudes and
beliefs; research may be conducted in
homes, stores, focus group facilities, etc
Uses EXPERIENCE methods to focus on
practical, measurable actions and responses
that occur around specific experiences and
what people actually DO when trying to
accomplish certain tasks
Often used to help validate or set strategy,
validate concepts/designs, get input on
pricing, identify areas for focus or growth
Focuses on the design of an experience,
product or website and involves speaking to
a much smaller number of users

11. UX & CI Across the Organization

12. Personas:
Summarizing Your
Customers

13. Personas are archetypes that describe the various goals and
observed behavior patterns among our potential users and current
customers. Personas focus on user goals, current behavior, and pain
points (as opposed to their buying or media preferences and
behaviors).
Definition of Persona

14. Aren’t Those Just Market Segments?!
Source: Designing for the Digital Age: How to Create Human-Centered Products and Services
Kim Goodwin, Alan Cooper

15. User Experience
Identify participant
selection criteria
&
Informs design
decisions
Development
Creates empathy
towards customer
&
Prioritizes bug fixes
Social / Content
Management
Informs editorial
calendar
&
Identifies gaps in
strategy
Product
Input for and
priority of features
Quality Assurance
Useful when generating
test scenarios
Personas: The Who & How
Icons by Richard Schumann from the Noun Project

16. Persona Narrative & Imagery

17. Demographics, Goals, Barriers, Etc.

18. Journey Mapping:
Walking in the Customer’s Shoes
Illustration by Lukasiniho

19. Definition of Journey Mapping
• The activity of evaluating a system, in its entirety, to
gain a comprehensive view of peaks and valleys in
the experience
• Gain insight into customer actions, motivations,
feelings, and barriers
• Build consensus via a cross functional team through
the activity of mapping an experience as a group
• Focuses improvement efforts on key moments and
helps prioritize

20. A Story: Doug & His Machine
Source: DesigningCX.com

21. Sofia & Her MRI Experience
• 6 ½ years old
• After a week of intense headaches, her
family doctor orders an MRI scan
• Had to wait 1 week to have the scan
• When she enters the room with the MRI
machine, she starts crying and resisting
• After a few inconsolable minutes, she is
sedated… like 85% of all child patients
for this procedure
• Doug is visiting and sees this unfold. His
machine is a success but the experience
is a failure.

22. GOES TO
FAMILY
DOCTOR
RIDES TO
HOSPIT’L
CHECKS
IN
WALKS
TO MRI
ROOM
SEES MRI
MACHINE
CRIES &
RESISTS
SEES
DOCTOR
ENTER
GETS
DRUGGED
GOES
THROUGH
MRI
………
AM I
REALLY
SICK?
THAT
LOOKS
SCARY!
WILL IT
HURT
ME?!
MOM
CAN’T
HELP?
NO! NO!
PLEASE
NO!
I’M
REALLY
NERVOUS
WISH I
WAS AT
SCHOOL
TECH.
WRITER
SYSTEM
ADMIN.
HOSPIT’L
BLDG
MGR
DOUG
PATIENT
SAFETY
TEAM
STAFF
PAGING
SYSTEM
PATIENT
RECORD
SYSTEM
RESERV.
SYSTEM
IMAGING
RECORD
SYSTEM
DRUG
ROOM
Behaviors
ATTITUDES
PEOPLE
THINGS
ONSTAGE
EXPERIENCE
PEOPLE
THINGS
BACKSTAGE
SUPPORT
Select a specific customer
persona to map
DOCTOR MOM
CHECKIN
NURSE
IMAGING
TECH
MOM
DOCTOR
ANESTH.
IMAGING
TECH
MRI
MACHINE
DESK &
COMPUT.
CHARTCAR
TAKE-
HOME
PACKET
MRI
MACHINE
NEEDLE &
DRUGS
NURSE
TISSUES

23. GOES TO
FAMILY
DOCTOR
RIDES TO
HOSPIT’L
CHECKS
IN
WALKS
TO MRI
ROOM
SEES MRI
MACHINE
CRIES &
RESISTS
SEES
DOCTOR
ENTER
GETS
DRUGGED
GOES
THROUGH
MRI
………
DOCTOR MOM
CHECKIN
NURSE
IMAGING
TECH
MOM
DOCTOR
ANESTH.
IMAGING
TECH
MRI
MACHINE
DESK &
COMPUT.
CHARTCAR
TAKE-
HOME
PACKET
MRI
MACHINE
NEEDLE &
DRUGS
AM I
REALLY
SICK?
THAT
LOOKS
SCARY!
WILL IT
HURT
ME?!
MOM
CAN’T
HELP?
NO! NO!
PLEASE
NO!
I’M
REALLY
NERVOUS
WISH I
WAS AT
SCHOOL
TECH.
WRITER
SYSTEM
ADMIN.
HOSPIT’L
BLDG
MGR
DOUG
PATIENT
SAFETY
TEAM
STAFF
PAGING
SYSTEM
PATIENT
RECORD
SYSTEM
RESERV.
SYSTEM
IMAGING
RECORD
SYSTEM
DRUG
ROOM
NURSE
TISSUES
IMPACT MORE
STAFF
COST OF
DRUGS
IMAGING
TIME
SIDE
EFFECTS
Evaluate & Prioritize
Identify moments that matter for the customer and the organization

24. Evaluate & Prioritize
Re-examine issue / opportunity based on deep customer understanding

25. GOES TO
INITIAL
DOCTOR
RIDES TO
HOSPIT’L
CHECKS
IN
WALKS
TO MRI
ROOM
SEES MRI
MACHINE
CRIES &
RESISTS
SEES
DOCTOR
ENTER
GETS
DRUGGED
GOES
THROUGH
MRI
………
DOCTOR MOM
CHECKIN
NURSE
IMAGING
TECH
MOM
DOCTOR
ANESTH.
IMAGING
TECH
MRI
MACHINE
DESK &
COMPUT.
CHARTCAR
TAKE-
HOME
PACKET
MRI
MACHINE
NEEDLE &
DRUGS
AM I
REALLY
SICK?
THAT
LOOKS
SCARY!
WILL IT
HURT
ME?!
MOM
CAN’T
HELP?
NO! NO!
PLEASE
NO!
I’M
REALLY
NERVOUS
WISH I
WAS AT
SCHOOL
TECH.
WRITER
SYSTEM
ADMIN.
HOSPIT’L
BLDG
MGR
DOUG
PATIENT
SAFETY
TEAM
IMAGING
TECH
STAFF
PAGING
SYSTEM
PATIENT
RECORD
SYSTEM
RESERV.
SYSTEM
IMAGING
RECORD
SYSTEM
DRUG
ROOM
Design New Experiences
Innovate to influence attitudes and change behaviors
TISSUES
NURSE
IDEA:
LET’S GO
CAMPING

28. GOES TO
FAMILY
DOCTOR
RIDES TO
HOSPIT’L
CHECKS
IN
WALKS
TO MRI
ROOM
SEES MRI
MACHINE
CRIES &
RESISTS
SEES
DOCTOR
ENTER
GOES
THROUGH
MRI
………
DOCTOR MOM
CHECKIN
NURSE
IMAGING
TECH
MOM
IMAGING
TECH
MRI
MACHINE
DESK &
COMPUT.
CHARTCAR
TAKE-
HOME
PACKET
MRI
MACHINE
AM I
REALLY
SICK?
THAT
LOOKS
SCARY!
WILL IT
HURT
ME?!
MUM
CAN’T
HELP?
NO! NO!
PLEASE
NO!
I’M
REALLY
NERVOUS
WISH I
WAS AT
SCHOOL
TECH.
WRITER
SYSTEM
ADMIN.
HOSPIT’L
BLDG
MGR
DOUG
IMAGING
TECH
PATIENT
RECORD
SYSTEM
RESERV.
SYSTEM
IMAGING
RECORD
SYSTEM
IDEA:
CAMP
GUIDE
WOW, A
CAMP-
FIRE!
FEELS
LIKE I’M
CAMPING
THIS
ISN’T
TOO BAD
THAT
WASN’T
SO BAD
THIS
PLACE IS
NEAT!
TODAY
COULD
BE FUN
LAYS
DOWN IN
MACHINE
LISTENS
TO
FOREST
SOUNDS
…
Test New Experiences
New attitudes, new behaviors….different result
NURSE
HAS
MRI
SCAN
IDEA:
CAMPING
SITE
IDEA:
CAMP
BACKPK
FEWER
STAFF
NO
DRUGS
IMAGING
TIME ↓
↓ SIDE
EFFECTS

29. Sample Journey Map

30. User Testing: Observing Your Customers
© Robert Matakovic

31. • User (or usability) testing is a
technique used to evaluate a
product, app, site or service with
current or potential customers.
• During a session, a participant
uses an interface and performs a
series of measurable tasks.
• A test moderator is present to ask
follow-up questions and further
clarify the user’s statements.
• User testing is NOT a focus group.
• User testing is NOT a forum for
asking certain kinds of questions…
Icon by Luis Prado (CC BY 3.0 US)
What is User Testing?

32. How well do you think
other users would
understand this
feature/interface?
A Bad User Testing Question…

33. A Better User Testing Question
Is this feature/interface
what you expected?
Why or why not?

34. Would you agree or
disagree with the
following statement:
“I was very lost and
confused.”
A Bad User Testing Question…

35. A Better User Testing Question
Would you agree or
disagree with the
following statement:
“The site caused me to
feel lost and confused.”

36. How often do you use
site/app/system?
A Bad User Testing Question…

37. A Better User Testing Question
In the past six months,
how often did you use
site/app/system?

38. How would you
design it?
A Bad User Testing Question…

39. A Better User Testing Question
Is it clear?
Why or why not?

40. How do you feel
about this brand?
A Bad User Testing Question…

41. A Better User Testing Question
Don’t even bother…

42. Observing what a customer
does versus what they say
Testing only 5-7 users reveals
most issues during a round of
testing
The sooner you discover an
issue, the cheaper it is to fix
Talking to your customers
generates empathy them
It’s time intensive, compared
to other ways of gaining
insight
Why Conduct User Testing?
Reference
Nielsen, Jakob, and Landauer, Thomas K.: "A mathematical model of
the finding of usability problems, "Proceedings of ACM INTERCHI'93
Conference (Amsterdam, The Netherlands, 24-29 April 1993), pp.
206-213.

43. Recruiting
Scheduling &
Lead Time
Test Set Up &
Equipment
Preparation
Moderating
Analysis &
Reporting
1-3 weeks 1-3 days Up to 1 week
Traditional User Testing Timing

44. Benefits
• Access to a panel of 1MM users
from across the globe.
• A screener questionnaire allows
us to reach our typical customer
• Video and questionnaire
responses in a little as 1 hour
Impact to AG:
• Service eliminates barriers to
quickly get customer feedback
• Provides a rich, behavioral
measure of customers
About UserTesting.com
This user (yvette4uk) was in the process of browsing for
the right card and stumbled upon one that she liked that
didn’t offer the Print to Store option.

45. Recruiting
Scheduling &
Lead Time
Test Set Up &
Equipment
Preparation
Moderating
Analysis &
Reporting
1-3 weeks 1-3 days 1 day Up to 1 week
UserTesting.com: Remote Testing FTW
Test Set Up
Analysis &
Reporting

46. While it’s a huge time saver, it’s not
quite perfect
• Further elaboration isn’t possible
• Conversations can’t lead to areas you
haven’t considered
• Some testing biases are still in play
• Additional recruiting time is required
for testing your users
Limitations of Remote Testing

47. So Let’s Put It All Together

49. Positioning Statement

50. Intro (Splash) Screen

51. “Learn” Page

53. Tom Bubnick, Manager User Experience and User Interface
Kristen Deevers, Senior Manager Consumer Insights