1.
Agile interviews:
A method to make
better CX
decisions
ANN MOREY
October 17, 2018

2.
Ann Morey
CX & UX Consultant

3.
3
● A formula for doing short, targeted customer interviews, either
moderated or unmoderated.
● Doesn’t involve any type of digital experience or prototype.
● Produces findings that enable better CX decisions with real input
from actual customers.
What are agile interviews?

4.
4
Discovery
“What is our customer support experience like?”
Mythbusting
“Survey data says most millenials want to pay bills with their debit cards,
so offering debit card payments would give providers an advantage
when attracting new customers.”
Making product decisions
“Should we include fund holds on customers’ daily account summary
emails? What do our competitors do?”
Agile interviews are valuable at all stages of
the product lifecycle

5.
5
Objective
Narrow your
research
objective down
to one key
question for
customers.
Recruit
Determine which
customers can
best answer
your research
question.
Script
Write a succinct
interview script
that gets to the
heart of your
research
objective.
Share
Look for patterns
and identify gaps
for iteration.
Iterate and
Analyze
Share
customers’
compelling
stories in their
own voices.
Agile interview overview

6.
Narrow down a research objective
USERTESTING

7.
7
● Looking for too much information in an interview can overwhelm
customers and researchers.
● Focus in on one core piece of information you’re looking for
● If you’re unable to narrow your research objective down to just one
question, run multiple studies with one question per study.
Narrow down your research objective to one
question

8.
8
Most of the time the question we start with can’t be directly answered by
customers. People can’t predict the future, including their own
behavior, and customers’ likes and preferences often don’t drive
their actions.
Instead, ask customers to tell a story about something they did or
something that happened to them.
Good questions prompt customers to
answer with a story

9.
9
Research objective examples
Business Question Target Users Research Objective
Why are customers calling support instead
of solving their problems themselves?
Our company’s customer
database
Tell me about the last time you
called customer service.
Is having a choice of bill payment methods
important to consumers?
Consumers who have updated
their auto insurance payment
method recently.
Tell me about updating your
auto insurance payment
method.
Will including fund holds in our daily
account summary emails drive up or
reduce call volume? How does our
competitor handle this and does it work for
them?
Our competitor’s customers who
have experienced a fund hold.
Tell me about having your
funds held.

10.
10
Run moderated studies when :
● Your research question is more broad and you anticipate the need
to ask follow up questions
● You “don’t know what you don’t know” and are unsure what follow-
up questions you’ll want to ask
If you’re not sure, start with an unmoderated study. If you find
yourself with many unanswered questions or wanting to ask why, iterate
with a moderated study.
Moderated vs unmoderated studies

11.
11
Recruit the right customers for your
objective
USERTESTING

12.
12
Use your own customer
database
If you can get permission to use
your own customer email
database, invite your own
customers to participate with My
Recruit.
Tip: Check with marketing and
whoever owns your
organization's NPS
communication.
How to recruit your own customers (or
get pretty close)
Find your customers in the
UserTesting panel
If your customers are well
represented in the general
population, screen for them in
the general UserTesting panel.
Find users that behave like
your customers
Screen the UserTesting panel for
participants that closely resemble
your customers.
Screen by behaviors, not
demographics

13.
13
Recruit customers who have recently experienced the story you want to
hear about.
● The more recent the better.
● Start with recruiting customers who have experienced their story
in the last 3 months, and extend it out to six or 12 months if
recruiting is slow.
● You may have to extend your window out for uncommon customer
experiences.
Recruit users who can best tell the research
objective story

14.
14
Write a succinct interview script
USERTESTING

15.
15
Expectations
Explain to the
user how the
test will work.
Background
Get critical
background info
about the user.
Story
Ask users to tell
their story and
be specific about
what you want to
hear.
Ask broad, open
ended questions
to capture the
user’s additional
thoughts and
opinions.
Conclusion
Anatomy of an agile interview script

16.
16
Let users know there won’t be a
prototype. Most users have never done an
unmoderated test without a prototype.
Inform users of the interview topic.
If the interview is short, tell users how
long the test will take. Some users will try
to keep talking if they feel their test was
“too short.”
Set expectations in the introduction
Discovery research example:
“There is no prototype for this test. Instead,
we're going to be asking you some
questions about your auto insurance
policy.”
Competitor research example:
“There is no prototype or website for this
test. Instead, we'll be asking you some
questions about your credit card merchant
processing service for your small business.
This test will take less than 10 minutes.”

17.
17
Only ask the bare minimum for what
you need. Background questions should
be easy to answer and not overly personal.
These questions warm up the user and
begin building a rapport.
Get critical background info (if needed)
Discovery research example:
“To start off with, who is your auto
insurance provider?”
Competitor research example:
“To start off with, please tell us about the
type of small business you own.”

18.
18
Ask open-ended
questions. Use “yes or no”
questions judiciously.
Tie the questioning back
to the screener to ease the
transition to “story time.”
Ask “Why?”
Prompt users to tell their story
Discovery research example:
“You indicated that you previously switched your method of making
reoccurring auto insurance payments.
● “What was your old method of making payments?
● “What is your new method of making payments?
● “Why did you decide to switch?”
Competitor research example:
“You indicated you experienced your merchant services provider
holding your funds before depositing them into your bank account.
● “How did you find out that funds were being held?
● “What actions did you take after learning that Square had held
funds?
● “How was the hold resolved?
● “Were you satisfied with the outcome of this process?”

19.
19
If you need specifics, include them in
follow-up questions. Users sometimes
interpret questions their own way.
If there is a finite range of options, give
examples of what you’re looking for.
If users repeat themselves in the follow-up
questions, that’s okay!
Include follow-up and clarifying questions
Discovery research example:
“If you haven't mentioned it already, what method
of payment (withdrawn from a bank account,
charged to a credit card, etc.) do you use to pay
your insurance bill?
● “Did you have options on what method to
use?
● “If so, why did you choose this method?
“If you haven't mentioned it already, what payment
term (monthly, quarterly, every 6 months, etc.) do
you pay your insurance bill?
● “Were you able to choose your payment
term?
● “If so, why did you choose this term?”

20.
20
When the interview asks questions about a
broader experience, conclude with the
“easiest/most difficult/magic wand”
questions.
If the interview asks narrow, more targeted
questions, conclude with a more broad
question.
Conclude with broad, open ended questions
Discovery research example:
“What was the easiest part of changing your
insurance payment method?”
“What was the most difficult part of changing
your insurance payment method?”
“If you had a magic wand and could change
anything about changing your insurance
payment method, what would you change?”
Competitor research example:
“Do you have any other thoughts or comments
to share about your experiences with your
merchant processor?”

21.
21
Iterate and analyze
USERTESTING

22.
22
Do a dry run with one user. If you make substantial changes to the
screener or script, do an additional dry run.
Add about 5 more users to your study once you’re satisfied with the
dry run.
○ Unmoderated: watch all 5 videos before adding more
○ Moderated: Pause after 5 users and assess
Total number of users is typically 7-20 users, depending on
iterations
Start small

23.
23
No strong patterns emerge after 5 users.
A type of customer is not represented among participants.
You discover a new question.
You did an unmoderated study and want to ask follow up questions.
The next iteration could be another unmoderated study or a moderated
study.
When to iterate

24.
24
Patterns occur when multiple users tell similar stories, both in what
happened and why.
A very compelling story from one or two users can feel like a pattern, so
double check recordings to make sure more than just one or two users
are telling the same story.
Making highlights reels of clips of patterns helps double check and
validate patterns.
Looking for patterns

25.
25
Pattern examples
Business Question Pattern
Why are customers calling support instead of
solving their problems themselves?
Users were calling customer support because their
experience didn’t match what their sales rep had told them.
They were calling because they thought the bill or features
they had were wrong, which they couldn’t fix themselves.
Is having a choice of bill payment methods
important to consumers?
Consumers would forego their preferred payment to get a
discount or avoid fees.
Will including fund holds in our daily account
summary emails drive up or reduce call
volume? How does our competitor handle this
and did it work for them?
The competitor did hold funds, and their customers reported
inconsistent experiences. Unless there was clear
communication about why funds were held and when they
would be released, customers called support whether or not
they discovered the hold themselves or were alerted to it
without full details.

26.
26
Video: Patterns in insurance
payment interviews

27.
Share your findings
USERTESTING

28.
28
Include highlights reels of themes and patterns in findings reports
and presentations.
Shorter clips (one to four minutes) are better.
Use audio only to minimize distractions from users’ screens.
Share the real “Voice of the Customer”
Source coy

29.
29
Create video
highlights reels
to share
Using a video editor, replace the video
image with relevant details about the
user (type of customer, age, location,
etc.)
For extra impactful videos, use stock
photos or other imagery.

30.
30
Video: Adding imagery to interviews

31.
31
Narrow down your research objective to one question users can
answer with a story.
Recruit customers who have recently experienced the story you want
to hear about
Write a succinct interview script that prompts users to tell their stories
and requests the details you need.
Iterate as you learn and analyze your findings to find patterns.
Share audio recordings so teams and stakeholders can hear
customers in their own words.
Agile interviews: Putting it all together

32.
Q&A
If you have any questions you can contact us at: webinars@usertesting.com

33.
33
Thank You!
Let’s keep in
touch!
Ann Morey
CX & UX Consultant
ann@slipstreamcx.com
https://www.linkedin.com/in/annmorey/
If you have any questions you can contact us at: webinars@usertesting.com