1.
Marketing Edition:
How we leverage
UserTesting
DECEMBER 12, 2018
Marketing

2.
2
Michael Mace
VP of Product Marketing

3.
Summary
● UserTesting is not just for usability
● It’s a great way to evaluate emotional reactions
○ Validate marketing content
○ The explanation behind analytics and AB tests
● My favorite use: Discovery interviews
○ Bridge the empathy gap

4.
UserTesting for Marketing, Part One
● Three Ways Fast Human Insight is Revolutionizing Marketing
● Topics:
○ Testing video
○ Testing images
○ Testing a webinar promotion
○ Customer use stories
● https://bit.ly/2GddKpp
● Today: How I use it

5.
Agenda
● Quick introduction to UT
● The revolution in research
● Understanding customers and buyers

6.
What We Do
● Human insights at the speed of digital business
○ In two hours, video and audio feedback from customers on almost anything
○ Target your particular customer
○ So easy that anyone in your team can use it

7.
How it Looks

8.
How We’re Used in Marketing
● Validate: Fast reactions to marketing materials and messages
○ Confirm whether it resonates
○ No more guessing
○ Persuasive video evidence
● Discover: Quick insight on customer attitudes, needs, and
journeys
● Iterate: Explain analytics, improve your A/B tests
○ Increase your win rate by pre-testing alternates

9.
Agenda
● Quick introduction to UT
● The revolution in research
● Understanding customers and buyers

10.
The Tech Transformation Hits Market Research
● Pre-2000
○ Survey: 1-2 a quarter, 6-12 weeks
○ Focus groups: Months of prep and travel plans
● 2000’s: Online surveying
○ Survey: Results in days, cost in pennies
○ Analytics too

11.
Imbalance
● Quant research has come
into the modern age
● Qualitative research is
stuck in 1980
● Result: We’re losing sight
of the humans behind the
data
○ Treating people like
Pavlovian dogs
○ Guess what: They can
sense it

12.
Agenda
● Quick introduction to UT
● The revolution in research
● Understanding customers and buyers

13.
How I Use UT: Discovery Interviews
● Understand customer needs and problems
● Empathy: Get inside their heads
● What can UT do to help them?
● Example: Marketers

14.
Interviews
● I don’t like focus groups
○ Hard to schedule
○ People don’t show up
○ Someone always dominates or skews it
● 1:1 interviews
○ Do them via video (to get the recordings)
○ Nationally
○ Pay compensation: $200 for executives (makes them try to give value)
○ Half an hour
○ About 10-15 (until they converge)

15.
What to Ask
● Most people like talking about themselves. Get them talking.
○ Be up front about that
○ Ask for examples
○ List of topics to cover
● Ask a question in several ways, as a paragraph
● What I used
○ What are your goals for this year
○ What are the barriers? What keeps you up at night?
○ How important is customer experience? How do you measure success?
● The first half to two-thirds of the conversation is about them; last half to
one-third is about reactions to our solution

16.
Live Versus Self-Guided
● Live
○ Establish relationship
○ Dig deep
○ See their faces
○ Follow interesting leads (good for initial scouting – figure out which questions to ask)
● Self-guided
○ They open up
○ They’re not trying to please you
○ Very, very fast
○ Quick to review, so you can talk to more people (fast playback, transcript)
● What we learned…

17.
The Underlying Need
Drive a lasting emotional relationship with customers…
…by creating a great customer experience…
…so they stick with us over time and don’t buy just on
price.

18.
The Experience Includes Every Touchpoint
It's marketing's job to represent the
client at the table and think through
everything the client is thinking,
doing, and seeing across the board.
Financial services company
It is all about the customer
experience. It's every touch point
where we can communicate and
have a relationship with a user or a
channel.
Consumer durable goods

19.
Customer Experience is Hard to Lead
When you add things like reviews
that impact other customers, the
customer experience becomes more
important. The consumer has much
more power than they did in the
past, and the experience is critical.
But because it impacts so many
different parts of the lifecycle or
consumer journey, it is not wholly
owned by one group. There are so
many stakeholders involved that
ownership is spread over so many
places.
Consumer durable goods
Put yourself in the shoes of the customer
and carry that through to something richer
and more compelling. To a degree, when it
comes to loyalty and messaging, that falls
within marketing. When you think about
other areas...the website team, product
management...they each own a piece of it.
Trying to bring that together is challenging
Major retailer

20.
We have done a lot of research around the buying process, what they are doing at each stage, who is
involved. We have mapped all of our tools across all of those people, and our salespeople have a
map that says at this stage to these people, these are the tools to use and the questions to ask.
We also map out how we interact with them through the showrooms, how do we plan a visit, how do we
host them. We track the best restaurants and bring food in, we also have our own chefs in house. We also take
them out on Lake Michigan. We do all of those experiences to educate and also make it a personal experience.
We are getting better at doing experience planning for our content campaigns. What are their choices if they come
to us from LinkedIn, etc. We spend a lot of time doing both big picture experience and the little details.
The culture of the organization really helps dictate the experience. The showroom staff reports to me, but the staff
that manages it reports to sales. But because of industry dynamics the expectations are understood and upheld
across the company. It's a cultural thing.
We were appalled when we went out to visit Salesforce. They served us this lunch on paper plates.
B2B Durable Goods
Real Customer Experience

21.
How They Work: Intuition vs. Science
● There’s an issue with balance

22.
Science and Intuition
Because the digital piece has
provided us so much data, we have
forgotten the creative side, the idea
of a brand that resonates with the
consumer. Trust me, I want
everything to perform, but it always
wins if it says 50% off. Everyone is
becoming a discount provider
because of that.
Hospitality chain
You can AB test a subject line and tell
which one is best, but the more you get
toward the actual behavior changes you
want, some are not easy to measure.
Things that might change people's’
hearts, some of those are unmeasurable.
The analytics can't tell you intent.
Major internet company

23.
It’s About Understanding Emotion
Product: Rational Problem
Solving
Marketing: Emotional Motivations
What problems do customers have? How do customers feel? What are their
emotional hot buttons?
How can we fit into their tasks? What are their needs and life goals?
Is our product intuitive? How do they feel about our company?
Are they satisfied with our products? What relationship do they have with our
brand?
Customer experience = product
experience
Customer experience = sum of every
touchpoint

24.
Pace of Business: “You Can’t Wait for
Research”
80% of our SKUs* turn over every
year. We are much more SKU
intensive and product heavy than
most of the CPGs, which makes it
hard for us to be as data heavy as
them because you can't wait for
research. There is more gut in places
like us or fashion.
Consumer durable goods
*SKU = Stock Keeping Unit. Any separately-named product is a
SKU.
You have the business meeting where
you break down the revenue number,
then as the marketing person you have
to figure out where to put resources to
have the greatest impact. This is why so
many CPG guys fail in tech, they do not
have two years to figure it out. We were
launching 145 platforms, I had to pick
four to invest in right there.
B2B and B2C tech company

25.
Prioritization: It’s Hard to Know What’s Right
There is always a laundry list of
initiatives; how do we pick the vital
ones to invest in? You never know,
are you picking the right horse?
Consumer durable goods
Sometimes I just say "let's hope this
works.“
Financial services

26.
The Dilemma
● You need to drive a
lasting emotional
relationship with
customers
● Creating a relationship
requires deep emotional
insight
● You don’t have time for
the market research that
gives emotional insight
● How do you decide?

27.
The Power of Self-Interviews

28.
The Power of Self-Interviews

29.
“What Just Happened?”
● With all our research and analytics, how did we get it so wrong?
● So many conflicting explanations. What really happened?
● What will happen next?
● Bonus points: Could our product handle this issue?

30.
Methodology
● Self-interviews (unmoderated user tests)
○ Wanted fast answers
○ Hoped to get people to open up
● How to do it: In our tool, choose a web test and point it to a blank web page
○ Blank.org works fine
● Don’t ask them to turn on the webcam
○ Wanted maximum candor
● Afterward, asked permission to share the recordings (should have put that
up front)

31.
Make the Screener Hard to Guess
● Question:
○ “We’re exploring reasons why people did or did not participate in the 2016 presidential
election. Did you vote in the election on November 8, 2016?”
● Answer:
○ I did not vote (REJECT),
○ I voted for Hillary Clinton (REJECT),
○ I voted for Donald Trump (ACCEPT),
○ I voted for someone else not listed above (REJECT)
● Improvement: Give multiple reasons for why they didn’t vote:
○ Not registered
○ Registered but didn’t want to vote
○ Registered but forgot
○ Put the focus away from what you’re looking for

32.
Introduction
● “This isn’t a usability test. We all know how the country voted, now we want
to understand why. Please read each question and then spend 2-3 minutes
explaining your views and your thinking. Please do your best to give us
thorough answers. We are not looking for a single sentence. There are five
questions.
● “Your answers are confidential.”

33.
Questions
1. Who did you vote for in the presidential election, and why did you choose them?
How did you make the decision? How difficult was it? If you have voted in other
presidential elections, was the decision easier or harder this time? Why?
2. When you made your choice for president, was it more about voting for the
person you chose, or was it against someone else? Why? How did you feel
about the people you didn’t vote for?
3. Were you surprised by the results of the presidential election? Why or why not?
4. Think ahead to ten years in the future. In ten years, do you hope the country will
be different in any particular way? Are there some things about the country that
you would like to change? If so, what are they and why? Are there some things you
worry might change that you don’t want to? What are those changes you would like
to stop, and why do you want to stop them?
5. How do you feel overall about the future of the country? Are you optimistic or
pessimistic (or something else)? Why? What makes you feel that way?

34.
What I Learned
● Amazingly candid
○ Religion, personal life, etc.
● It was a heartfelt decision
○ People researched it and thought about it carefully
● Deep frustration against the establishment
○ System corrupted by insiders
○ Elites in cities ignoring the rest of the country
○ People cheating the system (the illegal in illegal immigration)
● It wasn’t Republican vs. Democrat, it was insider vs. outsider
● Hillary Clinton was seen as the ultimate elite insider, Trump was the
ultimate outspoken outsider

35.
In Their Own Words

36.
What It Means
● We mistook a therapy session for a policy debate
● How do you help a friend who’s upset?
● Empathy gap

37.
It’s Not Just Politics
● Analytics give us the illusion of connection
“We’re going to a world of empathy, through
AI and chatbots.”

38.
Listen and Think
● Is your company blinding itself with data?
● Do you really understand what your customers are thinking?
○ Their problems, frustrations, dreams
● What are you going to do about it?

39.
Conclusion: It’s About Empathy
● See the world through someone else’s eyes
● Balance the quant surveys and analytics
● What to do
○ Mix of live and self-guided interviews
○ Make them feel safe
○ Get them talking about themselves and their problems
○ Spend most of the time on them
○ Never assume you understand someone’s motivation

40.
40
Questions?
USERTESTING
If you have any further questions please contact us at: webinars@usertesting.com

41.
41
Thank You!
USERTESTING
If you have any further questions please contact us at: webinars@usertesting.com