Weitere ähnliche Inhalte Ähnlich wie 2011 World Usability Day - Darren Kall - Kall Consulting (20) Kürzlich hochgeladen (20) 2011 World Usability Day - Darren Kall - Kall Consulting1. 1 Unbiased Customer UX Review
Interviews
Unbiased Customer Interviews:
What insight can we trust?
@darrenkall
#cognitive #bias #ux
#WUD2011
KALL Consulting
customer and user experience design and strategy
Half-hour version: 10Nov2011
© Kall Consulting 2011 --- +1 937-648-4966 --- darrenkall@kallconsulting.com 1
2. 2 Consulting Specialties
KALL
Unbiased Customer UX Review
Interviews
Product and Service Development and Improvement:
Applying user experience techniques to the end-to-end customer and
user experience design
Product Integration:
Integrating the functionality of multiple products, combining several
products into one product, and integrating acquired products into a
product family
Merger and Acquisition User Experience:
User experience evaluations in due diligence for buyers, user
experience strategy as part of long term exit plan strategy for
sellers, integration of products post M&A
Strategic User Experience Management:
Strategic user experience initiatives with your organization; driving
user-centric product development, angry customer mediation and
resolving user-centric issues, user experience turnaround campaigns,
© Kall Consulting 2011 --- +1 937-648-4966 --- darrenkall@kallconsulting.com 2
3. 3
What’s the problem?
Unbiased Customer UX Review
Interviews
1. Informed business decisions need accurate customer
insight.
2. Customer interviews for insight are different than other
interactions with customers.
3. People accidentally introduce biases in customer
interactions.
4. These biases taint the information they bring back.
5. Business decisions are made on this tainted information.
6. Tainted decisions increase business costs
unnecessarily, and result in lost revenue.
© Kall Consulting 2011 --- +1 937-648-4966 --- darrenkall@kallconsulting.com 3
4. 4
Volunteers Needed
Unbiased Customer UX Review
Interviews
Every customer interaction is not an interview
“Selling” Effect
Your approach can influence responses
“That’s not it!” Effect
How information is gathered makes a difference
Number Bias
Demand Characteristics Effect
© Kall Consulting 2011 --- +1 937-648-4966 --- darrenkall@kallconsulting.com 4
5. 5
Customer Interview Workshop
Unbiased Customer UX Review
Interviews
Recording your Interviews
An Interview Coach
Interview Logistics
Biases in Interviewing
Picking an Interview Methodology
Question Design
Using Persona to Pre-categorize Interviewees
Data Analysis and Reporting
Learning to share: Customer insight database
© Kall Consulting 2011 --- +1 937-648-4966 --- darrenkall@kallconsulting.com 5
6. 6 Biases and Effects
12
Unbiased Customer UX Review
Interviews
1. “Selling Effect” 7. Confirmation Bias
2. “That’s not it!” Effect 8. Framing Effect Bias
3. Number Bias 9. Encouragement Bias
4. Demand 10. Correlation and
Characteristic Effect Causality Bias
5. Scaling and Anchor 11. Ingrained Biases
Biases 12. Status Quo Bias
6. Consistency Bias
© Kall Consulting 2011 --- +1 937-648-4966 --- darrenkall@kallconsulting.com 6
7. 7 Biases and Effects
Extra
Unbiased Customer UX Review
Interviews
• Not Remaining • Recency Effect
Neutral Effect • Anchor Bias
• Script Bias 1 • Impact Bias
• Script Bias 2 • Not Invented Here
• White Lie Effect Bias
• Not Listening Effect • Bandwagon Effect
• Value Bias
© Kall Consulting 2011 --- +1 937-648-4966 --- darrenkall@kallconsulting.com 7
8. 8 More Biases not Covered Today
22
Unbiased Customer UX Review
Interviews
• Ambiguity effect • Information bias
• Black swans (attention) • Loss aversion bias
• Choice supportive bias • Misunderstanding
• Clustering illusion probability
• Contrast effect • Negativity bias
• Denomination effect • Omission bias
• Detection bias • Overconfidence effect
• Endowment effect • Reporting bias
• Funding bias • Scope neglect bias
• Halo effect • Unit bias
• Hindsight effect • Zero-risk bias
• Hyperbolic discounting • Etc. The list goes on.
© Kall Consulting 2011 --- +1 937-648-4966 --- darrenkall@kallconsulting.com 8
9. 9 or Effect Name
Bias
Unbiased Customer UX Review
Interviews
The behavior, perception, or preference that
causes the problem.
How the bias gets introduced and/or
what happens when it does.
What you should do instead.
Rationale for why people behave this
way or why you should avoid this bias.
© Kall Consulting 2011 --- +1 937-648-4966 --- darrenkall@kallconsulting.com 9
10. 10
“Selling Effect”
Unbiased Customer UX Review
Interviews
© Kall Consulting 2011 --- +1 937-648-4966 --- darrenkall@kallconsulting.com 10
11. 11
“Selling Effect”
Unbiased Customer UX Review
Interviews
Selling your product or your idea instead of
interviewing.
Customers respond to pitch, not to the
interview. Your insight is limited.
Don’t sell. Save the selling for another
interaction.
You’re not there to sell. You're trying to understand who the
customer is in order to get insight into their world.
© Kall Consulting 2011 --- +1 937-648-4966 --- darrenkall@kallconsulting.com 11
12. 12 not it!” Effect
“That’s
Unbiased Customer UX Review
Interviews
© Kall Consulting 2011 --- +1 937-648-4966 --- darrenkall@kallconsulting.com 12
13. 13 not it!” Effect
“That’s
Unbiased Customer UX Review
Interviews
Correcting, demeaning, blaming, etc. the
customer.
Treating the customer this way gets them
defensive or shuts them down.
Don't be corrective or pedantic. Reward the customer for
telling you things even if you disagree with them. Clarify
your understanding, capture the ideas, and move on.
The customer has their opinions and ideas. A customer
interview is not the place to be correcting them, even if
the idea is something the company will never do or that
you disagree with.
© Kall Consulting 2011 --- +1 937-648-4966 --- darrenkall@kallconsulting.com 13
14. 14
Number Bias
Unbiased Customer UX Review
Interviews
© Kall Consulting 2011 --- +1 937-648-4966 --- darrenkall@kallconsulting.com 14
15. 15
Number Bias
Unbiased Customer UX Review
Interviews
Thinking and reporting of numbers is highly
influenced without awareness.
13
People avoid some numbers. They “round” numbers to
align with their base system. Different cultures have
different biases. If asked for a number, people will make
up answers.
Be aware that the numbers you get back are not absolute
but are shaped. They are not data.
Culture, base 10, 6, 8 or 60 numbering systems, whole
numbers over fractions, having 24 hours, 7 days, all frame
the number bias.
© Kall Consulting 2011 --- +1 937-648-4966 --- darrenkall@kallconsulting.com 15
17. 17
Demand Characteristics Effect
Unbiased Customer UX Review
Interviews
You provide cues to your purpose and
interviewees give you a
“right” answer.
You don’t get a true answer, you get the biased answer you
“ask” for.
Remove demand characteristics, encourage open and
difficult responses.
Only by eliminating the expectation of a “right” answer can
you get to unbiased answers.
© Kall Consulting 2011 --- +1 937-648-4966 --- darrenkall@kallconsulting.com 17
18. 18 and Anchors Biases
Scaling
Unbiased Customer UX Review
Interviews
© Kall Consulting 2011 --- +1 937-648-4966 --- darrenkall@kallconsulting.com 18
19. 19 and Anchors Biases
Scaling
Unbiased Customer UX Review
Interviews
Making up scaling anchors, making up
scales, changing scales.
Introduces biases in customer
answers, make interpretation
impossible.
Get an expert to build your scales and set anchors. Don’t
make them up, change them, or try to be funny.
Inappropriate scales, changing them, etc. makes them
meaningless and impossible to compare.
© Kall Consulting 2011 --- +1 937-648-4966 --- darrenkall@kallconsulting.com 19
20. 20
Consistency Bias
Unbiased Customer UX Review
Interviews
© Kall Consulting 2011 --- +1 937-648-4966 --- darrenkall@kallconsulting.com 20
21. 21
Consistency Bias
Unbiased Customer UX Review
Interviews
Filtering out information accidentally to
make things consistent. Extrapolating
from one example to all cases.
Conclusions drawn won’t reflect the
reality of the customer’s variety.
Report what you observe not what you want to see. Report
all that you observe.
Forcing consistency where it does not exist is biasing the
information.
© Kall Consulting 2011 --- +1 937-648-4966 --- darrenkall@kallconsulting.com 21
22. 22
Confirmation Bias
Unbiased Customer UX Review
Interviews
© Kall Consulting 2011 --- +1 937-648-4966 --- darrenkall@kallconsulting.com 22
23. 23
Confirmation Bias
Unbiased Customer UX Review
Interviews
© Kall Consulting 2011 --- +1 937-648-4966 --- darrenkall@kallconsulting.com 23
24. 24
Confirmation Bias
Unbiased Customer UX Review
Interviews
Find only supportive information, interpret or
avoid contradictions to fit your idea.
You only report supporting information
and you don’t SEE what you filtered
out with your bias.
Attempt to capture everything and make fair assessments.
Get someone else to review your interview recordings.
Confirmation bias is ingrained and hard to undo without
conscious attention and practice.
© Kall Consulting 2011 --- +1 937-648-4966 --- darrenkall@kallconsulting.com 24
25. 25
Framing Effect Bias
Unbiased Customer UX Review
Interviews
© Kall Consulting 2011 --- +1 937-648-4966 --- darrenkall@kallconsulting.com 25
26. 26
Framing Effect Bias
Unbiased Customer UX Review
Interviews
600 People with a fatal disease
Drug A saves 200 people's lives
Drug B has a 33% chance of saving all 600
people and a 66% possibility of saving no
one
© Kall Consulting 2011 --- +1 937-648-4966 --- darrenkall@kallconsulting.com 26
27. 27
Framing Effect Bias
Unbiased Customer UX Review
Interviews
600 People with a fatal disease
If Drug C is taken, then 400 people die
If Drug D is taken, then there is a 33%
chance that no people will die and a 66%
probability that all 600 will die
© Kall Consulting 2011 --- +1 937-648-4966 --- darrenkall@kallconsulting.com 27
28. 28
Framing Effect Bias
Unbiased Customer UX Review
Interviews
Framing a question or choice in a way that
manipulates its perceived value.
How a question or choice is presented biases the listener’s
answers.
Get an expert question designer to review your question
script. Explore different ways to present the question to
see if it influences the answer.
Framing effects work because of human nature preferring
gain over loss, certainty over doubt, positive over
negative.
© Kall Consulting 2011 --- +1 937-648-4966 --- darrenkall@kallconsulting.com 28
29. 29
Encouragement Bias
Unbiased Customer UX Review
Interviews
© Kall Consulting 2011 --- +1 937-648-4966 --- darrenkall@kallconsulting.com 29
30. 30
Encouragement Bias
Unbiased Customer UX Review
Interviews
Encouraging a particular result, giving
“good answer” cues.
Responses can be biased and shaped in
one direction or another.
Don’t reveal desired goal, don’t reward
positive or punish negative responses;
or vice versa. Remain consistent
customer to customer.
Encouragement sends information to the user as to the
desired outcome and they may respond without even
being aware.
© Kall Consulting 2011 --- +1 937-648-4966 --- darrenkall@kallconsulting.com 30
31. 31
Correlation and Causality Bias
Unbiased Customer UX Review
Interviews
© Kall Consulting 2011 --- +1 937-648-4966 --- darrenkall@kallconsulting.com 31
32. 32
Correlation and Causality Bias
Unbiased Customer UX Review
Interviews
Assuming that because things occur
together, or close in time, that one causes
the other.
When you ask interviewees for root
causes they will report correlations
which may not be the true root cause.
Correlations do not imply causality. Consider but don’t
blindly accept the rationale without triangulation.
General rules (heuristics) are built from experience and
applied incorrectly to all occasions.
© Kall Consulting 2011 --- +1 937-648-4966 --- darrenkall@kallconsulting.com 32
33. 33
Ingrained Biases
Unbiased Customer UX Review
Interviews
© Kall Consulting 2011 --- +1 937-648-4966 --- darrenkall@kallconsulting.com 33
34. 34
Ingrained Biases
Unbiased Customer UX Review
Interviews
Cultural, gender, national, geographic and
other backgrounds bias
behavior, preferences, and perceptions.
Both interviewers and interviewees
respond to each other filtered with
these biases.
Mitigate your own ingrained biases as much as possible.
Counter balance interviewers. Use interviewers from the
same culture and country.
You can’t get away from your accent or other culture you
wear. You cannot control how others see you.
© Kall Consulting 2011 --- +1 937-648-4966 --- darrenkall@kallconsulting.com 34
35. 35 Quo Bias
Status
Unbiased Customer UX Review
Interviews
© Kall Consulting 2011 --- +1 937-648-4966 --- darrenkall@kallconsulting.com 35
36. 36 Quo Bias
Status
Unbiased Customer UX Review
Interviews
Some people you will interview will have a
preference to keep things as they are
regardless of what you are proposing.
This biases the response away from new
things that will introduce change even
if they are better options giving them
false values.
Understand the motivations and loss for this persona and
address them.
Study on rats, cheese, hunger and shock grid: status quo
tipping point.
© Kall Consulting 2011 --- +1 937-648-4966 --- darrenkall@kallconsulting.com 36
37. 37 Biases and Effects
Extra
Unbiased Customer UX Review
Interviews
• Not Remaining • Recency Effect
Neutral Effect • Anchor Bias
• Script Bias 1 • Impact Bias
• Script Bias 2 • Not Invented Here
• White Lie Effect Bias
• Not Listening Effect • Bandwagon Effect
• Value Bias
© Kall Consulting 2011 --- +1 937-648-4966 --- darrenkall@kallconsulting.com 37
38. 38 Bias 1
Script
Unbiased Customer UX Review
Interviews
© Kall Consulting 2011 --- +1 937-648-4966 --- darrenkall@kallconsulting.com 38
39. 39 Bias 1
Script
Unbiased Customer UX Review
Interviews
Skipping questions in the script and
improvising question variations.
Skipping questions and significant
improvisation force biases in
analysis when combining
information.
Hit all the questions. Don't improvise too much or you won't
have fair comparisons.
These are parts of normal conversation but interviews are
not normal conversations.
© Kall Consulting 2011 --- +1 937-648-4966 --- darrenkall@kallconsulting.com 39
40. 40 Bias 2
Script
Unbiased Customer UX Review
Interviews
© Kall Consulting 2011 --- +1 937-648-4966 --- darrenkall@kallconsulting.com 40
41. 41 Bias 2
Script
Unbiased Customer UX Review
Interviews
Ignoring or cutting off customer attempts to
roam away from script.
You can miss gems of insight. You are
blind-sided by your focus/belief in your
survey and your bias limits the scope
of information.
Allow enough time to let the customer lead you “off track”.
Explore, evaluate, capture, then get them back on track.
The best stuff may be in what the customer knows that you
did not think to ask about.
© Kall Consulting 2011 --- +1 937-648-4966 --- darrenkall@kallconsulting.com 41
42. 42 Lie Effect
White
Unbiased Customer UX Review
Interviews
© Kall Consulting 2011 --- +1 937-648-4966 --- darrenkall@kallconsulting.com 42
43. 43 Lie Effect
White
Unbiased Customer UX Review
Interviews
Telling white lies for
approval, support, encouragement, etc
..
Lies introduce biases in future
responses since based on
falsehoods.
Don't lie. Don't make false promises. Don’t confirm just
because they ask.
White lies seem like a kindness or an easy avoidance but
they have a devastating effect on the value of the
information. And you have to remember your lie.
© Kall Consulting 2011 --- +1 937-648-4966 --- darrenkall@kallconsulting.com 43
44. 44 Listening Effect
Not
Unbiased Customer UX Review
Interviews
© Kall Consulting 2011 --- +1 937-648-4966 --- darrenkall@kallconsulting.com 44
45. 45 Listening Effect
Not
Unbiased Customer UX Review
Interviews
Allowing interruptions such as
email, or thinking of next
appointment, to shift your focus
from the customer.
This biases the customer against
you and changes their
responses.
Stay focused. Take break if need one. Reschedule if needed.
Actively listen.
If you’re not listening, you’re missing things.
© Kall Consulting 2011 --- +1 937-648-4966 --- darrenkall@kallconsulting.com 45
46. 46 Bias
Value
Unbiased Customer UX Review
Interviews
© Kall Consulting 2011 --- +1 937-648-4966 --- darrenkall@kallconsulting.com 46
47. 47 Bias
Value
Unbiased Customer UX Review
Interviews
Using customer information as a weapon.
It and your conclusions are complete
perfect answers.
This blinds you to alternatives, gives
you false confidence in partial
information, & shuts down other
people.
Do interviews, bring back impressions, draw conclusions
and innovate ideas. But don’t treat them as data and facts.
They are only impressions; treat them that way.
© Kall Consulting 2011 --- +1 937-648-4966 --- darrenkall@kallconsulting.com 47
48. 48 Remaining Neutral Effect
Not
Unbiased Customer UX Review
Interviews
© Kall Consulting 2011 --- +1 937-648-4966 --- darrenkall@kallconsulting.com 48
49. 49 Remaining Neutral Effect
Not
Unbiased Customer UX Review
Interviews
Show positive or negative reactions to
customer responses, opinions, or actions.
Giving feedback teaches the customer
too much about your views, and
biases their responses.
Remain neutral and indifferent to their
answers, opinions, etc. Remain calm and non-committal.
Show no value reactions.
It is human nature to give feedback but doing so taints
responses.
© Kall Consulting 2011 --- +1 937-648-4966 --- darrenkall@kallconsulting.com 49
50. 50
Recency Effect
Unbiased Customer UX Review
Interviews
© Kall Consulting 2011 --- +1 937-648-4966 --- darrenkall@kallconsulting.com 50
51. 51
Recency Effect
Unbiased Customer UX Review
Interviews
Human memory biases toward the last item
in lists being remembered better.
The most recent interview or last
comments will influence your
conclusions or reporting of
findings more.
Take notes on all interviews, review all materials when
doing analysis.
This is a core way human memory works along with the
Primacy Effect.
© Kall Consulting 2011 --- +1 937-648-4966 --- darrenkall@kallconsulting.com 51
52. 52 Bias
Anchor
Unbiased Customer UX Review
Interviews
© Kall Consulting 2011 --- +1 937-648-4966 --- darrenkall@kallconsulting.com 52
53. 53 Bias
Anchor
Unbiased Customer UX Review
Interviews
Interviewees focus on one aspect
(speed, color, size, etc.) and give it more
importance making it a reference or anchor
for all other comparisons.
People are biased by this one aspect
skewing all their responses.
Discover and dismiss the anchor “Ignoring color…”. Use
paired comparisons. Define the anchor instead.
First exposed or obvious aspects often become the
comparison for all other judgments.
© Kall Consulting 2011 --- +1 937-648-4966 --- darrenkall@kallconsulting.com 53
54. 54 Bias
Impact
Unbiased Customer UX Review
Interviews
© Kall Consulting 2011 --- +1 937-648-4966 --- darrenkall@kallconsulting.com 54
55. 55 Bias
Impact
Unbiased Customer UX Review
Interviews
When people make judgments of future
impacts they exaggerate both the positive
and negative aspects.
The estimates you get back are biased to
overestimation in both directions.
This is not data, it is perspective and opinion information
that you should note on questions where you asked for
judgments of future impact.
This is a natural tendency without awareness. People
believe the numbers they give you.
© Kall Consulting 2011 --- +1 937-648-4966 --- darrenkall@kallconsulting.com 55
56. 56 Invented Here Bias
Not
Unbiased Customer UX Review
Interviews
© Kall Consulting 2011 --- +1 937-648-4966 --- darrenkall@kallconsulting.com 56
57. 57 Invented Here Bias
Not
Unbiased Customer UX Review
Interviews
Bias in interviewees and interviewers that
what we create is better than what someone
else creates.
Interviewees give false feedback on
ideas that are not theirs. Interviewers
miss better solutions that customers
may create.
Be aware of biases on both sides and keep consideration
open.
There are many reasons: fear of unknown, unwilling to
value others, lack of control, etc.
© Kall Consulting 2011 --- +1 937-648-4966 --- darrenkall@kallconsulting.com 57
58. 58
Bandwagon Effect
Unbiased Customer UX Review
Interviews
© Kall Consulting 2011 --- +1 937-648-4966 --- darrenkall@kallconsulting.com 58
59. 59
Bandwagon Effect
Unbiased Customer UX Review
Interviews
Some people influence group opinion and
others adopt it.
Strong personalities can influence
people interviewed in groups.
Not always a bad thing. If you want unbiased
opinions, interview alone. If you want to generate
conversation to get more information, use a group.
Group dynamics influence, behaviors, perceptions and
preferences.
© Kall Consulting 2011 --- +1 937-648-4966 --- darrenkall@kallconsulting.com 59
60. 60
Caution
Unbiased Customer UX Review
Interviews
What I’ve told you about today can also be used for
eviL
Please don’t.
60
© Kall Consulting 2011 --- +1 937-648-4966 --- darrenkall@kallconsulting.com
61. 61 You
Thank
Unbiased Customer UX Review
Interviews
Darren Kall
• darrenkall@kallconsulting.com
• http://www.linkedin.com/in/darrenkall
• @darrenkall
• +1 (937) 648-4966
I’m glad to help your company become more
customer and user-centric.
© Kall Consulting 2011 --- +1 937-648-4966 --- darrenkall@kallconsulting.com 61
62. 62 Credits
Photo
Unbiased Customer UX Review
Interviews
blood splat http://www.centrepatriotes.com
clock http://www.armyproperty.com/Resources/NSN-Listings/Clocks.htm
salesman http://www.expressionsofexcellence.com/GIFs/creative_selling.gif
buttons http://dragonlily.com.au/shop/popup_image.php?pID=154
script http://www.hearingreview.com/issues/articles/2003-10_02.asp
wrangler round up http://www.thebigquestions.com/wp-content/uploads/2010/04/roundup.jpg
liar http://www.mediabistro.com/fishbowldc/files/original/art_lie.jpg
scolding teacher http://2.bp.blogspot.com/_NjXJS2YMaYw/SyBiwnntCkI/AAAAAAAAAGY/hZjiRbF4TZg/s200/scold.gif
ear bud yeller http://quickbase.intuit.com/blog/wp-content/uploads/2010/08/92038203_5d8d68f920.jpg
knowledge is power
cartoon http://llenrock.com/blog/wp-content/uploads/2009/12/knowledge-is-power.gif
Home Depot survey http://www.homedepotopinion.com/
neutral shirt http://www.zazzle.com/
Rosie riveter http://en.wikipedia.org/wiki/Rosie_the_Riveter
confirmation bias http://www.agenarisk.com/resources/probability_puzzles/confirmation_bias.shtml
matrix choices http://imo-networker.com/two-roads-to-take/
brick framed http://twistedphysics.typepad.com/.a/6a00d8341c9c1053ef0148c85e67e9970c-pi
brain http://www.internettime.com/wp-content/uploads/2009/12/brain-300x226.png
anchor http://www.boatweb.com.au/images/admiralty.gif
fear status quo http://writingcreativenonfiction.files.wordpress.com/2011/07/fear-of-change.jpg
bingo balls http://www.bingo-balls.com/
family circus cartoon http://www.oxfamblogs.org/fp2p/wp-content/uploads/causation-v-correlation-cartoon-246x300.gif
cultural bias faces http://news.brown.edu/node/10388?size=large
flying car http://www.metro.co.uk/weird/58822-flying-cars-are-here-at-last
bridge jumpers http://diyblogger.net/wp-content/uploads/2010/12/jumping_bridge.jpg
© Kall Consulting 2011 --- +1 937-648-4966 --- darrenkall@kallconsulting.com 62