2. Areas of research
Behavioral decision theory
The study of how people judge, decide and behave
Cognitive psychology
The study of how people perceive, think, remember
and learn
Social psychology
The study of how people are influenced and affected
by other people
3. No neutral design
Preferences, judgments, and
decisions are remarkably sensitive to
the context in which they are made.
9. The power of the subconscious
Rosenthal/Jacobsen 1968
10. ‘When we expect certain behaviors of others,
we are likely to act in ways that make the
expected behavior more likely to occur.’
- Rosenthal and Babad
11. Confirmation bias
The tendency to seek out and assign more
weight to evidence that confirms one’s
hypothesis, and ignore or underweight
evidence that could disconfirm it.
12. So what?
We may be influencing participants –
and research findings – in ways
we’re not even aware of.
16. Frame the study
The purpose of this study is to test
the design, not you.
17. Child custody
Parent A
Average income
Average health
Average working hours
Reasonable rapport with the child
Relatively stable social life
Shafir, Simonson, Tversky, 1993
Which parent should get sole custody?
Award – 64%
Parent B
Above-average income
Minor health problems
Lots of work-related travel
Very close relationship with the child
Extremely active social life
18. Child custody
Parent A
Average income
Average health
Average working hours
Reasonable rapport with the child
Relatively stable social life
Shafir, Simonson, Tversky, 1993
Which parent should not get sole custody?
Award – 64%
Deny – 55%
Parent B
Above-average income
Minor health problems
Lots of work-related travel
Very close relationship with the child
Extremely active social life
19. Positive/negative
What did you like/dislike…
How useful, or not,…
How confident, or not…
Why would you continue/stop using…
20. How we ask the question
Positive/negative
Breadth/depth – broad/narrow
21. Did you notice anything different
about this process…
How does this process compare to
how you currently do this?
22. What information do you need in
order to feel confident about
doing x…
What would make you feel
confident about doing x…
23. How often do you use the x
website…
Tell me about how and when you
use the x website…
25. ‘A small question can never
hold a large answer.’
- Gerald Zaltman (How Customers Think)
26. Assumptions
Be careful that your assumptions
aren’t built into the question
Turn your assumptions into questions
27. Rating scales
How successful have you been in life,
so far?
0 1 2 3 4 5 6 7 8 9 10
Not satisfied Very satisfied
-5 -4 -3 -2 -1 0 1 2 3 4 5
Not satisfied Very satisfied
34%
13%
Schwartz et al, 1991
28. Scholarships
Two scholarships are available.
You prefer a scholarship with less waiting
time and more money.
Which one would you pick?
Sun, Li, Bonini 2010
29. Graphical scale
Figure 1: wait time salient – preferred A
Figure 2: $ amount salient – preferred B
Sun, Li, Bonini 2010
30. Graphical comparison
0 10 20 30 40
Toothpaste B
Toothpaste A
People (per 5000) with serious gum disease
E.R. Stone et al. 2003
34. Survey design
Please rate your overall satisfaction.
How many times have you had
repairs?
35. German students
How happy are you these days?
How many dates did you have last month?
Strack et al, 1988
36. Reverse order
How happy are you these days?
How many dates did you have last month?
How many dates did you have last month?
How happy are you these days?
Strack et al, 1988
37. ‘The sequential arrangement of information often
creates the very meaning of that information.
The order of presentation determines the reaction.’
- Dr. Frank Luntz (Words That Work)
38. So what?
In what ways are you creating
(unintended) meaning?
What are you measuring?
41. An auto accident
At what speed did car A contact car B?
At what speed did car A smash into car B?
Loftus/Palmer 1974
31.8 mph
40.5 mph
Actual: 12 mph
42. Power of words
Make contact
Smash into
Crash into
Bump into
Run into
Collide with
43. So what?
How are your words impacting
user perception – and consequently, research
findings?
how your business partners perceive the
findings?
47. Research design
‘The function of a research design is to ensure
that the evidence obtained enables us to answer
the initial question as unambiguously as possible.’
49. ‘All research methods involve
compromises with reality.’
- Gerald Zaltman (How Customers Think)
50. The solution
Be aware
Control what we can
How we ask questions
Ordering of questions
Choice of words
Assumptions and their effect
Use of ‘scale’ – charts, graphs, rating scales
Comparison – graphs, ratings