7. Findings
If (Berkeley)preschoolers read about nutrition during snack time,
they are more likely to eat more veggies. They will also understand
more about why veggies are important (including details that were
not in the reading.)
Practitioner takeaways
"
• When research findings don’t make sense, be dubious.
Citation:
Gripshover,
S.
J.,
&
Markman,
E.
M.
(2013).
Teaching
Young
Children
a
Theory
of
NutriHon:
Conceptual
Change
and
the
PotenHal
for
Increased
Vegetable
ConsumpHon.
Psychological
Science,
24(8),
1541–1553.
doi:
10.1177/0956797612474827
11. Insight 2
You hit diminishing returns on finding
new usability problems at about the 5th user test
12. AssumpHons:
#
problems
found:
N(1-‐(1-‐λ)i)
N=
Total
#
of
usability
problems
λ
=
probability
of
finding
the
average
usability
problem
when
running
a
single,
average
subject
or
using
a
single,
average
Evaluator.
For
this
curve
λ
=
33%.
I
=
#
of
parHcipants
or
evaluators
14. Findings
If you make assumptions about the % of problems an individual test
participant or evaluator will find, you can model when you will reach
diminishing returns for testing/evaluating.
Practitioner Takeaways
• If the “takeaway” is that simple, you should probably re-evaluate
for yourself.
Citation: Faulkner, L. (2003). Beyond the five-user assumption: benefits of increased sample sizes in usability testing.
Behavior Research Methods, Instruments, & Computers : A Journal of the Psychonomic Society, Inc, 35(3), 379–83.
17. Findings
With 5 participants, you find 85% the problems on average, but the
range is 55%-98%. When you test 10, its 94%/82%. For 15, its
97% /90%.
Practitioner Takeaways
• Target to test 10-12 participants, not 5.
Citation: Faulkner, L. (2003). Beyond the five-user assumption: benefits of increased sample sizes in usability testing.
Behavior Research Methods, Instruments, & Computers : A Journal of the Psychonomic Society, Inc, 35(3), 379–83.
20. Findings
When you show more than one design to a usability testing
participant, they can make contrastive comments.
Testing multiple designs leads to more detailed, comparative and
negative comments from participants
Takeaways
• Show participants more than one design during usability tests
and allow them to compare/contrast the user experiences.
Citation: Tohidi, Maryam., Buxton, William., Baecker, Ronald., and Sellen, Abigail. . (2006) Getting the Right Design and
the Design Right: Testing Many is Better Than One CHI 2006 Proceedings. Usability Methods. April 22-27, 2006.
Montreal, Quebec, Canada.
24. Findings
Treadmill tests and controlled walking tasks yield similar
performance in mobile evaluations.
Stopping/restarting is not ecologically valid on a treadmill.
Takeaways
Controlled walking tasks provide a better in-context evaluation.
• Feedback given
Citation: Barnard, L., Yi, J. S., Jacko, J. A., & Sears, A. (2005). An empirical comparison of use-in-motion evaluation
scenarios for mobile computing devices. International Journal of Human-Computer Studies, 62(4), 487–520. doi:10.1016/
j.ijhcs.2004.12.002
27. Findings
Parallel prototyping leads to more effective designs and greater
designer self efficacy
Takeaways
• Feedback given on a single design tends to lock designers into a
path or set. Create/Request feedback on more than one design
• Parallel prototyping promotes comparison without
defensiveness: Forces designers to think of alternative designs so
you don’t get attached to one early in the design process
Citation: Dow, S., Glassco, A., & Kass, J. (2011). The effect of parallel prototyping on design performance, learning, and
self-efficacy. ACM Conference on …, (September), 10. doi:10.1111/j.1432-2277.2009.00960.x
32. Findings
People liked left- and right-aligned forms best and rate them easiest
to fill out.
People were most efficient (fewest fixations) on top-aligned forms.
People looked longest at top- and in-field aligned forms.
In-field alignment was rated most difficult.
Flow was rated least appealing.
Practitioner takeaways:
This really doesn’t matter as much as we want to argue about it.
But, consider what people are doing and using when you pick.
"
Citation: Bojko,
A.
A.,
&
Schumacher,
R.
M.
(n.d.).
Eye
Tracking
and
Usability
TesHng
in
Form
Layout
EvaluaHon,
1–13.
"
38. 38
No instructions
Examples
Description
Description
+ Examples
Findings
Participants made the most errors with no instructions.
Descriptions resulted in fewer errors than examples.
Giving descriptions and examples doesn’t help (significantly),
but it doesn’t hurt either.
Practitioner takeaways:
Format examples are better than nothing.
Descriptions work better than examples.
Citation: Bargas-Avila, J., Orsini,S., Piosczyk, H., Urwyler, D., Opwis, K. (2011) Enhancing online forms: Use format
specifications for fields with format restrictions to help respondents. Interacting with Computers, 23(1).
42. Findings - How to present instructions
• “Expert” users and older users interact with instructions equally
frequently whether they are hidden or not.
• Low literacy participants tended not to interact with instructions
even when they need to (They didn’t help much, when they did.)
• Links draw more clicks that instruction icons.
Practitioner Takeaways
• Provide visible instructions.
• If you must hide them, use links rather than icons to let users
know they exist.
"
Reference: Alton,
N.,
Rinn,
C.
,
Summers,
K.,
and
Straub,
K.,
(Forthcoming)
Using
Eye-‐Tracking
and
Form
CompleHon
Data
to
OpHmize
Form
InstrucHons.
IEEE
–
IPPC
14.
Piesburgh,
PA
USA.
44. Findings - Where to put instructions
• Low literacy people and older participants made significantly
more errors and missed more fields in the accordion
presentation.
• Participants tended not to read page headers in the wizard.
Practitioner Takeaways
• Place instructions above text input field.
"
Reference: Alton,
N.,
Rinn,
C.
,
Summers,
K.,
and
Straub,
K.,
(Forthcoming)
Using
Eye-‐Tracking
and
Form
CompleHon
Data
to
OpHmize
Form
InstrucHons.
IEEE
–
IPPC
14.
Piesburgh,
PA
USA.
48. Findings - How to segment and present forms
• Low literacy users and older participants, and even expert users
(to a lesser extent) miss more items in accordion designs.
• Wizards help lower literacy and older individuals.
Practitioner Takeaways
• Avoid accordion presentations for presenting long forms.
"
Reference: Alton,
N.,
Rinn,
C.
,
Summers,
K.,
and
Straub,
K.,
(Forthcoming)
Using
Eye-‐Tracking
and
Form
CompleHon
Data
to
OpHmize
Form
InstrucHons.
IEEE
–
IPPC
14.
Piesburgh,
PA
USA.
52. Findings
• Mobile users click more frequently on an icon that says MENU.
Practitioner Takeaways
• Use words when you can.
"
Reference:
Test 1: http://exisweb.net/mobile-menu-abtest
Test 2: http://exisweb.net/menu-eats-hamburger
54.
The
trial
court
erred
in
giving
flawed
essenHal
elements
instrucHons
to
the
jury
and
thereby
denied
the
defendant
due
process
and
fundamental
fairness
since
it
is
error
to
give
the
jury,
within
the
essenHal
elements
instrucHons,
one
statement
containing
more
than
one
essenHal
element
of
the
crime
and
requiring
of
the
jury
simple
and
singular
assent
or
denial
of
that
compound
proposiHon,
fully
capable
of
disjuncHve
answer,
which
if
found
pursuant
to
the
evidence
adduced
would
exculpate
the
defendant.
The
trial
judge
erred
by
instrucHng
the
jury
to
affirm
or
deny
a
single
quesHon.
That
quesHon
included
all
the
major
elements
of
the
crime.
By
joining
all
the
major
elements,
the
judge
prevented
the
jury
from
acquinng
the
defendant
even
if
they
found
him
innocent
of
a
major
element
.This
error
denied
him
his
due-‐
process
rights.
55.
The
trial
court
erred
in
giving
flawed
essenHal
elements
instrucHons
to
the
jury
and
thereby
denied
the
defendant
due
process
and
fundamental
fairness
since
it
is
error
to
give
the
jury,
within
the
essenHal
elements
instrucHons,
one
statement
containing
more
than
one
essenHal
element
of
the
crime
and
requiring
of
the
jury
simple
and
singular
assent
or
denial
of
that
compound
proposiHon,
fully
capable
of
disjuncHve
answer,
which
if
found
pursuant
to
the
evidence
adduced
would
exculpate
the
defendant.
The
trial
judge
erred
by
instrucHng
the
jury
to
affirm
or
deny
a
single
quesHon.
That
quesHon
included
all
the
major
elements
of
the
crime.
By
joining
all
the
major
elements,
the
judge
prevented
the
jury
from
acquinng
the
defendant
even
if
they
found
him
innocent
of
a
major
element
.This
error
denied
him
his
due-‐
process
rights.
Findings
• The plain language lawyer was rated easy to understand,
trustworthy, logical and specific and concise by other lawyers.
• Lawyers and non-lawyer thought clients, juries and peers would
understand him better
• Lawyers would be more satisfied with the plain language lawyer
as their lawyer
Practitioner Takeaways
• Plain language enhances credibility, even for professionals.
"
Reference:
Straub,
K.,
Mahaffey,
S.
and
Cheek,
A.
(Forthcoming).
Even
lawyers
want
to
understand
their
lawyers:
New
evidence
showing
plain
language
increases
lawyers’
credibility.
To
be
presented
at
Clarity
2014,
Antwerp,
Belgium.
66. Findings
• Graphic designers prefer non-standard and pictorial
visualizations.
• Laypeople prefer the abstract/accurate visualizations.
• Standard and abstract visualizations reduce interpretive
response times.
Practitioner Takeaways
• If understanding your visuals are important to your story, test
them
Reference Petrie, H., & Power, C. (2012). Quispel, A., & Maes, A. (2014). Would you prefer pie or cupcakes? Preferences
for data visualization designs of professionals and laypeople in graphic design. Journal of Visual Languages & Computing,
25(2), 107–116.
73. Findings
• There is some overlap but also some non-overlap in the issues
users focus on and the issues UXers focus on
Practitioner Takeaways
• Be skeptical of your intuitions.
"
Reference:
Petrie,
H.,
&
Power,
C.
(2012).
What
do
users
really
care
about?
In
Proceedings
of
the
2012
ACM
annual
conference
on
Human
Factors
in
Compu@ng
Systems
-‐
CHI
’12
(p.
2107).
ACM
Press.
doi:10.1145/2207676.2208363
76. JUST
BECAUSE
ITS
SEXY
DATA,
DOESN’T
MEAN
ITS
USEFUL
Findings
• (Dead) Salmon respond differentially to human emotions
Practitioner Takeaways
• Be skeptical, particularly of new/emerging/sexy/neuroscienc-y
researchy methods.
"
Reference
Bennee,
Baird,
Miller,
and
Wolford
(2009)
Neural
correlates
of
interspecies
perspecHve
taking
in
the
post-‐
mortem
AtlanHc
Salmon:
An
argument
for
mulHple
comparisons
correcHon.
Compared to serial prototypers, parallel prototypers
Designs out performed on all measures
Click through rates
Time spent on site
Ratings by clients & professionals
Generated more diverse designs
Reported increased self-efficacy
Serial prototypers reported negative responses to critique
Experienced designers outperformed novices in performance, not diversity
Method
Participants: 33 adults from Chicago
Task: Filled out all 5 versions of the 15 field forms (Counterbalanced)
Between subjects comparison
Method
Participants: 33 adults from Chicago
Task: Filled out all 5 versions of the 15 field forms (Counterbalanced)
Between subjects comparison
166 German university community members ranging from 15-64 yrs old
(X = 27)
Complete a form with 7 fields including personal information
And password/login name.
Instructions varied across participants.
166 German university community members ranging from 15-64 yrs old
(X = 27)
Complete a form with 7 fields including personal information
And password/login name.
Instructions varied across participants.
18 expert users
Ages 18 - 47
72% male, 18% female
25 At-Risk Users
Ages 28 – 77
52% male, 48% female
9 participants over 60 years old
18 Low Literacy (REALM score under 60)
18 expert users
Ages 18 - 47
72% male, 18% female
25 At-Risk Users
Ages 28 – 77
52% male, 48% female
9 participants over 60 years old
18 Low Literacy (REALM score under 60)
18 expert users
Ages 18 - 47
72% male, 18% female
25 At-Risk Users
Ages 28 – 77
52% male, 48% female
9 participants over 60 years old
18 Low Literacy (REALM score under 60)
18 expert users
Ages 18 - 47
72% male, 18% female
25 At-Risk Users
Ages 28 – 77
52% male, 48% female
9 participants over 60 years old
18 Low Literacy (REALM score under 60)
18 expert users
Ages 18 - 47
72% male, 18% female
25 At-Risk Users
Ages 28 – 77
52% male, 48% female
9 participants over 60 years old
18 Low Literacy (REALM score under 60)
30 students majoring in graphic design
41 people majoring in something else
Is the ADP party the second biggest or second smallest party?
Attractiveness ratings 5 point scale Very unattractive to very attractive
Information retrieval Is the ALP the second from the top or the second from the bottom
Clarity rating 1-5 Very unclear to Very Clear
Overall rating Extremely bad – Extremely good (1-10)
Which three do you like best
Is the ADP party the second biggest or second smallest party?
Attractiveness ratings 5 point scale Very unattractive to very attractive
Information retrieval Is the ALP the second from the top or the second from the bottom
Clarity rating 1-5 Very unclear to Very Clear
Overall rating Extremely bad – Extremely good (1-10)
Which three do you like best
Attractiveness ratings 5 point scale Very unattractive to very attractive
Information retrieval Is the ALP the second from the top or the second from the bottom
Clarity rating 1-5 Very unclear to Very Clear
Overall rating Extremely bad – Extremely good (1-10)
Which three do you like best
Attractiveness ratings 5 point scale Very unattractive to very attractive
Information retrieval Is the ALP the second from the top or the second from the bottom
Clarity rating 1-5 Very unclear to Very Clear
Overall rating Extremely bad – Extremely good (1-10)
Which three do you like best
Attractiveness ratings 5 point scale Very unattractive to very attractive
Information retrieval Is the ALP the second from the top or the second from the bottom
Clarity rating 1-5 Very unclear to Very Clear
Overall rating Extremely bad – Extremely good (1-10)
Which three do you like best
Attractiveness ratings 5 point scale Very unattractive to very attractive
Information retrieval Is the ALP the second from the top or the second from the bottom
Clarity rating 1-5 Very unclear to Very Clear
Overall rating Extremely bad – Extremely good (1-10)
Which three do you like best