workshop for UXPA DC on April 12, 2014, entitled "All this UX data! Now what?" Attendees learned how to deal with large amounts of user experience data from tests, and how to combine certain data to tell a succinct story.

1. So much UX data! Now what?
Jennifer Romano Bergstrom
April 12, 2014
UXPA-DC Workshop| Arlington, VA
@romanocog

2. 2
Measuring the UX
• How does it work for the
end user?
• What does the user
expect?
• How does it make the
user feel?
• What is the user‟s story
and habits?
• What are the user‟s
needs?
Usability = “the extent
to which a product can
be used by specified
users to achieve
specified goals with
effectiveness, efficien
cy, and satisfaction in
a specified context of
use.” ISO 9241-11
+ emotions and
perceptions = UX
@romanocog @uxpadc

3. 3
Usability vs. User Experience (UX)?
The 5 Es to Understanding Users (W. Quesenbery): http://www.wqusability.com/articles/getting-started.html
User Experience Design (P. Morville): http://semanticstudios.com/publications/semantics/000029.php
Whitney‟s 5 Es of
Usability
Peter‟s User Experience
Honeycomb
@romanocog @uxpadc

4. 4
Krug, S. Don‟t Make Me Think
What People do on the Web @romanocog @uxpadc

5. 5
UX data
OBSERVATIO
NAL+ Ethnography
+ Time to complete task
+ Reaction time
+ Selection/click behavior
+ Ability to complete tasks
+ Accuracy
IMPLICIT
+ Facial expression analysis
+ Eye tracking
+ Electrodermal activity (EDA)
+ Behavioral analysis
+ Linguistic analysis of
verbalizations
+ Implicit associations
+ Pupil dilation
EXPLICI
T+ Post-task satisfaction
questionnaires
+ In-session difficulty ratings
+ Verbal responses
+ Moderator follow up
+ Real-time +/- dial
@romanocog @uxpadc

6. Explicit data
6
*Satisfaction Questionnaire Q7: Please rate how difficult it was to log in on this device. 1= not difficult at all to 5= extremely
difficult.
0
3
6
9
Participant Ratings
How likely would you be to recommend this site to a
friend?
Not likely at all or
Slightly likely
Moderately Likely
Very Likely
0
3
6
9
Participant Ratings
How likely would you be to use this site in
the future?
Not likely at all or
Slightly likely
Moderately Likely
Very Likely
“Love the picture in the middle of it.”
“It looks very clean and very simple.”
“It looks pretty organized, it's a nice design.”
When asked how they would save
information, four of six participants said
they would bookmark the page or take a
screenshot of the information. Only two
mentioned that they would use the site
functionality to save for later use.
83%
9%
9%
Percentage of Difficulty Ratings*
1 & 2
3
>=4
@romanocog @uxpadc

8. 88
Implicit data
8
6
6.5
7
7.5
8
8.5
9
00:00.00 00:07.50 00:15.00 00:22.50 00:30.00 00:37.50 00:45.00 00:52.50
P29 iPhone
Begin date and
time selection.
End date and
time selection.
Trouble with scrolling. More trouble with scrolling.
@romanocog @uxpadc

9. 99
Implicit data
9
6
6.5
7
7.5
8
8.5
9
00:00.00 00:07.50 00:15.00 00:22.50 00:30.00 00:37.50 00:45.00 00:52.50
P29 iPhone
Begin date and
time selection.
End date and
time selection.
Trouble with scrolling. More trouble with scrolling.
@romanocog @uxpadc

10. Eye tracking
10
@romanocog @uxpadc

11. Heat maps
11
@romanocog @uxpadc

12. @romanocog @uxpadcGaze plots
12

13. @romanocog @uxpadcGaze plots
13

14. @romanocog @uxpadcGaze plots
14

15. Gaze plots and comments
15
M
“Man, this is a long paragraph.”
“There's a lot of information, it'd be a lot better in list form. Ideally, you want to
get your information quick without reading through all this.”
@romanocog @uxpadc

16. Usability test of a low-fi prototype
16
@forsmarshgroup @romanocog
@romanocog @uxpadc
• video

17. Usability test of a high-fi prototype
17
@forsmarshgroup @romanocog
@romanocog @uxpadc
• video

18. More data
18
Fixation count Heat map of the Round 2 home page
from 7 participants during the “scam” task
Left
navigation
Center
icons
Mean time
elapsed before
AOI is fixated
79.55 (18.54).
N = 8
29.36 (9.2).
N = 6
0
3
6
9
1
#ofParticipants
“How clear is the information on
this page?”**
Not at all to Slightly Clear
Moderately Clear
Very to Extremely Clear
@romanocog @uxpadc

19. 19
• “Where and how you click is a bit counter-intuitive. [It‟s] not
super obvious which button to click to get to next sections.”
• “I feel like the „Next‟ should be at the bottom and not the top.”Intuitive „Next‟ button location
Non-intuitive „Next‟ button location
Usability test across devices @romanocog @uxpadc

20. Combining data
20
• “I‟m not expecting them to email
or call me. I don‟t expect any
person to notify me.”
• “I‟m not sure when I‟d get an
answer. It‟s not like Yahoo!
Answers where it‟s immediate.”
Participants had different
expectations about what would
happen next.
• Five expected to hear back
via email
• One expected an immediate
response
• One expected to see the
answer posted somewhere
on the site
• Two said they would call for
assistance.
@romanocog @uxpadc
video

21. So how much data do I need?

22. 22
Considerations
• Qualitative
• Quantitative
• Iterative testing
• User groups
@romanocog @uxpadc

23. Questions
Jennifer Romano Bergstrom
@romanocog
jbergstrom@forsmarshgroup.com
UXPA-DC Workshop| Arlington, VA