This document summarizes post-test feedback mechanisms for assessing user experiences. It discusses the System Usability Scale (SUS) questionnaire and Product Reaction Cards (PRC). An example is given of an international hotel group that tested a "green" initiative over three rounds, seeing improvements in SUS scores and positive PRC selections. The document encourages using both SUS and PRC to get a comprehensive view of user experience. It provides details on how to administer and analyze PRC in person and remotely.

1. Post-test feedback
mechanisms
Assessing user experiences in 2 ways
Presented by Carol Barnum, Ph.D.
UX Firm, LLC

2. SUS
PRC

3. System Usability Scale
Strongly
Disagree
Strongly
Agree
1. I think that I would like to use this website frequently.
2. I found this website unnecessarily complex.
3. I thought this website was easy to use.
4. I think that I would need assistance to be able to use
this website.
5. I found the various functions in this website were well
integrated.
6. I thought there was too much inconsistency in this
website.
7. I would imagine that most people would learn to use
this website very quickly.
8. I found this website very cumbersome/awkward to
use.
9. I felt very confident using this website.
10. I needed to learn a lot of things before I could get
going with this website.
This questionnaire is based on the System Usability Scale (SUS), which was developed by John Brooke
while working at Digital Equipment Corporation. © Digital Equipment Corporation, 1986.

4. Average score
68

5. International hotel group
“green” initiative

6. Round 1
54
Round 2
73
SUS scores for 1st and 2nd test

7. SUS scores for 3rd test
Round 1
54
Round 2
73
Round 3
94!!!!!!

8. Microsoft creates desirability toolkit
1. Faces Questionnaire
2. Product Reaction Cards
118 Cards
6 Faces

9. What’s in the cards?

10. In person
 Spread them out on table
 Instruct user to
◦ walk along the table and pick up cards that express the user’s experience
◦ Share the meaning of the cards
◦ User’s story emerges
In remote testing
 Provide a Word table or Excel spreadsheet
◦ User highlights selections
◦ Explains choices
In analysis
 Count the total number of cards selected
 Collate the results in clusters of similar/same cards
 Present the results in a word cloud or other visual display
How to use the cards

12. Several examples

14. 14
3 TV weather websites
0
5
10
15
20
25
30
35
40
Positive
Negative
Positive
Negative
Positive
Negative
Station A
Station B
Station C
26/13 39/5 24/17

15. Repeated positive card selections
focused on ease of use,
relevance, and speed

Easy-to-use
Helpful
Straightfor
ward

Fast
Relevant
Reliable
Useful

16. The 1-2 punch. Use both!

17. International “green” initiative
Round 1= 
SUS = 52
42% PRC
positive
Round 2= 
SUS = 73
82% PRC
positive
Round 3 =  
SUS = 96
100% PRC
positive

18. For more . . .
http://booksite.elsevier.c
om/barnum/testingessent
ials/
Contact me
carolbarnum@uxfirm.com