User Testing workshop ran at the inaugural UX South West event.

1. USER TESTING
WORKSHOP
Joe Dollar-Smirnov @fruitskin
Director of UX, Red Badger & Lead Instructor, General Assembly

2. USER TESTING WORKSHOP
OBJECTIVES
‣ Set up scenario
‣ Run a usability test
‣ Record feedback

3. USER TESTING WORKSHOP
APPROACH

4. USER TESTING WORKSHOP
WHAT DOES IT TAKE?
‣ Bravery
‣ Humble Pie

5. USER TESTING WORKSHOP
TOOLS
‣ Analytics Tools - GA, Web Trends, Optimal Workshop, Qubit, Crazy Egg,
Inspectlet the list goes on…
‣ Silverback, Camtasia, Screenflow
‣ Webcam, GoPro
‣ Paper, Pens and Empathy

6. USABILITY & USER TESTING
METHODS
‣ Formal
‣ Guerilla (Cafe Test)
‣ Remote
‣ Unmoderated
‣ Face to Face

7. USABILITY & USER TESTING
METHODS
‣ Formal
‣ Guerilla (Cafe Test)
‣ Remote
‣ Unmoderated
‣ Face to Face

8. USABILITY & USER TESTING
METHODS
‣ Formal
‣ Guerilla (Cafe Test)
‣ Remote
‣ Unmoderated
‣ Face to Face

9. USABILITY & USER TESTING
METHODS
‣ Formal
‣ Guerilla (Cafe Test)
‣ Remote
‣ Unmoderated
‣ Face to Face

10. USABILITY & USER TESTING
METHODS
‣ Face-to-face
‣ This is the recommended method
‣ Provide users with tasks to complete on a prototype
‣ Encourage them to talk out loud as they move through
interactions
‣ Observe their actions

11. USABILITY & USER TESTING
PREPARATION
‣ Create task scenarios
‣ Should describe the task in a clear and unambiguous way
‣ Written in the user’s language (don’t use jargon)
‣ Don’t prompt the solution

12. USABILITY & USER TESTING
PREPARATION
‣ Create a discussion guide
‣ Should keep the participant talking (think aloud protocol)
‣ You are trying to learn without leading
‣ Keep the feedback flowing
‣ Prepare to note down reactions not just words
‣ “What would you expect” not “Do you like?”

13. USABILITY & USER TESTING
OUTPUTS
‣ Notes
‣ Video, Audio
‣ Usability Labs will have highly finished reports
‣ Unmoderated tests - data reports and recorded sessions
‣ Report good and bad, don’t make conclusions (yet), stick to
what you see
‣ SUS Score?

14. USABILITY & USER TESTING
THE TESTS
‣ QUIS - Questionnaire for User Interaction Satisfaction
‣ SUMI - Software Usability Measurement Inventory
‣ PSSUQ - Post-Study System Usability Questionnaire
‣ SUS - Software Usability Scale
‣ ASQ - After-Scenario Questionnaire
‣ ER - Expectation Ratings
‣ UME - Usability Magnitude Estimation
‣ SEQ - Single Ease Question
‣ SMEQ - Subjective Mental Effort

15. USABILITY & USER TESTING
SUS
‣ Since mid-80s by John Brooke
‣ Benchmark and Summative
‣ Self Described as “Quick and Dirty”
‣ Scale of 1 - 5 (strongly disagree - strongly agree)

16. USABILITY & USER TESTING
THE QUESTIONS
1. I think that I would like to use this system frequently.
2. I found the system unnecessarily complex.
3. I thought the system was easy to use.
4. I think that I would need the support of a technical person to be able to use this system.
5. I found the various functions in this system were well integrated.
6. I thought there was too much inconsistency in this system.
7. I would imagine that most people would learn to use this system very quickly.
8. I found the system very cumbersome to use.
9. I felt very confident using the system.
10. I needed to learn a lot of things before I could get going with this system.

17. EXERCISE
KEY OBJECTIVE(S)
TIMING
1 min
1 min
5 min
5 min
DELIVERABLE
ACTIVITY
Write scenarios for your test that allows you to carry out usability testing.
Scenarios
1. Split group and choose website/app or per group
2. Get in to pairs
3. Create a scenario
4.Discuss

18. EXERCISE
KEY OBJECTIVE(S)
TIMING
2 min
5 min
DELIVERABLE
ACTIVITY
In your pairs write a testing discussion guide.
1.Write a testing discussion guide as a group
2. Refine for your own needs
Usability testing discussion guide

19. EXERCISE
KEY OBJECTIVE(S)
TIMING
1 min
8 min
1 min
10 min
DELIVERABLE
ACTIVITY
Carry out a usability test , take notes and ask the participant to fill in
a Software Usability Scale questionnaire
None.
1.Choose another pair
2. Tests project on the each other (take notes)
3.Ask participant to complete SUS
4. Swap and repeat

20. EXERCISE
KEY OBJECTIVE(S)
TIMING
5 min 1. Do it on the board
DELIVERABLE
ACTIVITY
Calculate SUS as a group
None.

21. USABILITY & USER TESTING
THE MATHS
‣ Odd numbered questions (x-1)
‣ Even numbered questions (5-x)
‣ Sum * 2.5 = Score out of a 100 (not a percentage)
‣ Average is 68 across over 5000 test results in different studies

22. USER TESTING WORKSHOP
FURTHER READING
‣ Quantifying the User Experience, J. Sauro & J. R. Lewis

23. USER TESTING WORKSHOP
THANKS!
Q&A