Presented at UX Brighton 2018: Advancing Research, 2nd November 2018

1. True stories of usability
benchmarking
Katie Taylor
Head of Experience Design
The Wellcome Trust
@katieminatie

2. Hello
Image: Wellcome Collection

3. What is usability
benchmarking?
What is usability
benchmarking?
Measuring the current usability to
provide a baseline against which
future versions can be compared

4. Jeff was born on 5th December 1959,
find out when he will be able to get his
state pension.

5. Jeff was born on 5th December 1959,
find out when he will be able to get his
state pension
% Correct % Incorrect % Abandon
% Timeout
(>10 mins)
Average time
taken

6. What is usability
benchmarking?
Why benchmark usability?
Has our work actually improved usability?
Improve product decisions
To set goals or the future
Make the case for teams to work together

7. http://bit.ly/2fxtf

8. Defined tasks
Driving
Passports
Travelling abroad
Visas
Tax
Benefits
Marriage abroad
Bank holidays
Pensions
Child maintenance

9. Defined method
Follow up:
moderated, in-
person sessions
Usability sessions:
remote,
unmoderated
Post-task survey:
satisfaction and
trust

10. How do we make findings relevant to
product teams?
How do managers use benchmarks?

11. Involve more people when defining
tasks.
Treat the first round like an alpha,
so you can iterate and improve.
(Tara Land)

12. 1.Benchmarking is prioritisation
2.The first benchmark is an alpha
3.The time to start is now

13. Image: Wellcome Collection

15. Choose which tasks to
measure, together

16. Julia has received her payslip and
thinks she is paying too much tax. Her
payslip says her tax code is 0T. Find
out the most likely reason why she is
paying too much tax.

17. Benchmarking measures the
whole experience, it’s up to
us to find the problem spots

18. http://bit.ly/2q5fj

20. 1.Benchmarking is prioritisation
2.The first benchmark is an alpha
3.The time to start is now

21. Treat the first round as an
alpha

22. Image: Wellcome Collection

23. http://bit.ly/2rnrk

24. 1.Benchmarking is prioritisation
2.The first benchmark is an alpha
3.The time to start is now

25. Why do you want to do it?
Which tasks will you benchmark?
What should you measure?
Do you need goals?
Prototype, run repeatable and robust sessions.
Think about reporting and who will use the results.

29. Image: Wellcome Collection

30. Why do you want to do it?
Which tasks will you benchmark?
What should you measure?
Do you need goals?
Prototype, run repeatable and robust sessions.
Think about reporting and who will use the results.

31. Fin