The presentation file for OpenHCI 2009 at Taipei

1. DESIGN FOR USABILITY
TESTING
by
Mumin Yu, User Experience Consultant, USER XING
Jen Wang, User Experience Specialist, MiTAC

2. What Are We Gonna Do Today

4. Don’t worry.
It is not that bad.

5. A TYPICAL USABILITY TEST

6. http://reportwritingskills.com/images/report_writing_skills11.jpg

7. http://www.pod.usace.army.mil/images/I-Want-You-Flat.jpg

9. http://theotherthomasotter.files.wordpress.com/2007/06/asug07-moraescreengrabs01.png

10. http://reportwritingskills.com/images/report_writing_skills11.jpg

11. PLANNING RECRUITING TESTING ANALYZING REPORTING

12. PLANNING RECRUITING TESTING ANALYZING REPORTING

13. GET OTHERS INVOLVED

14. http://www.cyber-cinema.com/gallery/others.jpg

15. Designers
Top Managers Product Managers
Sales
Marketings
Engineers
Customer Services

16. The more/earlier you communicate your plan,
the better the result will be.

17. GIVE ME A BREAK

18. If you are feeling tired, so is the participant!

19. Make no session longer than 2 hours.

20. Do no more than 4 sessions a day.

21. Allow at least 60 minutes break between sessions.

22. Projects Sun Aug 30 – Sat Sep 5, 2009 (Taipei)
Sunday 8/30 Monday 8/31 Tuesday 9/1 Wednesday 9/2 Thursday 9/3 Friday 9/4 Saturday 9/5
9am
10am
Test Session Test Session Test Session
10am - 12pm 10am - 11:30am 10am - 11am
11am
12pm
Test Session
12pm - 1pm
1pm
Test Session
1pm - 2:30pm
Test Session
2pm 1:30pm - 3:30pm
Test Session
3pm 2:30pm - 3:30pm
Test Session
4pm 3:30pm - 5pm
Test Session Test Session
5pm 4:30pm - 6:30pm 4:30pm - 5:30pm
6pm
7pm
8pm

23. Projects Sun Aug 30 – Sat Sep 5, 2009 (Taipei)
Sunday 8/30 Monday 8/31 Tuesday 9/1 Wednesday 9/2 Thursday 9/3 Friday 9/4 Saturday 9/5
9am
10am
Test Session Test Session Test Session
10am - 12pm 10am - 11:30am 10am - 11am
11am
12pm
Test Session
12pm - 1pm
1pm
Test Session
1pm - 2:30pm
Test Session
2pm 1:30pm - 3:30pm
Test Session
3pm 2:30pm - 3:30pm
Test Session
4pm 3:30pm - 5pm
Test Session Test Session
5pm 4:30pm - 6:30pm 4:30pm - 5:30pm
6pm
7pm
8pm

24. Projects Sun Aug 30 – Sat Sep 5, 2009 (Taipei)
Sunday 8/30 Monday 8/31 Tuesday 9/1 Wednesday 9/2 Thursday 9/3 Friday 9/4 Saturday 9/5
9am
10am
Test Session Test Session Test Session
10am - 12pm 10am - 11:30am 10am - 11am
11am
12pm
Test Session
12pm - 1pm
1pm
Test Session
1pm - 2:30pm
Test Session
2pm 1:30pm - 3:30pm
Test Session
3pm 2:30pm - 3:30pm
Test Session
4pm 3:30pm - 5pm
Test Session Test Session
5pm 4:30pm - 6:30pm 4:30pm - 5:30pm
6pm
7pm
8pm

25. Projects Sun Aug 30 – Sat Sep 5, 2009 (Taipei)
Sunday 8/30 Monday 8/31 Tuesday 9/1 Wednesday 9/2 Thursday 9/3 Friday 9/4 Saturday 9/5
9am
10am
Test Session Test Session Test Session
10am - 12pm 10am - 11:30am 10am - 11am
11am
12pm
Test Session
12pm - 1pm
1pm
Test Session
1pm - 2:30pm
Test Session
2pm 1:30pm - 3:30pm
Test Session
3pm 2:30pm - 3:30pm
Test Session
4pm 3:30pm - 5pm
Test Session Test Session
5pm 4:30pm - 6:30pm 4:30pm - 5:30pm
6pm
7pm
8pm

26. THINK OUT OF THE LAB

27. On Treadmill
http://www.bodyqueststore.com/Commercial/Treadmills/Nautilus%20718%20Treadmill.jpg

28. On Foot

29. In Car

30. PLANNING RECRUITING TESTING ANALYZING REPORTING

31. PLANNING RECRUITING TESTING ANALYZING REPORTING

32. SHOW THEM THE MONEY

33. http://blogs.edmunds.com/greencaradvisor/money_stack.jpg

34. Enough to motivate them,
but not enough for them to pretend

35. NT$1000 per hour seems to work just fine for
now.

36. What if you don’t have the money?

37. GIFT CERTIFICATE

38. FOOD
http://www.singleandhappy.net/.a/6a00e554e9a52b88330112797c5b5328a4-320wi

39. SCHOOL CREDIT

40. THANK YOU
http://upload.wikimedia.org/wikipedia/commons/thumb/6/6a/Heart_left-highlight_jon_01.svg/491px-Heart_left-highlight_jon_01.svg.png

41. SNOWBALL EFFECT

42. http://wpcontent.answers.com/wikipedia/commons/thumb/2/23/Giant_snowball_Oxford.jpg/180px-Giant_snowball_Oxford.jpg

43. You still need to do the screening yourself

44. PLANNING RECRUITING TESTING ANALYZING REPORTING

45. PLANNING RECRUITING TESTING ANALYZING REPORTING

46. A GOOD EXAMPLE OF A BAD EXAMPLE
http://www.breadwig.com/uploads/sketches/bunnies/000554.jpg

47. Shut Up and Listen/Watch!

48. RULES OF ENGAGEMENT

49. You need participants to move on to the next task
because time is limited.

50. You need to bring participants back on track
because they have gone too far down the wrong
path.

51. Participants are stuck. You think you can learn
more if they continue.

52. Moderator, please give them more time.

53. WATCH YOURSELF

54. WATCH OUT FOR
Less-than-neutral questions
Unfocused sessions
Annoying gestures
Repeating words
Facial expressions
more...

55. PLANNING RECRUITING TESTING ANALYZING REPORTING

56. PLANNING RECRUITING TESTING ANALYZING REPORTING

57. METRICS

58. (!"
'!"
!"#$%&'%!()*+,)$-.+
&!"
%!"
$!"
#!"
!"
)*+,-"#" )*+,-"$" )*+,-"%" )*+,-"&" )*+,-"'" )*+,-"(" )*+,-"."
PERFORMANCE METRICS

59. PERFORMANCE
Task success
Time-on-task
Errors
Efficiency
Learnability

60. )"
("
!"#$%&'()'*+&,-.*+/01'
'"
&"
%"
$"
#"
!"
*+,"-./"011" 7.-8501,"9:" *+,"-./"4-.?"3.?" 7.-8501,";9.5/"
23124.5/"6+-4" -;<+=;>.-"23.+210" /."@1/5@-"/." 03+AA+-=".A>.-0"
03.AA+-="
ISSUED-BASED METRICS

61. ISSUES
Producing an error
Assuming a task is complete when it is not
Preventing task completion
Creating some level of confusion
Not seeing something that should be noticed
Assuming something is correct when it is not

62. ()*+,-.*))"
'!!"
&!"
%!"
$!"
#!"
607)+0871." !" /0)*"1+"()*"
/0)*"1+"2*03.4.5"
SELF-REPORTED METRICS

63. SELF REPORTS
After-Scenario Questionnaire (ASQ)
Post-Study System Usability Questionnaire (PSSUQ)
Computer System Usability Questionnaire (CSUQ)
System Usability Scale (SUS)
Questionnaire for User Interface Satisfaction (QUIS)
Usefulness, Satisfaction, and Ease of Use Questionnaire (USE)

64. BEHAVIORAL & PHYSIOLOGICAL METRICS
http://www.simpleusability.com/images/heuga-home-eye-plot.jpg

65. BEHAVIOR & PHYSIOLOGY
Verbal behaviors
Facial expression
Eye-tracking
Pupillary
Skin conductance
Heart rate

66. PLANNING RECRUITING TESTING ANALYZING REPORTING

67. PLANNING RECRUITING TESTING ANALYZING REPORTING

68. Number of usability reports to read 5
4
3
2
1
1 2 3 4 5
Hours slept during office hours
USABILITY REPORTS VS OFFICE HOURS SLEPT
http://www.slideshare.net/gregbell/usability-20

69. Not everyone needs to read a long report.

70. So you need to have a more effective way to
communicate your findings

71. A VIDEO IS WORTH A THOUSAND WORDS

72. CASE STUDY