Myths about usability testing                                                                                 Copies of Sl...
Myths about usability testing                                              Copies of Slides  Dogma 1  ► Usability professi...
Myths about usability testing                                         Copies of Slides                      Sweep before  ...
Myths about usability testing                                             Copies of Slides                 Excerpt from Us...
Myths about usability testing                           Copies of Slides  Dogma 2  ► Data trumps opinion at any time      ...
Myths about usability testing                           Copies of Slides                                A Usability Bake-o...
Myths about usability testing                                      Copies of Slides                       Data to Fight   ...
Myths about usability testing                           Copies of Slides                                 Five users are en...
Myths about usability testing                                     Copies of Slides                       ”The parabola of ...
Myths about usability testing                            Copies of Slides                      Five users are             ...
Myths about usability testing                                Copies of Slides                                 The main goa...
Myths about usability testing                           Copies of Slides  Goals of usability test  1.    Find problems so ...
Myths about usability testing                           Copies of Slides                                 Expert reviews pr...
Myths about usability testing                           Copies of Slides  Beware!  Expert reviews can be political dynamit...
Myths about usability testing                           Copies of Slides                                 At least 25% of t...
Myths about usability testing                           Copies of Slides  Positive Findings ”A spoonful of sugar helps the...
Myths about usability testing                                            Copies of Slides  Required by UPA CPC  Positive f...
Myths about usability testing                                     Copies of Slides  Real World Data  Number of reported po...
Myths about usability testing                           Copies of Slides                                 Usability testing...
Myths about usability testing                                     Copies of Slides                           If you       ...
Myths about usability testing                                     Copies of Slides  Client Actions  Actions taken by clien...
Myths about usability testing                           Copies of Slides                                 Did this presenta...
Nächste SlideShare
Wird geladen in …5
×

Rolf Molich - “Five users will find 85% of the usability problems” – and other myths about usability testing.

3.877 Aufrufe

Veröffentlicht am

Rolf Molich owns and manages DialogDesign, a small Danish usability consultancy that he founded in 1993. Rolf conceived and coordinated the Comparative Usability Evaluation studies in which more than 100 professional usability teams tested or reviewed the same applications. Rolf was a principal investigator in the Nielsen Norman Group’s large-scale usability test of 20 US e-commerce websites, involving more than 60 users. He has worked with usability since 1984 and wrote the best-selling Danish book User Friendly Computer Systems, of which roughly 30,000 copies have been sold. The book is now available in English, with the title Usable Web Design. Rolf is also the co-inventor of the heuristic evaluation method (with Jakob Nielsen). Rolf is an experienced speaker. His 24 tutorials at the Nielsen Norman Group World Tour attracted roughly 1,000 participants. The overall average participant evaluation of his tutorials was 4.46 on a five-point scale.

Veröffentlicht in: Technologie
0 Kommentare
3 Gefällt mir
Statistik
Notizen
  • Als Erste(r) kommentieren

Keine Downloads
Aufrufe
Aufrufe insgesamt
3.877
Auf SlideShare
0
Aus Einbettungen
0
Anzahl an Einbettungen
1.695
Aktionen
Geteilt
0
Downloads
12
Kommentare
0
Gefällt mir
3
Einbettungen 0
Keine Einbettungen

Keine Notizen für die Folie

Rolf Molich - “Five users will find 85% of the usability problems” – and other myths about usability testing.

  1. 1. Myths about usability testing Copies of Slides ”Five users will find 85% of the usability problems” - and other myths about usability testing November 9, 2012 Rolf Molich, DialogDesign (www.DialogDesign.dk) This document and its content is Copyright ©2012 Rolf Molich and UCD UK Limited. Supporters Sponsors Organiser 1UCD 2012, London Rolf Molich, DialogDesign Page 1
  2. 2. Myths about usability testing Copies of Slides Dogma 1 ► Usability professionals must be highly usable Reports, presentations, responses to comments, etc. 3 When you point your finger at someone, remember that three fingers point at yourselfUCD 2012, London Rolf Molich, DialogDesign Page 2
  3. 3. Myths about usability testing Copies of Slides Sweep before your own door before you sweep the doorsteps of your neighbors Excerpt from Usability Test Report - CUE-6, Team GUCD 2012, London Rolf Molich, DialogDesign Page 3
  4. 4. Myths about usability testing Copies of Slides Excerpt from Usability Test Report - CUE-6, Team G Thank you for your attention! Any questions? Contact: Rolf Molich, DialogDesign, molich@DialogDesign.dk Picture credit: Rob Gonsalves – ”The Sun Sets Sail”UCD 2012, London Rolf Molich, DialogDesign Page 4
  5. 5. Myths about usability testing Copies of Slides Dogma 2 ► Data trumps opinion at any time 9 A Good Old American Bake-offUCD 2012, London Rolf Molich, DialogDesign Page 5
  6. 6. Myths about usability testing Copies of Slides A Usability Bake-off CUE-9: 19 Atlanta +16 Chemnitz Professionals 5 Videos >100 Different FindingsUCD 2012, London Rolf Molich, DialogDesign Page 6
  7. 7. Myths about usability testing Copies of Slides Data to Fight Unsubstantiated Myths and Voodoo -- CUE motto Please use a few minutes to mark your position on each of the above 5 statementsUCD 2012, London Rolf Molich, DialogDesign Page 7
  8. 8. Myths about usability testing Copies of Slides Five users are enough to catch 1 85% of the usability problems in practically any product Agree Both agree & disagree Disagree Truth or myth? Five users are enough to catch 1 85% of the usability problems in practically any product Agree Both agree & disagree Disagree Truth or myth?UCD 2012, London Rolf Molich, DialogDesign Page 8
  9. 9. Myths about usability testing Copies of Slides ”The parabola of optimism” (Jared Spool) Real data: CUE studies CUE = Comparative Usability Evaluation ► 10-17 professional teams test the same website. ► In all CUE-studies, more than 60% of problems were uniquely reported. ► In CUE-4, more than 300 issues were reported. 18UCD 2012, London Rolf Molich, DialogDesign Page 9
  10. 10. Myths about usability testing Copies of Slides Five users are enough to drive a useful iterative cycle The main goal of a usability test is 2 to discover usability problems Agree Both agree & disagree Disagree Truth or myth?UCD 2012, London Rolf Molich, DialogDesign Page 10
  11. 11. Myths about usability testing Copies of Slides The main goal of a usability test is 2 to discover usability problems Agree Both agree & disagree Disagree Truth or myth? Real cowboy programmers don’t need no stinkin’ usability test -- AnonymousUCD 2012, London Rolf Molich, DialogDesign Page 11
  12. 12. Myths about usability testing Copies of Slides Goals of usability test 1. Find problems so they can be corrected 2. Demonstrate convincingly to skeptical stakeholders (developers, management) that problems exist and that there are efficient ways to find and correct them 23 Expert reviews provide results that 3 are as reliable as those from usability tests Agree Both agree & disagree Disagree Truth or myth?UCD 2012, London Rolf Molich, DialogDesign Page 12
  13. 13. Myths about usability testing Copies of Slides Expert reviews provide results that 3 are as reliable as those from usability tests Agree Both agree & disagree Disagree Truth or myth? Expert review vs. Utest Data from CUE-studies, most notably CUE-4, consistently shows ► Expert reviews find as many problems as usability testing ► Expert reviews overlook problems – but so do usability tests ► Expert reviews are cheaper than usability tests These findings apply only when expert reviews are carried out by Experts 26UCD 2012, London Rolf Molich, DialogDesign Page 13
  14. 14. Myths about usability testing Copies of Slides Beware! Expert reviews can be political dynamite in organizations that are immature with respect to usability. Before doing an expert review, consider what you will answer an upset stakeholder or developer who asks you: ► “Why are your opinions better than mine?” 27 At least 25% of the comments in a 4 usability test report should be positive Agree Both agree & disagree Disagree Truth or myth?UCD 2012, London Rolf Molich, DialogDesign Page 14
  15. 15. Myths about usability testing Copies of Slides At least 25% of the comments in a 4 usability test report should be positive Agree Both agree & disagree Disagree Truth or myth? Positive Findings 1. Make it easier for you to sell inconvenient truths 2. Ensure that the development team doesn’t remove a feature that users actually liked 30UCD 2012, London Rolf Molich, DialogDesign Page 15
  16. 16. Myths about usability testing Copies of Slides Positive Findings ”A spoonful of sugar helps the medicine go down” Mary Poppins 31 Even developers have feelingsUCD 2012, London Rolf Molich, DialogDesign Page 16
  17. 17. Myths about usability testing Copies of Slides Required by UPA CPC Positive findings are required by the UPA Code of Professional Conduct ► The CPC requires you to "accurately report both the positive and negative feedback". ► If you can prove that there is nothing positive to say, then you have fulfilled the CPC. I am not sure how I would prove this. 33 When you have a hammer in your hand, everything looks like a nail -- The Law of the Instrument, Abraham KaplanUCD 2012, London Rolf Molich, DialogDesign Page 17
  18. 18. Myths about usability testing Copies of Slides Real World Data Number of reported positive findings for each CUE-9 team 50% 30% 10% 0 0 1 1 2 2 2 2 2 3 3 4 4 5 8 10 35 Usability testing can be conducted 5 by anyone Agree Both agree & disagree Disagree Truth or myth?UCD 2012, London Rolf Molich, DialogDesign Page 18
  19. 19. Myths about usability testing Copies of Slides Usability testing can be conducted 5 by anyone Agree Both agree & disagree Disagree Truth or myth? Quality testing can be conducted 5 by anyone Agree Both agree & disagree Disagree Truth or myth?UCD 2012, London Rolf Molich, DialogDesign Page 19
  20. 20. Myths about usability testing Copies of Slides If you don’t care about quality, everything else is trivial -- Gerald M. Weinberg 10 Assessments 1 professional passed with flying colors 2 professionals had serious deviations but no critical deviations from generally accepted good practice in usability testing. 5 professionals had both critial deviations and serious deviations 2 professionals did so badly that the assessor recommended that the client should not pay for the service, or that the test should be repeated at no cost for the client. A "serious deviation" from good practice may skew important results. A "critical deviation" from good practice will skew important test results seriously. 40UCD 2012, London Rolf Molich, DialogDesign Page 20
  21. 21. Myths about usability testing Copies of Slides Client Actions Actions taken by clients to prevent future quality problems: ► Develop and publish usable guidelines for good usability testing. The guidelines are based on observed user behaviour. ► Quality Assurance as part of procurement “Do a test while we observe” Is their understanding of a good usability test the same as ours? ► Make assessments available to teams and insist that they respond. Fortunately, most usability professionals take pride in doing things right 41 Certification ► The German UPA is currently working on a professional certification scheme for ►Foundation level (“Do You Speak Usability?”) ►Usability Engineer ►Usability Tester ► Certification procedure: ►40 multiple choice questions ►Relevant deliverable (advanced levels only) 42UCD 2012, London Rolf Molich, DialogDesign Page 21
  22. 22. Myths about usability testing Copies of Slides Did this presentation change some 6 of your views of usability testing? Yes Don’t know No Evaluation Takeaways ► As usability professionals we must set a good example for the world ► Our products, presentations, doings must be usable and useful: Correct, relevant, brief, constructive ► Five users are enough to drive a useful iterative cycle. Never claim that you can find all usability problems ► Expert reviews are valuable – but politically challenging ► Include positive findings 44UCD 2012, London Rolf Molich, DialogDesign Page 22

×