Getting Started with Usability Testing
•Als PPT, PDF herunterladen•
3 gefällt mir•1,488 views
This document provides an agenda and overview for a workshop on getting started with usability testing. The workshop will include introductions, presentations on measurements and testing, and activities for planning test logistics and interacting with participants. It will cover why usability testing is important, common myths, and tips for getting started. Participants will learn about planning and preparing for tests, recruiting profiles, scripts, test materials, facilitating sessions, and reporting results. The goal is to help attendees understand the basics of usability testing and conduct initial tests.
Empfohlen
Empfohlen
Weitere ähnliche Inhalte
Was ist angesagt?
Was ist angesagt? (20)
Andere mochten auch
Andere mochten auch (9)
Ähnlich wie Getting Started with Usability Testing
Ähnlich wie Getting Started with Usability Testing (20)
Mehr von UXPA International
Mehr von UXPA International (20)
Kürzlich hochgeladen
Kürzlich hochgeladen (20)
Getting Started with Usability Testing
- 1. Getting Started with UsabilityTesting Instructors: Thyra Rauch, John Schrag, and Carol Smith UXPA 2016
- 2. Schedule 05:30 Introductions, Agenda, Materials 05:40 Measurements & Testing 06:05 Activity I 6:25 Planning Test Logistics 6:40 Activity II 7:00 Selecting & Preparing Materials 7:10 Interacting with Test Participants 7:30 Break 8:00 Activity III 8:20 Testing Experiences 8:25 Activity IV and Team Presentations 9:10 Team Presentations 9:25 Final Q&A and Wrap-Up 9:30 Adjourn 2
- 4. Why Test? You are not your user May miss details; too close to design Unforeseen requirements Validate understanding of tasks and context More satisfied users Less training needed Fewer services calls 4
- 5. Myths of Usability Testing Need a lab Testing occurs at the end of the dev cycle Must test everything Need to test 100 people A “good” test means no changes Usability is “system testing”, “user-acceptance testing”, etc. Need a large test team, each playing a specific role/part 5
- 6. Just Do It! Make it realistic Any location Any stage of development (earlier is better) Start small Test an idea (not a feature) Just a few users Anytime (un-moderated) 6
- 8. Plan and Prepare for the Test • Scope the effort –Budget, resources, goals, time –Put together a test team • Design the test –Goals, participants, tasks, measures, methodology, team roles etc. –Summative or formative? 8
- 9. Testing Plan Goals What testing and how to measure success Test protocol – remote or in person? User groups and participants Recruiting/incentives Scenarios Share with team Keep concise 9
- 10. Create Test Scenarios Representative of typical tasks Describe what, not how “You want to buy a shirt for your father. Find a man’s blue dress shirt for under $40.” “As a systems administrator for your company, you want to install, configure, and test the xyz application.” Avoid leading wording (e.g., actions) 10
- 11. Measurement s Success Time on task % of tasks completed/not completed Number of steps to accomplish task Learning time Number of errors Number of times help consulted Satisfaction 11Photo By Kenyaboy7Andrew Chipley http://www.flickr.com/photos/kenyaboy7/3749535540/
- 12. Facilitator Role Conducts test and manages time and frustration Communicates with participant (before, during, after) Prompts for more information Ensures legal requirements are met (NDAs, recording permissions) 12
- 13. Additional Roles Observer(s)* Takes careful notes Observations (notes inferences as well) Exact quotes Context “Computer” (paper prototyping, wizard of oz) Mimics actions of computer 13 *See guidance for Observers in Resources section
- 15. Activity I 15
- 16. Basic Usability Study Select item to test Discover stakeholder’s needs High level goals Measurements to be made High level scenario (sentence or two) 16
- 18. Recruit Participants Use a participant screening profile Two weeks to complete recruit (average) Recruiters Marketing or product marketing department Professional recruitment agency Members of your team or yourself 18
- 19. Run a Pilot Study Discover problems with study or concept being tested Estimate time needed for the test Refine script and tasks for test Verify if tasks are typical (is it something users actually do?) Practice before going live with participants Get new ideas for follow-on questions or things to observe 19
- 20. Do I need a lab? 20 Rubin, Jeffrey. Handbook of Usability Testing. John Wiley & Sons, Inc.; 1994. Computer / Concept Facilitator Participant Observer Timer Logger
- 21. Portable Labs Temporary setup Capture screen, video and audio Complicates observation Anywhere (conference room, remotely) 21Photo by Roebot at http://www.flickr.com/photos/roebot/2964156413/
- 22. Remote Testing Moderated and Automated Capture screen, video and audio (not all automated collect this) Moderated: Connect to both computer and phone Technology Bandwidth, operating system, browser Test setup and connections ahead of time 22
- 23. Activity II 23
- 24. Recruiting Profile Key elements Mimic population using product Critical factors that make a difference Age Experience How they will be screened Phone Online Follow-up and scheduling 24
- 25. Recruiting Profile Samples Why decisions made What going after Create recruiting profile for participants for usability study designed in Activity 1 25
- 27. Why Use a Script? Promotes consistency in conducting the test Reminds to welcome and thank people Reminds to reset/configure between participants Ensures NDAs and other legal requirements are not missed Prepare for non-native language speakers Tell me more… 27
- 28. Parts of a Script Welcome to participants Steps in test (forms, tutorials, tasks, questions) Thank you to participants (and incentives if any) Notes to yourself Reset/configuration prompts 28
- 29. Test Folders Organize materials per participant Easily access individual results Store as a group Prepare a folder for each participant All test materials (script, forms, printed tasks etc.) Don’t use names- label folders: Pilot, P1, P2, P3 29
- 30. Informed Consent Your responsibility to inform participant Every time Ethical and legal responsibility Do no harm (physical, emotional) Explain consent form* Plain language – what, why, stop anytime, etc. Participant signs (Google form remotely) Rarely lose participants over form 30 *Examples in Appendix
- 32. Welcome Make sure the participant feels welcomed and appreciated. Stress that you are not testing them, rather you are testing… Encourage feedback, positive or negative “I was not involved in the design of this so you can’t hurt my feelings” 32
- 33. Facilitating Follow your script Frame questions in unbiased manner Remain passive in facial expressions and body language as much as possible while participant is working Don’t lead or give clues (unless frustration is clear and evident) Probe don’t demand Gently prompt during think-aloud protocol if necessary 33
- 34. Probes Ask open-ended questions Encourage participant to elaborate Use participant’s words Listen for vocalizations and watch non-verbal gestures Have participant tell you when they are done Give them time to think 34
- 35. Example Phrases How does this compare with your expectations? What specifically are you referring to? What are you looking for? Tell me more… Can you give me an example? 35
- 36. Common Issues Refusal to critique Ask preference and to compare interfaces Watch what they DO, not what they SAY Taking control - diverting to their issue Let them see you are listening and writing it down If no resolution, thank and end session - give them the incentive. 36
- 37. Getting Users Unstuck It’s not a question of “if” but “when” Users might be embarrassed at what they perceive to be their fault Users may not be able to articulate what they don’t understand Don’t directly answer initial questions 37
- 38. Getting Users Unstuck Progressive assistance What are you trying to do right now? What do you think the next step is? What would you do in this situation if you were at work? Hints Do you see anything that might help you? (small hint) Have you checked the Help? (medium hint) What do you think the xxx button does? (large hint) End before frustrated, or take a break 38
- 39. Debriefing Verbal and written questions What did you like best? What was the worst thing? What can be improved? Describe in 3 words Follow up to specific items noted during testing Ratings Ease of use Willingness to use Thank participant 39
- 40. Activity III 40
- 41. Probes – Group Exercise Are you thinking of using the search feature? Are you trying to find the price of the book? 41
- 42. Alternatives Are you thinking of using the search feature? (biased) Can you tell me what you are thinking right now? (unbiased) Are you trying to find the price of the book? (biased) What are you trying to do? (unbiased) 42
- 43. Rational Can’t know what participant is thinking Keep questions brief and clear Avoid Asking participants to predict the future Putting “words in their mouth” Showing bias or hinting at your opinions 43
- 44. Probes – Team Exercises Break into teams Each get a set of biased probes Generate unbiased alternatives Present/discuss with entire group 44
- 46. Low- fidelity/Paper Prototypes Includes wireframes Good time to get initial feedback on concept, word choices and placement Participants May not be able to understand May not be act the same as online Always test again later in development 46
- 47. High Fidelity Prototypes Ideal time for testing – early enough in development, but participants can “kick the tires” May be difficult to test with (broken links, undeveloped material, etc.) 47
- 48. Functioning Products Great for redesign projects – prior to start Get baseline and goals for improvements Reduce task sequence from 10 clicks to 5 Reduce time to perform task from 10 min to less than 3 min Increase satisfaction from 4 to 5 on a 5-pt scale Use existing data (Analytics, Help Desk, etc.) to create tasks and inform study 48
- 49. Reporting What was tested, how and why Success and Failure and why Recommendations Future testing – benchmarking Just enough to communicate – short & sweet 49
- 50. Activity IV 50
- 51. Activity 4 Mock up (15 minutes) Round 1 of testing (5 minutes) Revise mock ups (10 minutes) Round 2 of testing (5 minutes) Prepare team presentation (10 minutes) 51
- 52. Hotel Reservation Kiosk Design and develop a new hotel check-in kiosk that can be installed in airports Hotel chain wants to make it easy for their customers: Check into their room at the airport Request a particular room type, if available Request transportation to the hotel 52
- 53. Ideal Scenario During your search for a local phone book, you notice a hotel chain’s digital Kiosk, and you decide to try it. You need to: Reserve a Hotel Room for the evening. (A room with 1 bed for 2 people, non-smoking) Request transportation to the Hotel Reserve transportation from the hotel back to the airport 53
- 54. Alternative Scenario You want to go to the hotel right away to change before going to a fancy restaurant downtown. You need to: Know if the hotel provides transportation. Reserve a room with a King-Size bed because you're fat and you want a view of the ocean. Make sure that your “friend” who is coming tomorrow will have a room available next to yours. 54
- 57. Survey Please complete the session survey: www.uxpa2016.org/survey/69 57
- 58. Contact Us Thyra Rauch IBM trauch@us.ibm.com +1 (408) 463-2465 Carol Smith IBM smithjc@us.ibm.com +1 (412) 212-8737 @carologic 58 John Schrag Autodesk john.schrag@autodesk.com @jvschrag
- 59. Resources 59
- 60. Usability by the numbers… 80% of SW life cycle costs occur during maintenance 80% of maintenance costs are due to unmet or unforeseen user requirements Only 20% are bugs or reliability problems Typical software program released with 40 usability design problems Over 70% of CIO's state that one of their biggest problems is communicating with users to understand their needs 60
- 61. Number of Participants • Contentious issue • Nielsen - 5-7 participants uncover ~80% of major usability problems • IF representative • IF doing representative tasks. • Testing more = significant diminishing return. • Virzi and Carol Barnum • Small number of users is sufficient. 61http://www.useit.com/alertbox/20000319.html Jakob Nielsen’s Alertbox. Why You Only Need to Test with 5 Users. March 19, 2000. Number of Test Participants
- 62. Recruitees Could be Customers General consumers Members of organization or company Difficulty of recruiting Specificity of screener Availability/interest in study (doctors vs. consumers) Experience (highly experienced vs. novice) 62
- 63. Recruiting Consideration s Accessibility of test location Include people with disabilities “We are all only temporarily able-bodied. Accessibility is good for us all.” Language interpreters Remote and on-site studies Technology availability Administrative rights Internet connectivity 63 Quote by @mollydotcom at #stirtrek 2011 via @carologic
- 64. Data Collection Forms Customize for each study Space for writing notes – each scenario spelled out. Include space for tickmarks regarding easily measurable activities. E.g. Participant accesses video clip E.g. Participant selects advertisement 64
- 65. Tool Consideration s In person or remote? Lab or on-site? Prototype limitations (can it be online?, is it a document or a clickable site?) Number of observers, number of participants? Number of facilitators? Logging and video editing needs (time on task, highlight video creation)? Surveys before or after? Eye tracking? 65
- 66. Usability Testing Software Morae Ovo SilverBack (Mac only) Loop11 (Remote only) Tobii (Eye-tracker) SMI (Eye-tracker) SurveyMonkey 66
- 67. Screen Sharing Software GoToMeeting – http://www.gotomeeting.com Lotus Sametime Unyte – http://www.unyte.com YuuGuu -- http://www.yuuguu.com WebEx – http://www.webex.com Yugma -- https://www.yugma.com/ Trouble Shooting: CoPilot - https://www.copilot.com/ 67
- 68. Satisfaction Questionnaires Standard Usability Measurement Inventory (SUMI) office/desktop software, purchase 50 questions Website Analysis and MeasureMent Inventory (WHAMMI) Purchase 20 questions System Usability Scale (SUS) Free 10 questions 68
- 69. Ethics Resources Treatment of human subjects Web-based course Gives certificate of completion http://cme.cancer.gov/clinicaltrials/learning/human participant-protections.asp Guidelines for writing informed consent http://ohsr.od.nih.gov/info/sheet6.html UXPA Code of Conduct 69
- 70. NDA’s Do I need a non-disclosure agreement (NDA)? Maybe Legal department (be friendly) One-way, two-way* AKA Confidentiality agreement (CA) Confidential disclosure agreement (CDA) Proprietary information agreement (PIA) Secrecy agreement (SA) 70
- 71. Activity ______________________________________ Who are you observing? (e.g. SME in soft-goods manufacturing, application admin) What is the situation? ___________________________ (e.g. phone interview, their work at their desk in person, remote usability study, etc.) Date: __________________ Time: __________________ Location: ________________ Note-Taker: ______________ People (P) Who are you observing? Who else is involved in the experience? Objects (O) What objects do they interact with? What apps? Phone, etc.? Environment (E) Characteristics of the setting, number/types of interruptions Messages (M) What is communicated? How is it transmitted? Services (S) What services are available to them? Watch for these types of observations and put additional Questions/Ideas on the back. Content in this document described by Vijay Kumar in his book: 101 Design Methods Physical What do they interact with? Cognitive How associate meanings? How learn? Social Interactions, decision making, scheduling, work? Cultural What are shared norms, habits, values? Emotional What emotions are expressed and how? What are your biggest takeaways? What did you learn? What surprised you? What is the participants need? Interview & Observation Notetaking Form
- 72. Observer RulesEveryone who observes a design session is asked to abide by a set of rules.The purpose of these rules is to minimize stress for the participants and to maximize the amount of information we get from the study. Stay for the entire study Distractions are unhelpful and participants may get the impression that you’re leaving because they’ve done something wrong (e.g. walking out in middle of a movie). If you can attend only part of a study, discuss with the facilitator beforehand to determine whether there is a way to accommodate this. Don’t reveal information about the study to the participant It is often more useful to explore an area of difficulty in detail rather than try to “get through” all the topics. The facilitator will track time so that we can cover as many of the important areas as possible. Respect participants and the confidentiality of their data We have promised the participants that their participation is confidential.This means that we should not include their names in any reports or other communication such as email, and we should refrain from discussing them by name outside the test setting. Do not make negative comments about people—there is always a risk that a derogatory comment could be overheard or otherwise make its way back to the user. Adapted from the book Paper Prototyping by Carolyn Snyder, published by Morgan Kaufmann Publishers. Copyright (c) 2003 Elsevier. All rights reserved.
- 73. Observer Rules (continued) Remain silent and silence phones You may notice something so surprising that you are tempted to laugh or exclaim out loud.This is not unusual. Unfortunately, participants might think you are laughing at them. Keep as quiet as possible. You will have opportunities to ask questions at the end. If necessary, pass a note to the facilitator. Observe only - no questions or support If reviewing an interface, it’s likely participants will have problems, and it is normal to feel a temptation to help. Please don’t. Instead, try to understand why it was that the user got stuck or went down the wrong path. It’s the facilitator’s role to get users back on track if they get really stuck. And if the facilitator poses a question during the test, he or she is asking the users, not you—please don’t answer unless the facilitator specifically directs a question to you. Avoid “Design Questions” - Questions that ask the user their opinions about how to design aspects of the application (such as, “Where would you like to see these navigation buttons?”) can take a lot of time to answer and produce only limited results. Instead, focus on trying to understand the problem—we’ll come up with solutions later, outside the test. In person, be conscious of your body language Although most studies are interesting, not every moment will be fascinating. If something is happening that isn’t of interest to you but may be to others, sit quietly without fidgeting.Take notes to stay alert. Adapted from the book Paper Prototyping by Carolyn Snyder, published by Morgan Kaufmann Publishers. Copyright (c) 2003 Elsevier. All rights reserved.
Hinweis der Redaktion
- Welcome!
- Our schedule for the evening includes:
- Introduce ourselves then gauge the experience level of the audience Questions for the audience How many have 1-3 years experience? How many have 3-5 years experience? How many have never done usability testing? How many have done usability testing
- Tell story about SSA prototype of button being located at top level because of things recorded during contextual inquiry, but then discovering during testing that when they actually used the system to do something, that step was not a top level step, it only occurred IF some other event had occurred – so presenting it at the top level was confusing. It was in the wrong place.
- Let’s start with a few myths of usability testing.
- In this slide we are listing the end to end set of high level steps once a decision has been made to do usability testing, up to conducting the test. These items need a full 1-2 day tutorial unto themselves.
- Even though creating test scenarios is part of designing the usability test, the reason we have highlighted this step is because this is very similar to some of the facilitating techniques we’ll be talking about later. There is a commonality of good guidelines in interacting with users, that is applicable to both written and verbal interaction. (show example of end user prototype) (show one I did for Yahoo! Edition) I tend to work with fairly explicit task scenarios early on, or with mockups, and then, as things get more resolved, I move to very open-ended scenarios that allow users to approach the task in their own way. It’s also important to ask your users if the task is something they’d actually do – a reality check. I usually have to revise my scenarios somewhat after the initial pilot test. Bad examples: “Use the markup tool to mark up your document”. “Can you find the Notification Wizard?”.
- There are many kinds of measurements you can take and different protocols you can use. The chosen protocol will affect your interaction with test participants and your note taking and the observations you record. The kinds of measurements you take will drive some of the note taking/observations.
- There are various roles involved in usability testing. Sometimes you have the option to assign different people to these roles and sometimes one person wears all the hats. Let’s discuss the roles…….. They are all important but we are going to particularly focus on facilitation in just a little bit. We will be doing some exercises to demonstrate proper facilitation techniques.
- There are various roles involved in usability testing. Sometimes you have the option to assign different people to these roles and sometimes one person wears all the hats. Let’s discuss the roles…….. They are all important but we are going to particularly focus on facilitation in just a little bit. We will be doing some exercises to demonstrate proper facilitation techniques.
- If you use the think-aloud method or the active intervention method – you won’t take time on task measurements. If you use the retrospective method, you will be interacting with the test participant AFTER the test is done and not during.
- The most important thing we can say about recruiting is to make sure that your participants are representative of your target user group. Using a profile that is based on your target user group will help whoever recruits participants to obtain the right people. If the application is targeted towards senior citizens, testing it with 20 somethings won’t give you the right data. Consider building a pool of regular testers that you can call on (for iterative formative testing)
- Running a pilot study is the last critical step before you begin actual testing. A pilot study lets you: Double pilot for remote – 1) Technology 2) Study
- Technical issues More time – trying to understand emotions
- A script is a very important tool for ensuring a well run usability testing. It serves to keep things consistent, so you collect data consistently and don’t introduce bias. It also reminds you to welcome and thank people and to do any kind of resetting or reconfiguring necessary between sessions, like clearing the cache or history of a browser for example. The main parts of a script include: Here is a sample of script. I often point out that I will be reading them some instructions in order to make sure that I cover everything with them. Sometimes you will need to rephrase or clarify things for the participants. Having some printed instructions and having the participants read them out loud and then summarize them for me is often useful because it gets them used to talking aloud and it makes sure that they understand what they are being asked to do.
- A script is a very important tool for ensuring a well run usability testing. It serves to keep things consistent, so you collect data consistently and don’t introduce bias. It also reminds you to welcome and thank people and to do any kind of resetting or reconfiguring necessary between sessions, like clearing the cache or history of a browser for example. The main parts of a script include: Here is a sample of script. I often point out that I will be reading them some instructions in order to make sure that I cover everything with them. Sometimes you will need to rephrase or clarify things for the participants. Having some printed instructions and having the participants read them out loud and then summarize them for me is often useful because it gets them used to talking aloud and it makes sure that they understand what they are being asked to do.
- Creating test folders seems like a no brainer but honestly I’ve seen plenty of people conduct tests with no scripts and no folders and then have trouble keeping things consistent and then trying to find data and observations to help them conduct analysis, make decisions and recommendations. Test folders per participants help you organize things, make sure you don’t forget things and also promote good data analysis. They also serve as a data archive if you want to go back months or a year later to see what happened during a previous test. By including all the materials we are suggesting you have encapsulated the context.
- Hand out some samples of consent forms and briefly discuss. An ethical usability test should have the following: Minimal risk Informed consent Minimize deception Participant may withdraw at any time Participant’s data should be kept confidential Participant my withdraw data from the test.
- Before we get to the exercise let’s talk a bit more about facilitating. Remember to always use your script. When you are facilitating, keep the following things in mind: Don’t talk too much. Let them talk. This is a test, not an interview. Be careful with your body language so you don’t give confirmations or rejections. Don’t help too soon. (See getting unstuck later) If the problem is already known, tell the participant. Don’t waste their time letting them struggle. E.g., bugs. Be sufficiently curious about following up on unexpected problems, even if they don’t follow your script. Tell story of observer giving clue to user (title bar).
- Let’s drill down a little bit more into probing participants. Some excellent examples of do’s and don’ts of probes in Understanding your Users by Courage and Baxter. They list them in the context of conducting interviews but they apply equally during testing. They also reference some of the do’s and don’ts from Dumas and Redish’s usability testing book. Later during our exercises we’ll also introduce you to the very valuable and timely book titled Moderating and Facilitating by Dumas Loring – it was just published in March of 2008 and some of our exercises today will be based on that book. (do exercise with open-ended questions) What will this do? What are you trying to do right now? What are you thinking? What’s going through your mind right now? Hmmmm….tell me more about that. What does xxx mean to you? What did you expect to see? You seem frustrated – what is causing that? (jvs I think it’s better to reflect back the observation ‘I heard you grumbling there’ or ‘I notice you frowned’ or ‘I notice you leaned forward’ rather than assuming an emotion – or at least confirm first.) Never never never ask “What if…” or “Would you have…” questions. (“If this button had been named ‘Open’, would you have clicked on it?”) – the answers you get will be bogus. While the silence may be a bit uncomfortable at first, learn to keep quiet so that your participant can think. It’s very hard for them to think and also talk aloud. Sometimes, the user thinks that they are done and they are not, or, they have not done the task correctly. Or, they may indeed be done, but not know that they are – they may not have gotten enough feedback to be sure. These are both important to know.
- If you explore the problem a bit at a time, you will ferret out the root cause of the problem. Verify that the user is trying to do what you thought. Does the user share your understanding of how to go about it? Can the user find and use the specific control or function needed to proceed with the task? (If not, then perhaps you have a clue what needs to be improved, e.g., if the user was unsure what the button would do, is there hover help now? And, if not, would that help? Would a brief “guide me” be helpful? Remember: if you do have to help them complete a task, that counts as a Task Failure.
- If you explore the problem a bit at a time, you will ferret out the root cause of the problem. Verify that the user is trying to do what you thought. Does the user share your understanding of how to go about it? Can the user find and use the specific control or function needed to proceed with the task? (If not, then perhaps you have a clue what needs to be improved, e.g., if the user was unsure what the button would do, is there hover help now? And, if not, would that help? Would a brief “guide me” be helpful?
- Debriefing is the last portion of conducting a usability test. You may have some previously prepared questions and satisfaction forms to administer. You may also have some questions you formulated during the test, based on something that occurred, that you want to get more clarification on etc. If you need to measure emotional reaction, there are special tools for that, as well. (card sorting, object selection – probably should have a reference here)
- What’s wrong with these questions?
- Reporting needs are different for different kinds of tests. For formative testing, you will probably want to focus on the few most important items. For summative testing you may want to produce a full report with all the details.
- First bullet is from early 90’s I believe and comes originally from Claire Marie Karat and can be found in Mayhew and Bias’ Cost Justifying Usability. 2nd bullet is from Tom Landauer’s book of 1995/1996, The Trouble with Computers, and the last is from a 1998 Ernst & Young survey of CIO’s.
- In formative testing, most everyone has heard the 5-7 users is enough speech. But just to be clear the research that Nielsen did on this was that 5-7 users will uncover about 80% of the major usability problems IF you have representative users doing representative tasks. And that testing more users after that is a significant diminishing return so not worth testing more. This still gets debated in our field. But other researchers such as Virzi and Carol Barnum have also concluded similarly from their research, that a small number of users is sufficient. One caveat is that if you are designing for a broader range of people, you will need a broader range of participants. And if you are designing for ADA standards, you will want to include people with the different disabilities that are being addressed in the design.
- Finally we do want to mention that there are all kinds of tools out there to support usability testing that can really make things easier. None of these are necessary to do usability testing, but they can certainly make data collection and analysis much easier and less time consuming and also free up facilitators to concentrate much more on interacting with the user.. This is only a small sample of available tools and we are not endorsing any one tool over another.
- There are a variety of validated satisfaction questionnaires out there for use. Some are free, some are commercial. These are just a few examples. There are many more. I personally have used SUMI many years ago. The one I have used the most is SUS, because it’s free and only 10 questions.
- Here are some resources regarding ethics. When I worked for a university I was required to take a course similar to the one listed in the first bullet. Additionally I had to apply to the Internal Review Board – an oversight body – and describe the test in detail and get approval before conducting the usability testing. I had to do something similar while a researcher at the National Institute of Standards and Technology. We’ve given you some samples of informed consent forms but the second bullet on this slide will give you some guidelines for writing your own. And finally, it is a good idea to be familiar with the UPA Code of Conduct as a usability professional.
- Can someone tell me what an NDA is? A non-disclosure agreement, or NDA, is a legal document that prohibits participants from discussion what they saw in the usability test unless, or until, your company makes that information publically available. They are typically used on products still under development that have not yet been announced. NDA are used to protect intellectual property – either a corporation’s alone or both the corporation’s and participants designs, ideas etc. Often, companies will test out their early prototypes of a product that hasn’t be released to the market yet and they don’t want their designs and ideas talked about, released etc because of competition. Consent forms are used to protect participants – the primary directive of involving people in testing products, services etc is to do no harm. This came out of incidents where people participated in medical or physical testing activities where they either weren’t aware of the risks and/or were actually harmed as a result of their participating in some study or activity. As a result, there are now strict guidelines when humans are involved in drug trials, new treatments, testing products etc. Some people don’t think this is needed for non-invasive things like usability testing designs or software, but depending on where you work, you may be required to include consent forms. That was the case with me as both a government employee and in academia where there are strict guidelines for involving humans. And I chose to carry that over into the corporate world also because I believe strongly in the ethics behind consent forms even if they are required.