This document provides an overview of conducting user research within an Agile development process. It discusses quick and inexpensive research methods like interviews, observations, card sorting and usability testing that can be integrated into Sprints. The goal is to understand user needs and behaviors to effectively share information with the team. It emphasizes starting research by identifying primary and secondary user groups and their tasks, goals and environments. The document recommends iterative user research and testing of prototypes to help focus efforts and get to an 80% understanding of users.

1. Getting Started
Conducting User Research
Carol Smith
@carologic
Agile 2010
August 12, 2010

2. What You’ll Learn
• Quick and cheap methods, integrated into Agile.
• Understand needs and behaviors.
• Effectively share and communicate information.
2

3. DESIGN FOR EVERYONE
IS
IMPOSSIBLE
3

4. Who are you
developing for?
What do they
need to do?
4

5. Scope for Success
• You’ve heard it takes time, energy, and budget
and ____________ to:
- Research and discover
- Document
- Analyze
- Understand
5

6. And we have
constraints…
6

7. Focus Your Efforts Until…
Get to 80%
7

8. We Are 80% Sure We Know…
• primary user tasks.
• user’s goals.
• Prioritize as needed with:
- why we are doing this (vision).
- business needs
- etc.
• Have awareness of what we don’t know (yet).
8

9. Where do I start?
9

10. Who are your users?
10

11. Same Job Title, May Differ in…
Priority of tasks Characteristics
Frequency of Motivations
use and attitude
Experience
User Expectations
11

12. Begin the Research
• Interview the experts:
- Customer Service
- Marketing (Web statistics)
- Training
- Sales/Business development
12

13. Begin the Research (cont)
• Who are the users?
• How many are there?
• Common complaints?
• Show stoppers?
• Understand:
- Assumptions and stereotypes
- Differences between users
13

14. Find Out About…
• Tasks: frequency, importance, complexity
• Environment: location, physical ability, situational
limitations
• Experience: job title and subject matter, product use
• Technology: Type of computer, mobile device, Internet
connection
14

15. Define Primary & Secondary Users
• Separate by:
- Needs
- Goals (Why will they use the product?)
- Environment (Where will they use it?)
- Context (When will they use it?)
- How else do they differ?
15

16. Now You Have User
Groups
[perhaps very loosely
defined]
16

17. Model Project
1st Prototype
• Card Sort • Sprint(s)
• What we know • More research
• Profile/Personas • Sprint(s) as questions
• Tasks / Mental • What we don’t arise
Models know - Interview • Usability Test
• User Stories or Observations Prototype
• Wireframes • Usability Test
Prototype
Discovery & 2nd
Goal Setting Prototype
17

18. Share What You Learn
18

19. Personas
• Help guide decisions about:
- Navigation
- Features
- Design
• Archetype, based on research.
19

20. 20

21. Task Analysis
Example of a Task Analysis by Todd Zaki Warfel from his Agile2010 presentation
"Opening the Kimono a look behind the design process."
21

22. Mental Models…
• Help the team
- understand user’s reasons for doing things.
- design for user’s needs and behaviors.
• Help management
- Prioritize content and solutions.
- Create solutions for opportunities.

23. Mental Model
Mental Space
(goal or intent)
Behaviors, thoughts,
goals or intents of users
Content and/or Opportunity
solutions provided

24. Actionable Gap Analysis
Mental States & High Level Tasks Suggested Solutions
•Make appointment •FAQ on how
Change
•Plan for future appointments
Situation to make an appointment
•Online Request for Appointment
•Coordinate appointments online
•800 number prominent

25. Are We Confident?
25

26. 26

27. Confirm Assumptions
• Representative users who DO the tasks.
• Visionaries, leaders, perhaps.
27

28. Observations
28

29. Go to the user
29

30. Why Observe?
• Great way to understand your user’s situation
- Find “cheat sheets” and other artifacts.
- Learn real process they use.
- Number and type of interruptions.
- Find out more about them as people.
30

31. Sit Back and Watch
• Arrive when they will be doing related tasks.
• Observe for as long as needed:
- 1/2 hour each - quick repetitive tasks
- >1 hour for longer processes
• Stay out of their “space” and don’t interrupt.
• Take photos and videos.
31

32. Take Detailed Notes
• Write down questions and when they occurred.
• Look for patterns and differences:
- Style of tasks
- Order of operations
- Environment
32

33. Clarify Observations
• After observation ask about:
- Why they do task?
- What is their goal?
- How typical was this process?
- Parts of the process you found confusing.
33

34. Collect, Copy, and/or
Photograph Artifacts!
34

35. Interviews
35

36. Interviews
• Discover facts and opinions
- Structured
- Open-ended
- Combination
• Scripts
- Promote consistency
- Memory tool for facilitator
36

37. Facilitate
• Unbiased questions
• Remain passive
• Ask your question and let them talk
• Encourage participant to elaborate
• Listen for vocalizations
• Watch non-verbal gestures
37

38. Silence is Golden
Wait for it…
Wait for it…
Wait for it…
38

39. Activity: Interview
• Pick a partner and interview about common subject:
- Washing clothing (size of family, separation, etc.)
- Getting gas (type of vehicle, when get it, etc.)
- Make travel arrangements (what’s important to them)
- Getting ready in morning (order of activities)
39

40. Interviews
• How many of you started to hear a pattern?
• How many more interviews would you need to verify?
40

41. Focus Groups
• Group interviews
• Reveal perceived desires, experiences and priorities
- What and how they think
• Good moderator is key
- Group can be dominated, biased, coerced
- May hear from minority of participants
• Relatively inexpensive
41

42. Usability Testing
(Demo, Acceptance Test)
42

43. Usability Testing
• Target users perform representative tasks.
• Understand impact on users.
• Measure success.
• Types of usability tests:
• Formal - usability measures
• Informal – qualitative data
• Lab, remote, on-site (home, conference, etc.)
• Observed by stakeholders!
43

44. Test Anything
at Anytime!
44

45. Card Sorting
Gaffney, Gerry. (2000) What is Card Sorting? Usability Techniques Series, Optimal Sort, Optimal Workshop - http://www.optimalworkshop.com/
Information & Design.
http://www.infodesign.com.au/usabilityresources/design/cardsorting.asp
Demo: https://livedemo.optimalworkshop.com/optimalsort/supermarketdemo
45

46. How Do I Find
Participants?
46

47. Create a Screener
• Guide that helps determine who will participate.
• Ask people to describe, then get details:
- Highest level of education.
- Computer activities.
- Web use.
• People who pass the screener should closely match
your user group definition
47

48. Hire a Recruiter
• Allows you to focus on activity.
• Can tell if person will be a good participant.
• May already have a list they can start with.
• Good recruiters:
- find right participants.
- give regular updates.
- take care of directions, confirmations, incentives, etc.
48

49. If You Must Do it Yourself...
• Go where users go and intercept
• Online user groups
• Professional organizations
• Craigslist
• Use tools like Ethnio through your website
http://ethnio.com
• Always conduct final recruiting by phone.
• Ask questions that force them to talk.
• Don’t recruit non-talkers.
49

50. Number of Users to Test
• As many as possible (rarely statistically significant)
• Usability Testing Research (in 1990’s)
- 5 from distinct sub-group of the user population will
yield 80% of the findings (Nielsen, Virzi, Lewis)
- Assumes expert has reviewed for obvious issues
• Recommend:
- Early tests with 8 – 12 users per user group
- Iterative testing (3 per day, iterate, 3 new users)
Barnum, Carol M. (Jan. 2003). What’s in a Number? STC Usability SIG Newsletter, Usability Interface.
http://www.stcsig.org/usability/newsletter/0301-number.html Retrieved: 20080323
50

51. 51

52. Welcome & Prepare
• Participation will help team and is appreciated.
• Purpose of research.
• Expectations of the participant.
• Sign paperwork:
- Non-Disclosure Agreement(s)
- Consent Form
52

53. Your Goal
53

54. We’re Looking for Patterns
• Identify repetition
• After pattern is found, continuation of study
- Adds cost
- Delays reporting
- Low probability of many new findings
54

55. Update Communications
55

56. Model Project
4th
• Sprint(s) Prototype • Sprint(s)
• Continue • Prepare for next
Research • Sprint(s) stage/project
• Usability Test • Prepare for next • Usability Test
Prototype stage/project Prototype
• Usability Test
Prototype
3rd 5th
Prototype Prototype
56

57. Do UX Early & Often
57

58. In Closing
• Get to your users
• Find out:
- Goals
- Tasks
• Share the information with your team
58

59. Recommended Readings
• www.UsabilityBoK.org
59

60. References
• Cato, John. User-Centered Web Design. Addison Wesley Longman; 2001.
• Gaffney, Gerry. (2000) What is Card Sorting? Usability Techniques Series,
Information & Design.
http://www.infodesign.com.au/usabilityresources/design/cardsorting.asp
• Hackos, JoAnn T., PhD and Redish, Janice C. User and Task Analysis for
Interface Design. Wiley; 1998.
• Henry, S.L. and Martinson, M. Evaluating for Accessibility, Usability Testing in
Diverse Situations. Tutorial, 2003 UPA Conference. (Activity)
• Kuniavsky, Mike. Observing the User Experience: a Practitioner's Guide to User
Research. Morgan Kaufmann, 2003.
• Mandel, Theo. The Elements of User Interface Design. Wiley; 1997.
• Nielsen, Jakob and Robert L. Mack. Usability Inspection Methods. John Wiley
& Sons, Inc. 1994.
• Powell, Thomas A. The Complete Reference: Web Design. Osborne/McGraw-
Hill; 2000.
• Rubin, Jeffrey. Handbook of Usability Testing. John Wiley & Sons, Inc.; 1994.
60

61. Thank You!
• Carol Smith
• Midwest Research, LLC
• http://www.mw-research.com
• Twitter: @carologic
• Blog: http://www.AskaUser.com
• Email: carol@mw-research.com
61