Interviewing is undeniably one of the most valuable and commonly used user research tools. Yet it's often not used well, because * It’s based on skills we think we have (talking or even listening) * It's not taught or reflected on, and * People tend to "wing it" rather than develop their skills. Results may be inaccurate or reveal nothing new, suggesting the wrong design or business responses, or they may miss the crucial nuance that points to innovative breakthrough opportunities. In this day-long session, we'll focus on the importance of rapport-building and listening and look at techniques for both. We will review different types of questions, and why you need to have a range of question types. This session will explore other contextual research methods that can be built on top of interviewing in a seamless way. We'll also suggest practice exercises for improving your own interviewing skills and how to engage others in your organization successfully in the interviewing experience.

2. Brought to you by

3. Introduction

4. Today Introduction 9:00 – 9:10 Best Practices Overview 9:10 – 10:00 Methods10:00 – 10:30 Break10:30 – 10:45 Interviewing 10:45 – 11:30 Interviewing Exercise11:30 – 12:15 Lunch! Observation Exercise 1:15 – 2:00 Synthesis 2:00 – 2:45 Break2:45 – 3:00 Ideation3:00 – 4:00 Share4:00 – 4:30 Q&A + Wrap-up 4:30 – 5:00

5. Take a fresh look at people Use existing ideas as hypotheses What to make or do Refine & prototype Launch Iterate & improve Use fieldwork throughout the development cycle Explore new ideas

6. Synthesis & ideation process Fieldwork Synthesis Ideation Development

7. Cultural data from fieldwork Case study: iPod accessories

8. Case study: iPod accessories Portigal Interviewing Users: Spinning Data into Gold 8

9. Research Project – Best Practices Overview

10. What do I mean by “research?” Ethnography Ethnographic interviews Video ethnography Depth-interviews Contextual research Home visits Site visits Experience modeling Design research User research User-centered design One-on-ones Camera studies User safaris

11. What do I mean by “research?” Ethnography Ethnographic interviews Video ethnography Depth-interviews Contextual research Home visits Site visits Experience modeling Design research User research User-centered design One-on-ones Camera studies User safaris What-ever!

16. The Business Question What new products and services can you offer to help partners increase social network stickiness (and thus revenues)? What entertainment activities should you support to tap into a growing middle-class in China?

17. The Research Question What are the motivations, successes, and frustrations for current and prospective users of our partners’ social media sites? How is family life changing in middle-class China? What are the critical digital and analog technologies that are being embraced?

18. Pain points: default research/business question?! While we always uncover so-called pain points, the bigger opportunity may come from understanding why– how did we get here?

19. It may not really be that painful Satisficing(coined by Herbert Simon in 1956) refers to our acceptance of good-enough solutions These can drive engineers and designers crazy…but the real problem isn’t always what it appears to be

25. Non-user

26. Extreme user

27. Peripheral user

28. Expert user

29. Subject-matter expert

30. Wannabe user

31. Should-be user

32. Future user

33. Past user

34. Hater

35. Loyal to competitorTriangulate through multiple perspectives By creating contrast, you reveal key influencing factorsthat you wouldn’t otherwise see

37. Do they even exist?Surface a broader sense – even prior to research – about who is affected by the product and who is being designed for Is your “typical customer” real or aspirational?

39. Stage of family

40. Retirement

46. Checking off criteria

49. Snowball recruiting (participants find more participants)

50. Craigslist

51. Intercepts

53. Quick but harder to control and manage

55. For a fee, they will handle sending a check, but I prefer the immediate gratification of an envelope of cash

58. Minimalist field guide

59. Detailed field guide

60. Include other methods

61. Documentation: photos Plan to take lots of photos They will reveal things you don’t remember noticing Essential for storytelling Make sure you have permission before you start snapping

63. Incentive – what the participant gets but they are not an employee

64. Model release – how images and video will be used

66. Can you influence them to create consumer-friendly, light-weight versions

68. Ask people how they would solve a problem Participatory design Doesn’t mean we implement the requested solution literally “I wish it had a handle” Many ways to solve the underlying need (“I need to move it around”) Designers work with this data to generate alternatives Engage people in the non-literal through games and role-playing Uncover underlying principles and explore areas of opportunity that don’t yet exist

69. Show people a solution Consider the difference betweentestingandexploring Avoid “Do you like this?” Don’t show your best guess at a solution; instead identify provocative examples to surface hidden desires and expectations Make sure you are asking the right questions What does this solution enable? What problems does it solve? Especially for new products, needed before getting into specifics of your implementation Image from Roberto and Worth1000.com

70. Use a range of methods

71. Workbook: Instructions

72. Workbook: Question and answer

73. Workbook: Text stimuli

74. Workbook: Visual stimuli

75. Workbook: Responses

76. Mapping

77. Storyboards

78. Mockups

79. Prototypes

80. Casual Card Sort

82. Break!

83. Interviewing

84. Principles inform tactics I don't skate to where the puck is, I skate to where the puck is going to be – Wayne Gretzky

85. Fieldwork principles Check your worldview at the door Embrace how other people see the world Build rapport Listen

88. Leave plenty of time so you aren’t rushed when you arrive

94. “Earlier, you told us that…”

98. Use natural language Talk like your subject talks!

99. Don’t make questions pass/fail (1/2) Client: So the concept of transferring, burning, and syncing, can you talk a little about those three concepts? So transfer, burn, and sync. Just do you understand the difference? Interviewee: Transfer, burn, sync. Burn is when I’m actually putting it onto some kind of disc. Okay… Transferring is I guess when I transfer the files from one place to the other. Mm-hmm… Whether it’s to a device or to a different drive or whatever or into the program I guess. And syncing, well, I know the phone always comes up and says it’s syncing, when it’s syncing up to the files or syncing up to the computer or stuff like that. That’s the only time I think I’ve ever really heard that. (cont’d)

100. Don’t make questions pass/fail (2/2) Client: So the only other question I have left in this area is: Would you expect to manually decide what music goes on your devices or would you rather that the machine does it for you? Interviewee: Decide what I want on my…? Let’s say your library is here on this machine, and you have a device, would you want it to put as much as it could put on from the library from the device when it’s connected to your computer? If I could hold it… if the device itself could actually hold all the files, I would love that, if it automatically… Just knew. Just knew that it wasn’t on there, the same thing when…what do you call it - when I have to go into the program and actually have it… downl…uhh, now I’m confused in what I should call it. No, no, don’t worry about it!

101. Don’t presume they accept your world view Client: So, really interesting the sort of things that you as old Derek used to value, such as efficiency of time, and some of those things have now influenced the new Derek. Derek: Right. Steve: Maybe that sort of begs a larger question…We’ve offered you this idea of old versus new you, but how do you think about this transition? Derek:Yeah, I don’t really see it.

103. Find your personal style The Myers-Briggs Type Indicator

104. Questions to gather context and collect details

105. Questions to probe on what’s unsaid

106. Questions to probe on what’s unsaid

107. Questions to uncover mental models

108. Why so many types of questions? Real interviews aren’t as simple as asking a question, getting an answer, and then moving onto the next question in your list. You are unlikely to get to the actual answer without asking a few different questions a few different ways. You need a range of tools and techniques. And you need to feel when you haven’t got to the real answer yet so you can keep going .

109. Prepare for exploding questions Well, my cousin never tells me when she has an updated bank balance so I figured I would handle it myself. That’s why I signed up for the PayPal service, I think it’s them but maybe not. Coping techniques Wait until these issues come up organically, without you having to ask Make notes on your field guide about what you want to loop back to so you don’t forget Triage based on what’s most pressing for your topic Triage based on what makes the best follow-up, to demonstrate listening Why does this matter? Let’s find out what service this is?! Okay. I decided I had to spend the money I had from last month in order to save month’s money and this service was going to help me do that. Even if it’s not the same password that my cousin would be using I don’t understand her financial model… Why does she expect that it would be the same?

110. Exploding questions can lead to a flow state

112. Getting better at interviewing Practice with hallway or other serendipitous micro-interviews Write surveys and participant screeners to practice crafting questions out of the moment How do you get to Carnegie Hall? Practice, man, practice!

113. Participant screeners as asking practice A good way to practice both framing a question and the empathic exercise of thinking through the respondent’s user experience with that question

114. Write and take surveys Develop your own critical eye (and interviewer’s voice) by looking for bad examples and identifying just what’s wrong with them

115. We learn from mistakes and mishaps Collect and share war stories with other interviewers THANX 4 NOTHING KITTEH

118. Each person plays each role once

119. Stay in the exercise!Group debrief

120. Giveaway #1 Everyone gets a license for TechSmith’s screen capture tool Snagit Expect it by email in a couple of weeks Everyone gets 3 tests from UserTesting.com Use code uxworkshop; expires in 2 weeks Draw 2 licenses for TechSmith’sCamtasia (Mac/PC) 1 license for Techsmith’sMorae (PC)

121. Lunch

122. (The rest of) today Introduction 9:00 – 9:10 Best Practices Overview 9:10 – 10:00 Methods10:00 – 10:30 Break10:30 – 10:45 Interviewing 10:45 – 11:30 Interviewing Exercise11:30 – 12:15 Lunch! Observation Exercise 1:15 – 2:00 Synthesis 2:00 – 2:45 Break2:45 – 3:00 Ideation3:00 – 4:00 Share4:00 – 4:30 Q&A + Wrap-up 4:30 – 5:00

123. Homework/Observation Exercise

124. Homework Check-in Your mission: Dedicate at least half an hour to walking around and observing people in your neighborhood Props to Dylan, Caroline, and David!

125. Homework Check-in Who was able to do the assignment? Was this anyone’s first experience doing observational fieldwork? Is there anyone who has not done user or observational research in the field?

126. Observing Notice what… people, places Notice how… processes, sequences, interactions Suspend your point of view Avoid conclusions Allow confusion Do it “out loud” Steve, practicing his “noticing.” You can tell because he looks like he may be a little confused.

127. You’re observing people within their culture. Notice how cultural artifacts reflect and define the environment; and reveal what is “normal” Normal isn’t “right or wrong” – it’s the set of background rules that define much of what people choose or ignore Media Products Advertisements Street Culture Trends/Fads Cultural context What are they selling?

128. Cultural context

129. Cultural context

130. Cultural context

131. Your mission: Imagine you are working on a project for Gentrific8, looking for ideas to redevelop parts Seattle around downtown and the Central Library. Form groups of 2 – 3. Mix it up Wander and observe people, interactions and environments Do it out loud! Capture (photos, notes) What, who, where, when? Why, how? This is not a design audit of signage or merchandise displays Exercise: Explore!

132. Neighborhood observations: Noe Valley, San Fran

133. Neighborhood observations: Noe Valley, San Fran

134. Neighborhood observations: Montara, California

135. Be back by 2:25! Exercise: Explore!

136. Synthesis: From data to insights

137. Analysis Synthesis Ideation Solutions Strategies Opportunities Detailed solutions Insights Synthesis & ideation process

138. Fieldwork Synthesis Ideation Development Why a process?

139. Avoid jumping to conclusions

140. DTDT: Analysis vs. Synthesis Analysis Break large piece(s) into smaller ones in order to make sense e.g., interviews, transcripts into anecdotes, stories

141. DTDT: Analysis vs. Synthesis Combining multiple pieces into something new e.g., developing themes, implications, opportunities Synthesis Analysis Break large piece(s) into smaller ones in order to make sense e.g., interviews, transcripts into anecdotes, stories

142. DTDT: Analysis vs. Synthesis The process gradually moves from one to the other Combining multiple pieces into something new e.g., developing themes, implications, opportunities Synthesis Analysis Break large piece(s) into smaller ones in order to make sense e.g., interviews, transcripts into anecdotes, stories

143. Analthesis???? Combining multiple pieces into something new e.g., developing themes, implications, opportunities Synthesis Analysis Break large piece(s) into smaller ones in order to make sense e.g., interviews, transcripts into anecdotes, stories

144. Sense-makingthrough an iterativeprocess of refining gathered data Early, Informaldata in your head First, process the experience you had collecting data Refer to debriefs and conversations Articulate and identify themes Outcome: Topline Report Process-based, Formalheavy lifting Then, process the data itself Individual and group analysis Pattern-identification, clustering, models, frameworks Outcome: Opportunities More narratively, what is synthesis? Review, Refine, Rinse, Repeat

145. Synthesis naturally begins inthe field Resist meaning (for now) Focus on observations Get the detail Create time to talk after each fieldwork experience Worksheet to facilitate the debrief Write up real-timesummaries for the team, ASAP In-field debriefing Fieldwork highlights captured in the wild.

146. After fieldwork, collate reflections and quickly externalize a starter set of 5 to 10 thematic areas based on Pre-identified areas of inquiry Refer to debriefs and conversations from the field New patterns that we observed Identify interesting areas; acknowledge that you don’t understand details yet, identify questions Outcome: Topline Report All right researchers… what did you see? Early, informal synthesis (data in your head)

147. This sheds light on what excites the team and the stakeholders and brings focus to the next stage of synthesis The Topline Report

148. Go back through your raw data very closely to move beyond the Topline Report Individually (heads-down) and collaboratively (heads-up) develop clusters, identify patterns, collate and refine findings Process maps, eco-systems Frameworks, models Design implications i.e.: What did other public announcements in the study look like? What are the layers of information and cultural context? What form factors are favored? Why? Process-based, formal synthesis (heavy lifting)

150. Photos, field artifacts

151. Transcripts

152. DV->MP3->FTP->.docx

154. Ask yourself questions; give labels; propose solutions

155. Don’t worry about implications, be descriptive and reactiveIndividual analysis (not today…)

157. Heads up! Present each interview (etc.) as a case study. Introduce each, and pick out the provocative highlights. Voice and document reactions, a-has, support and questions Clustering with stickies White-board notes Develop a new shared point-of-view, beyond “findings” Collaborative analysis

158. Easy to scan for patterns and relationships Play with data by rearranging individual elements Lo-fi way to makes data tangible, visible, and sharable Sticky work

159. As you are telling stories, quickly get the key points (notes, themes, observations, quotes) up Code with the source (interview name, etc.) Separate what was observed from what you think it means Write big and try to code visually (e.g. colored dots, colored post-its, symbols) Sticky work

160. Group stuff Be opportunistic, using whatever makes sense at first You may want to re-use your topline headings or you may want to be fresh Initial groupings may be “All things related to shopping” or “what people are doing” or “what people are feeling” or “pain points” Sticky work

161. Sticky work Re-group stuff Now, go back and re-group at a higher level What it means What people are trying to accomplish (i.e., needs/motivations/goals)

163. Spreadsheet analysis enables immersive refinement of data Play with data by generating alternate views of the dataset Rewrite each sticky in a cell (adding commentary, explanation, context, quote) Processing each entry allows further synthesis and thought More individual analysis Categories of columns will vary by project Comment comes from rewritten sticky Tag each comment with person, segment, market etc. to allow you to manipulate data

164. Uncover patterns through additional layers of keywords Prioritize and better understand themes and relationships Search for quotes and evidence as you transition to writing presentation Go back to your data in search of more insights or further inspiration More individual analysis Coding entries with keywords builds a taxonomy for future reference Spreadsheet tools let you see data in different ways Use themes from sticky work and/or presentation sections to reveal relationships and shore up shaky thinking

165. Opportunities are not A reporting of “interesting findings” A list of solutions Opportunities are Change we can envision based on what we heard and observed About people In the context of, but reframing the business questions Generative, inviting many solutions Keep the human touch in communication Allow people to move seamlessly between places Allow people to integrate seamlessly across different devices and systems Developing opportunities What should we do?

166. Topline Summary of analthesis activities Opportunities Collaborative Analysis Individual Analysis Individual Analysis Keep the human touch in communication Allow people to move seamlessly between places Allow people to integrate seamlessly across different devices and systems Externalize the data in your head The heavy lifting Determine generative directions Play with the entire data set Play with individual data elements

167. Get in groups of 4 Quickly review what happened (today and from your homework) and what you saw. Collate reflections. Resist the urge to move too far towards conclusions Don’t refer to notes or photos yet Keep your own experiences, existing hypotheses, cultural clichés, etc. in the background Develop 3 - 5 themes as a “Topline Report” sketching out the big takeaways, leading into further synthesis Don’t fuss over exact wording Exercise: Develop a topline (7 minutes) All right researchers… what did you see?

168. Evolve your “Topline Report;” flesh out and enrich themes Write your themes and put them up Leave space for new ones Now (!) tell stories from the field (from your neighborhood and today), using photos, notes and memory Rethink the relationships between the themes, pick your strongest themes and write a sentence with a point of view Go from “Graffiti everywhere” and “Teen gangs hanging out” to “Public spaces in the neighborhood are used to communicate identity and belongingness” Exercise: Develop findings (7 minutes)

169. Build on your findings Start each opportunity with a verb Opportunities are not A reporting of “interesting findings” A list of solutions Opportunities are Change we can envision based on what we heard and observe About people In the context of but reframing the business questions Generative, inviting many solutions Exercise: Identify opportunities (7 minutes) Keep the human touch in communication Allow people to move seamlessly between places Allow people to integrate seamlessly across different devices and systems Keep the human touch in communication Allow people to move seamlessly between places Allow people to integrate seamlessly across different devices and systems What should we do? What should we do?

170. Break!

171. Ideation: From insights to solutions

172. A simple step moves you from Opportunities to Ideation Questions, reframing them into actionable language How can we keep the human touch in communication allow people to move seamlessly between places allow people to integrate seamlessly across different devices and systems How can we How can we Ideate! Ideation questions

173. Scope of solutions Solutions exist across many different business areas Functionality Visual design Marketing Architecture Public Services Partnerships Events Software Form factor Packaging Policy Retail design Even if you are unlikely to impact certain business areas, it’s crucial that you set that constraint aside for ideation How many business and civic areas to impact can you spot in this picture?

174. Developing strategies Responses to any ideation question can lead in different strategic directions Finding:Students have to smoke outside, but they get cold and wet Opportunity: Improve the experience of students who smoke Ideation Question:How can we improve the experience of students who smoke? Support underlying needs and behavior by embracing the finding Question needs and behavior, seek change by challenging the finding Create a protected environment for smoking Eliminate smoking

175. Strategies can inspire solutions Finding:Students have to smoke outside, but they get cold and wet Opportunity: Improve the experience of students who smoke Ideation Question:How can we improve the experience of students who smoke? Strategies Create a protected environment for smoking Eliminate smoking Solutions FacilitiesBuild a pavilion AdminAllocate interior room PartnersAlign with nearby cafe OnlineSmoking cessation games AdminBan smoking PartnersStop smoking coaches

176. Solutions can suggest strategies Finding:Students have to smoke outside, but they get cold and wet Opportunity: Improve the experience of students who smoke Ideation Question:How can we improve the experience of students who smoke? Strategies Create a protected environment for smoking Eliminate smoking Solutions AdminAllocate interior room AdminBan smoking

177. Collaborative generation This is a collective, out-loud activity! Talk, listen, build on each other’s ideas Don’t worry about a “bad” idea… it may lead to a “good” idea Don’t correct; generate alternatives “Yes, and…” This is a visual activity! Sketch, draw… Quantity over quality; go quickly Individual ideas matter less than what the collective produces overall How can a sour lemon help keep things working smoothly?

178. Stuck? Come up with bad ideas Immoral Dangerous Bad for business

179. Don’t forget your second wind Pace of idea generation Obvious but necessary, problem-solving, need-filling, low-hanging fruit Time

180. Don’t forget your second wind Pace of idea generation Obvious but necessary, problem-solving, need-filling, low-hanging fruit Wacky, transgressive, innovative, breakthrough, weird Time

181. Summary of ideation exercises Questions Business Areas Ideation and Sharing 2 minutes 3 minutes 40 minutes 30 minutes How can we keep the human touch in communication allow people to move seamlessly between places allow people to integrate seamlessly across different devices and systems How can we How can we Ideate! Shift to “How can we…?” Figure out where we can play Remember, “Yes, and…”

182. Exercise: Ideation questions (2 minutes) Apply How can we…? to each of your Opportunities How can we keep the human touch in communication allow people to move seamlessly between places allow people to integrate seamlessly across different devices and systems How can we How can we Ideate!

183. Exercise: Business areas (3 minutes) Let’s collectively list possible business areas to design for Think about whatever Gentrific8 could do or affect Use this list as a starting point Functionality Visual design Marketing Architecture Public Services Partnerships Events Software Form factor Packaging Policy Retail design Incentives How many business and civic areas to impact can you spot in this picture?

184. Exercise: Ideation (40 minutes) Use your ideation questions to generate strategies and solutions Out loud Visual Collaborative Consider the range of possible business areas Bounce back and forth between generating strategies and solutions Most ideas will not turn out to be winners; the goal is to practice connecting research data to solutions Apply lemon as needed. Don’t forget your second wind

185. Exercise: Prepare to share (2 minutes) Rapidly align on your team’s best ideas and message Choose a messenger The wise team will choose a bold, expressive spokesperson

186. Exercise: Pitch it back!

187. Prioritization

188. Big group voting

189. Small group ranking…and reconciliation Ranking factors may even include how clear the idea is Color indicates voting winner

190. Wrap Up

191. How experts use frameworks Fieldwork Synthesis Ideation Development

192. 2-3 weeks 2-3 weeks 2-3 weeks Who do you want to talk to? What do you want to do with them? Do something with the data! Typical timelines Fieldwork Screening criteria, recruiting Methodology, field guide, stimuli Analysis, synthesis, design Interviews, self-reporting, debriefs When working in tighter timeframes, consider where you want to cut back. Be mindful of the tradeoffs!

193. Going rogue 1 day?! 1 day?! 2 days?!! Who do you want to talk to? What do you want to do with them? Do something with the data! Fieldwork Who can you get? Co-workers, intercepts on the street or in the mall, etc. Wide-eyed observation, winging it Debrief Small sample, massively parallel data gathering

194. Coming in 2012! A book by Steve Portigal The Art and Craft of User Research Interviewing http://rosenfeldmedia.com/books/user-interviews/

195. I’ve got a tip (that you didn’t cover) that works well for me… Yeah, I’ve got a question for ya… One new thing I learned today is…

196. Brought to you by

197. Giveaway #2 Draw 8 Rosenfeld Media titles Steve Krug’s Rocket Surgery Made Easy UserTesting t-shirt(s) etc.

198. Thank you! Portigal Consulting www.portigal.com @steveportigal steve@portigal.com 415-894-2001