Hearing that focus groups are a dead "yesterday" methodology? Wondering if & when they could be relevant to your company? This presentation outlines what they are, current trends, best practices for when to use and when to avoid focus groups, and the dreadfully-boring but oh-so-important topic of quality recruiting for qualitative research
Call Girls In DLf Gurgaon ➥99902@11544 ( Best price)100% Genuine Escort In 24...
BellaVia Research: Mozilla: When to use focus groups
1. To Focus Group,
or Not To Focus Group:
This is the Question
Julie Francis / BellaVia Research
July 2012 / Presentation to Mozilla
1
2. Who Am I? Julie Francis
Founder and principal of BellaVia Research
20 years experience
Market Research Background & Usability/UX since 2000
• Creative Good (NYC & SF): online UX strategy consultant
• Before that: Socratic Technologies, Audits & Surveys Worldwide
Range of work:
• Usability & User Experience
• Mobile / online / multi-channel
• Contextual inquiry / “ethnography-lite”
• Persona creation
“Agnostic” when it comes to methodology
• What is the best way to answer the research question?
• What methodology best fits budget, need for observation, desired geographic reach, timing, etc.
• Huge range of methodologies / approaches
Arizona Public Service Diageo Sony
Arhaus.com Groupon.com Southern California Edison
Art Institute of Chicago Interbrand StubHub.com
ArtThatFits.com Motorola thebar.com
Bank of America OPOWER Walmart.com
Busch Gardens Planned Parenthood Wetpaint.com
BellaVia Research • User Experience & Market Research • Santa Cruz CA • 831.454.8217 • info@bellaviaresearch.com 2
3. How we’ll spend our time today…
60 minutes
½ me talking
• Level setting – What the heck is a focus group?
• When to use focus groups
• When not to use focus groups
• About recruiting….
½ us talking together
• What‟s on your mind?
BellaVia Research • User Experience & Market Research • Santa Cruz CA • 831.454.8217 • info@bellaviaresearch.com 3
4. Research Methodologies: Decisions, Decisions
We’re inundated with methodology options…
Qualitative Research approaches:
In person
• Focus groups
• IDIs / one-on-one interviews
• Mini-groups / Dyads / Triads
• Contextual? In-lab?
Online/Remote
• Bulletin boards (open or closed)
• Diary studies, including mobile phone / text diary studies
• Remote moderated phone/online
• Social media “listening”
And more…
Hybrid methodologies
• Infinite combinations of the above
• Hybrid is the current trend –
No methodology is perfect.
All have strengths & weaknesses…
How to choose?
• a hybrid approach lets us
capitalize on upside of various approaches
BellaVia Research • User Experience & Market Research • Santa Cruz CA • 831.454.8217 • info@bellaviaresearch.com 4
5. Baseline: What’s a Focus Group?
Focused Group Discussion
Moderated (trained human being)
1.5 – 2 hours (usually)
6-10 people (usually)
• Historically groups were larger (10 – 12)
• Current trend: Smaller groups (6-8 )
- Why? Deeper inquiry (the whole point of qual)
• Also Dyads & Triads (go really deep, but still get the benefit
of dialogue you miss in a one-on-one interview…)
(M&Ms are the official food of focus groups)
BellaVia Research • User Experience & Market Research • Santa Cruz CA • 831.454.8217 • info@bellaviaresearch.com Photo credit Willie Lunchmeat 5
6. Baseline: What’s a Focus Group?
Pre-recruited participants
• Professional recruiters (facility or independent)
• Client or facility-lists
• Strangers (usually)
• As with any kind of research, good recruiting is a must
(weed out “cheaters & repeaters”)
(be careful with Craig‟s List & Friends/Family recruiting)
• Recruited at least in part by phone (online surveys & email
blasts may also be involved) (definitely use the phone –
casual conversation helps ensure the person is a good fit
and helps identify cheaters)
• Over-recruit in anticipation of no-shows
Paid to participate (almost always)
• Lures „em in, & thanks people for their time
• Helps with “show rate”
• Addresses participation bias Tangent: for bulletin board and other online/written research,
(We want busy people not just broke people) use a written open-ended screening question for articulation.
• $75 & up Can the person express themselves well in writing with
minimal prodding?
• Better incentive better show rate
BellaVia Research • User Experience & Market Research • Santa Cruz CA • 831.454.8217 • info@bellaviaresearch.com 6
7. Baseline: Logistics – Lab with Conference Table
Most groups are held in research facilities using a conference table setup & a one-way mirror…
Got lots of written exercises or
paper to juggle?
Use a conference table setup…
watchLAB
One way mirror in facility
lots of real time observation
(get the most bang for your buck by inviting team
to watch real-time)
Budget conscious? Research doesn‟t need to
be “blind”? Use your in-house lab.
In rural markets we sometimes set up in hotels. Amplify, Pleasanton
BellaVia Research • User Experience & Market Research • Santa Cruz CA • 831.454.8217 • info@bellaviaresearch.com 7
8. Baseline: Logistics – Lab with Living Room Setup
There‟s a growing trend towards Living Room Setups at research facilities
watchLAB
A living room setup can feel more comfortable &
less foreign to people who aren‟t used to sitting
around conference tables.
GREAT for body language
(much of which is hidden by conference table)
watchLAB, San Francisco
Not good if there are lots of papers to juggle
(unless a clipboard will suffice)
BellaVia Research • User Experience & Market Research • Santa Cruz CA • 831.454.8217 • info@bellaviaresearch.com 8
9. Baseline: Logistics – in Context
A lot of research (including focus groups) takes place in-
context, in the “real world”
• Street intercept -- a great way to find “fresh” participants
(beware of intercept bias)
• In-homes / Contextual – always good to see people in context –
Friend Groups, Mom‟s groups
• But… you lose the ability for your team to observe (one of the
greatest benefits of research)
TecEd, Inc. Ethnographic Field Study
Susan Hosking & Jon Littel
(photo from MomsSupport.org)
BellaVia Research • User Experience & Market Research • Santa Cruz CA • 831.454.8217 • info@bellaviaresearch.com 9
10. Overheard Recently…
“I don‟t believe in focus groups”
(followed by)
“I‟m thinking about getting
together a group of people
to talk about this topic.”
(Um…call it what you will…
but that’s what I call a focus group)
BellaVia Research • User Experience & Market Research • Santa Cruz CA • 831.454.8217 • info@bellaviaresearch.com 10
11. Focus Groups are getting a bad rap
“Focus groups are
dead. The paradigm
has shifted.” “it‟s observational
vs. directed market
research”
“awkward tactic”
Do they deserve this bad rap? IMHO,
most “problem groups” stem from
unrealistic expectations, improper use,
DIY moderators who lack training/
experience
Clients tell researchers “I want to do some
focus groups” when what they mean is “I
want to understand my target” (and focus
groups aren’t always the right fit)
BellaVia Research • User Experience & Market Research • Santa Cruz CA • 831.454.8217 • info@bellaviaresearch.com 11
12. Let’s not bury focus groups
All approaches can work well
when used appropriately
There is a role for focus groups
It can be incredibly insightful to hear
a group of people banter about your
product/service/category
See Francesca Johnson, Research Arts:
http://www.researcharts.com/2011/01/in-praise-of-focus-groups-and-moderators/
BellaVia Research • User Experience & Market Research • Santa Cruz CA • 831.454.8217 • info@bellaviaresearch.com 12
13. When to use focus groups?
IMHO, Focus Groups are best when you want/need to:
Explore needs / Explore
thoughts / feelings Explore brand consumer
perceptions language / issues
(fuel for innovation, not (yours, competitors)
iteration) (to inform/explain survey
research)
Understand Uncover what Explore use
decision process you don‟t cases
/ factors know… (why, where, when, &
how…)
BellaVia Research • User Experience & Market Research • Santa Cruz CA • 831.454.8217 • info@bellaviaresearch.com 13
14. Focus Groups are not my first choice for…
IMHO, Focus Groups are not the best fit for…
Usability
Advertising impact /
(product / website / Design iteration
communication
mobile – don’t do it)
Understanding Situations where
what people really you worry about
do (instead of what people influencing
they say they do) each other
Sometimes we use groups anyway. Why?
Efficiency / Speed
(You can interview 24 people in groups much faster than you can in 1:1)
Budget (if it’s faster, it’s often cheaper too)
Client comfort (Clients ask for focus groups because it’s what they are used to)
BellaVia Research • User Experience & Market Research • Santa Cruz CA • 831.454.8217 • info@bellaviaresearch.com 14
15. Focus Groups are Best When…
They are open & exploratory
• You don’t know what you don’t know
• Use projective techniques to get below the surface (don‟t knock it until you‟ve tried it!)
Banter happens!
• You use the group dynamic to its full advantage! (and use a trained moderator, to help
manage the chaos)
• Let people interact, banter, build on each other‟s ideas, trigger each other with
experiences/questions, etc. That‟s where some of the richest insights come from…
The guide isn‟t too packed
• Allow conversation to happen! Give people time to talk, banter, build, interact – let the
magic unfold
• Litmus test: Aim to wonder “Uh oh, is this enough to fill 90 minutes?”
• (FYI, we’re seeing a trend towards clients packing guides…which diminishes our ability to
get good quality insights.)
There is no counting
• It‟s not quant. It‟s qual. It‟s all about what drives opinion (the WHY), & the range of
opinion/experience. It‟s not definitive. It‟s often followed by quant research, for good
reason.
BellaVia Research • User Experience & Market Research • Santa Cruz CA • 831.454.8217 • info@bellaviaresearch.com 15
16. What do I mean by projective technique?
One example: picture selection
• i.e., “Which of these images best describes how
you feel about _______________”? How and why
does that fit? (then probe, probe, and probe some
more)
• This seemingly bizarre approach is brilliant for
cutting through all the superficial, high level
words down to the core issue. Trust me, it‟s
brilliant.
• Each pictures should have multiple interpretations.
Avoid literal images
• istockphoto is great for abstract searches (but the
pictures can be really expensive, and it drives me
CRAZY that their photo credits expire.
• Not just for focus groups; Works great in 1:1
interviews, too. Also, works both online & in-person.
I‟ve used them in all sorts of methodologies with great
success.
There are lots more projective techniques, but
this is one of the most common and least
terrifying to use.
BellaVia Research • User Experience & Market Research • Santa Cruz CA • 831.454.8217 • info@bellaviaresearch.com 16
17. How to Choose a Methodology?
Always start with your objectives: What are you trying to learn?
Corporate See behavior Consider
User Tactical
Usability Identity (logo, in action
Experience Feedback
name) (observational)
IDI (1:1)
Explore drivers /
To help you
Understand Consider
When you don‟t barriers / behavior/decision
Fuel for innovation know what you perceptions /
prepare for quant
drivers, longitudinal Focus
research (identify
don‟t know attitudes / brand behaviors Groups
issues & language)
perceptions (reflective)
BellaVia Research • User Experience & Market Research • Santa Cruz CA • 831.454.8217 • info@bellaviaresearch.com 17
18. More Methodology Considerations
How important is team observation?
• In-lab can be more efficient (complete more sessions per day)
• You think going to a focus group is weird? Having 2 researchers & a videographer come into your home is even weirder!
Observation Less important – Observation More important –
do your research in-context do your research in a lab
• If you don‟t have observers, you can go in- • Labs are great for team observation. When else do
context! In-homes, street you get to spend a full day as a group in the presence
of your customers?
• Use a videographer
• Cram up to 20 observers in the back room. Great
• Consider a highlights reel (people have bang for your research buck. (facilitate the
good intentions to watch videos, but they observers!)
don‟t) (mic your participant)
Best practice:
Hybrid! (capitalize on the upside / strength of each approach)
Other considerations: Timing / budget / need for geographic reach
BellaVia Research • User Experience & Market Research • Santa Cruz CA • 831.454.8217 • info@bellaviaresearch.com 18
19. Focus Group Moderating Toolbox
What‟s in my toolbox?
Moderating to the Max: A full-tilt guide to Dominators, Cynics & Wallflowers:
creative, insightful focus groups and depth Practical Strategies for Moderating
interviews (Bystedt, Lynn and Potts) Meaningful Focus Groups (Kahle)
• My favorite reference book on “how to” for projective • Moderating groups is really, really hard.
techniques • This book is a lifesaver for dealing with group
dynamics.
Secrets of a Master Moderator Riva Training Institute: Hands-on Training
(Henderson) • Moderate in front of masters & your peers, get
• Naomi Henderson is a brilliant moderator, & the critiqued (rinse, repeat)
founder of the RIVA Training Institute • Rockville, MD (unless you organize a custom course)
• Fundamentals of Moderating course: 3 days:
$3600, worth every penny (really)
QRCA (Qualitative Research Consultants Impulse Survey of Focus Facilities
Association) • Find the best facilities in every market! Rated by
• Webinars recruit, location, value, etc.
• Free QRCA Views Magazine • Aim for an overall rating of 85 with lots of reviewers.
• Find a Researcher • Online version is $65
• Great annual conference (research suppliers only)
BellaVia Research • User Experience & Market Research • Santa Cruz CA • 831.454.8217 • info@bellaviaresearch.com 19
20. A word on recruiting…
You didn‟t ask me about recruiting, but I can‟t resist a chance to talk about it
Getting harder & harder to find quality participants who truly qualify (and aren‟t oddballs)
Lots of “friends and family” research – is that really your target?
Craigslist can be a problematic source! Easy to find warm bodies, but hard to find quality recruits
• Tend to be in it for the money (why else would you be trolling for focus groups on CraigsList?)
• Lots of dishonesty -- i.e., Woman in SF with >75 aliases, fake Facebook profiles, etc.
• According to one recruiter, 50% of her leads from Craigslist try to fake their way into research.
• Be careful! Many professional recruiters have database they build in a variety of ways – use them, use your own customer
lists, and use Craigslist sparingly & carefully (i.e., for niche recruits)
It‟s very easy to “lie” your way into
research when the posting or email
blast says exactly the type of
participant they are looking for…
BellaVia Research • User Experience & Market Research • Santa Cruz CA • 831.454.8217 • info@bellaviaresearch.com 20
21. Tips for Finding Great Research Participants
Open 2-way
Use carefully created Find cheaters / Ensure qualifications &
communication with the
materials! repeaters engagement
recruiter
• Use a professional • Hide your topic in the • Does your recruiter • Maximize open-ended
(it‟s worth it. Budget email blast (approve have a “wall of screening (engage in
$75-$150 per or write it yourself) shame”? Shared with conversation, don‟t
consumer recruit, more • Hide your topic in the other recruiters? give them a list of
for specialized B2B) pre-screener survey • Use “dummy” brands/behaviors to
• Bring the recruiter onto (approve or write it questions to sniff out click on)
the project team yourself) dishonesty (ask about • Make it easy for
before you start work • (don’t bias your flavors / features that people to be honest
on the screener research before it don‟t exist (i.e., “we‟re looking for
even starts. People • Require ID & write a mix, so please be
• Explain the type of do, and will, cheat check to that exact honest”)
person you are looking their way into name • Re-screen: confirm
for, & agree upon
strategies to find
research) • Verify identities key questions (pretend
• B2B: Call employer it‟s important to “get it
these people exactly right,” i.e., “I‟m
• (In the dog example: and ask for them by
• Agree what “research with pet title (esp. if they sorry I didn‟t take good
sources/approaches owners” should be won‟t share their notes on this”)
will & will not be enough, especially if company email • Homework! (send me
used (i.e., when/how you make people address) a picture of your music
to use Craig‟s List, collection, see
prove they have a • Look at their
email blasts, Survey dog via lots of Pinterest page, see
LinkedIn Profile (but
Monkey) pictures, & bring their your kitchen, a
beware of fake
• Include original treats into the profiles). For senior screenshot of your
participant source on group… pictures of execs, look at the iPhone apps…)
recruiting profiles dog’s gnarly teeth?) corporate website.
BellaVia Research • User Experience & Market Research • Santa Cruz CA • 831.454.8217 • info@bellaviaresearch.com 21
22. Great Recruiters
Hagen Sinclair
•Cynthia Hagen-Cross, President & CEO
•National recruiting, general pop & hard to find niche populations, online & in-person methodologies
•www.hagensinclair.com/ / hagenc@hagensinclair.com / (530) 899-3140
Blarry House Research
•Annaliese Furnas, CEO
•National recruiting, general pop & hard to find niche populations, online & in-person methodologies
•www.blarry.com / annaliese@blarry.com / 415-648-2196
Satellite Research
•Claudia Page, Owner & Research Specialist
•National recruiting, general pop & hard to find niche populations, medical, online & in-person methodologies
•www.satelliteresearch.net/ / claudia@satelliteresearch.net / (888) 639-5312
Intact Qualitative
•Richard Ngo, Partner
•SF recruiting only (& SOMA loft facility on 3rd Street)
•Great for SF Bay Area in-home or contextual interviews
•www.intactqualitativeresearch.com/ / richard@intactqualitativeresearch.com / 415.400.5945
Your Research Facility
•Doing research in a hired research facility? Use their team
•First, check Impulse Survey to make sure they are highly reviewed on recruiting
BellaVia Research • User Experience & Market Research • Santa Cruz CA • 831.454.8217 • info@bellaviaresearch.com 22
23. Your Turn!
BellaVia Research • User Experience & Market Research • Santa Cruz CA • 831.454.8217 • info@bellaviaresearch.com 23
Steve Jobs didn’t believe in focus groups for design. “It's really hard to design products by focus groups. A lot of times, people don't know what they want until you show it to them.“ Steve Jobs, BusinessWeek, May 25 1998Focus groups shouldn’t be used to ask people for design insights / solutions. Instead, they should be used to identify pain points so the strategists & designers can come up with the solutions!
For innovation? Yes, for early stage. For design iteration? Nope. Famous henry Ford quote:
Focus Groups are great when you know relatively little about a topic. Need to know how people perceive your brand / service / product? Want to understand what issues are most important? How people talk about something (language)? Need to define topics or develop questions for quantitative survey research? Need to listen for what you don’t even know about…as fodder for innovation?Examples:Thoughts / needs / feelings: Tequila – woman casually mentions that she does shots only to cut down on calories, bc margaritas have too many caloriesBrand perception & personality: People have a hard time articulating the differences between brands. It’s not a simple “ask & answer.” But by using projective techniques, you can cut straight to the heart of it. (colleges, amusement parks)Innovation & Use cases:Party planning…How do you use the internet to help you plan parties? (also did usability testing, and lots of it, but it was focus groups that provided big picture fuel for innovation)Decision process & factors: How do you decide which piece of art to buy? (to fuel site development)Communication: campaign for new alcoholic beverageWhat you don’t know: Why use us? Why are we losing customers? Situations where I wish I had done 1:1 interviews:Naming (highly subjective)Logos (highly subjective)Advertising (agencies are terrified of group think)“just project the site and see what they think” (only good for brand impressions, nothing about usability)
Also, great for projectives. Get below the high level, gut responses.