AWS Community Day CPH - Three problems of Terraform
Running a Focus Group 101
1.
2. Presentation (40 minutes)
Practice moderating (60 minutes)
Debrief and questions (20 minutes)
3. Maintain neutrality
Control the environment
Guide the conversation
Generate valuable information
4. Focus groups are unstructured qualitative
research tools, meaning the goal is to have a
conversation with the participants and hear
their responses in their own words
5. Focus groups are a tool for gathering
qualitative data from a population of interest
Focus groups can be used to explore a topic in
order to gather feedback on that topic
Focus groups are not instruction/outreach
opportunities (though that is often a by-
product)
We believe that the participants have
valuable information to share with us
6. “Researchers attempt to capture peoples explanations
for how things happen”
Focus groups are good for answering “how?” and
“what?” questions – open ended question
Focus groups are great when you want to look at a
topic up close rather than get a panoramic view of the
entire issue
Focus groups include the researcher in an active way
Focus groups are designed to uncover insights and
perspectives that are not retrievable by other
methods from a small group of participants
7. Standard
Uses a rigid set of questions
Guided
Uses a set of topics to explore, but the question
wording is flexible
Exploratory
The most informal, questions arise through the
course of the conversation about the topic
8. Engagement questions: introduce participants to
a topic and make them comfortable with the
topic of discussion
What is your favorite toothpaste?
What do you notice when you look at other
people’s teeth?
http://assessment.aas.duke.edu/documents/How_to_Conduct_a_Focus_Group.pdf
9. Exploration questions: get to the meat of the
discussion
Who in particular has influenced your dental
habits?
What are the pros and cons of flossing your teeth?
When you floss, how do follow through? When
you don’t, why not?
How do you feel when told about possible
damage caused by not flossing?
How do you feel about yourself when you floss
regularly? When you don’t?
http://assessment.aas.duke.edu/documents/How_to_Conduct_a_Focus_Group.pdf
10. Exit question: check to see if anything was missed
in the discussion
Is there anything else you would like to say about
why you do or do not floss your teeth on a regular
basis?
http://assessment.aas.duke.edu/documents/How_to_Conduct_a_Focus_Group.pdf
11. Representative samples of your population
“Statistically significant”
12. Quantitative (e.g., surveys) Qualitative (e.g., focus groups)
Objective: factual, able to be counted Subjective: opinions, judgments, beliefs
Deductive: arrives at a specific Inductive: progresses from observing
conclusion based on information individual cases to develop a general
assumed to be true understanding
Tests a hypothesis Uses grounded theory: generates a
hypotheses from the data gathered
Employs instruments that reduce data to Employs instruments that produce
numbers linguistic data
Results are reported as numbers and Results are reported as words and
statistics phrases
Researchers use a tool to collect data Researcher is actively involved in the data
collection
Research design is fixed Research design is flexible and can evolve
throughout the process
13. Researchers (us!) have an ethical
responsibility to protect participant privacy
The Institutional Review Board at Columbia
monitors Human Subjects Research
14. Distribute the questionnaire
Welcome, thank you and introduction
Read from script
Engagement question
Exploratory questions + follow-ups
Exit question
Thank you!
15.
16. Respect the participants
Show empathy and positive regard (be Buddha)
Understand the purpose of the study
Communicate clearly
Be open, not defensive
Nonjudgmental
Get the most useful information for your
colleagues about the topic at hand
You ARE the research tool
Exercise emotional discipline.
20. Smile
Listen actively
Good eye contact
Head nodding, slowly (not in agreement (fast),
but encouraging further disclosure)
Use neutral language responses to encourage
the speaker to continue: “yes” “ok” “mm hm”
Avoid judgmental language: “that’s correct”
“you’re right” “I agree” “no” “good idea!”
“what?!”
Don’t interrupt
Be careful with humor – just be yourself!
21. Can you explain more?
Can you give an example?
Would you say a little more about that?
Tell us more.
Say more.
Is there anything else?
Please describe what you mean.
I don’t understand.
Can you be more specific?
What do you mean by….?
If other participants are nodding their heads in agreement or
disagreement with the speaker, this should be recorded in the
notes. The facilitator should ask the other participants for more
information: “You’re nodding/shaking your head. Can you share
what you think about this?”
22. “I think you’re saying…”
“Let me know if I heard correctly…”
Asking participants to confirm what you’re
hearing can be very valuable. Always ask for
clarification if you need it. Be careful not to
sway the conversation by introducing your
own perspective.
23. Be aware of your biases and judgments
Treat everyone equally
Be aware of your body language
We already know your opinions and experience;
your input will be taken into consideration at a
different point in the project
Nonjudgmental
Create a comfortable, non-threatening
environment.
24. A note-taker’s role is to take notes. This
person is neither a participant, nor a
facilitator.
The note-taker should refrain from
contributing to the discussion unless called
on directly by the facilitator
Note takers should try to be as descriptive as
possible, practically transcribing the
conversation.
25.
26. Groups should be relatively homogenous, to
maximize disclosure
Gender
Age
Power
“Cliques”
27. Some people “think aloud.”
Restrict your eye contact with this person, as
necessary. Look at your papers, or other
participants.
Do not nod, smile, or otherwise encourage them
to continue speaking with your body language
“Thank you. Let’s hear from someone else now.”
“Does anyone feel differently?” “Does anyone
have anything else to contribute on this topic?”
28. Others may be reflective thinkers who don’t
“think aloud.”
Maximize eye contact, smile encouragingly
“Chris, what do you think?” “Ted, you haven’t
had a chance to speak. How do you feel about
this?” “Carole, I know you’ve had experience
with this. Can you share it?”
If no one is talking, call on people. It’s ok!
29. Silence isn’t bad; give people time to think about
their responses
Be aware of your body language; wait patiently
Restate the question to prompt a response
Call on someone. You can often tell who wants
to speak, but might be shy.
Don’t force the development of an opinion
Try not to interrupt the natural flow of the
conversation
30. We want to encourage people to state different
points of view, even if they’re contrary to
what someone else may have said. Always
allow for a different perspective.
Is there something we haven’t thought of/discussed?
Has anyone else had a different experience?
Does anyone feel differently about this?
Do you have a different point of view or opinion?
31. Resist the urge to correct participants – usually
the group will self-correct itself. You’ll be
surprised!
32.
33. Keep a clock or watch on hand and track the
timing carefully.
“I can see you all feel passionately about
this topic, but we need to move on to
something else, in the interest of time.”
34. It’s always a good idea to offer refreshments!
Other incentives should be distributed at the
end of the session. Participants who choose
to leave without completing a session should
receive the incentive.
35. Give each participant a blank piece of paper at
the beginning of the session.
“If there’s anything you don’t get to say during
the session, or something important that we
don’t cover, please write it on the paper, and
I’ll collect them at the end.”
36. As soon as the focus group is over, the
facilitator and note-taker should spend at
least one hour going through the notes to
recall as much information as possible to
include in the notes.
Refrain from making judgments or
interpretations at this point. This is not
analysis. Focus on recording as much
information as possible.
37. Focus Groups: A Practical Guide for Applied
Research by Richard Krueger and Mary Anne
Casey
Advanced Focus Group Research by Edward R.
Fern
Practical Research Methods for Librarians and
Information Professionals by Susan Beck and
Kate Manuel
38. Engagement questions
I’d like you to tell me a little about your last vacation. Where did you go? With whom?
Do you usually go on family vacations, with friends, or travel alone? Which do you prefer?
Exploration questions
How did you plan your last vacation?
Did you use a travel agent or book everything yourself?
If you used online resources to book your vacation which ones were the most helpful, which were the
least helpful?
What types of activities do you usually do when you’re on vacation?
Do you take photos when you go on vacation? If so, what do you do with your photos after the vacation?
If you flew, how was that experience for you?
Do you feel you have enough vacation time?
Did you rent a car on your last vacation? If so, how was that experience?
How do you budget for your vacations? (Do you save money for each trip specifically or have an annual
“vacation” budget?)
What do you love about going on vacation?
What do you hate about going on vacation?
Exit questions
Is there anything else you’d like to share about going on vacation?