This document provides guidance on how to effectively ask questions to gather user feedback. It discusses identifying goals and assumptions, engaging the right participants, formulating good open-ended questions, using follow-up questions and considering question format. Effective listening is also covered, including remaining neutral, engaging with participants and allowing silence. The overall aim is to facilitate discussions that prepare teams for gathering insightful client and user feedback.

1. How To Ask
The Right Questions
Hannah Fiechtner, Sr. Human Factors Researcher & Designer
hannahfdesign@gmail.com | hannahfdesign.com
Content Contributors from Cerner:
Jen Lundquist, Saul Aguilar, Jill Kurz, & James Bray

2. 1. How to Identify Goals
2. How to Engage the Right People
3. How to Ask Questions
4. How to be an Effective Listener
Notes
This mini-workshop will
• facilitate internal discussions
• prepare the development teams
for gathering client and user
feedback
• combination of content and group
activities
Asking the right questions, with a
dash of the scientific method, is
fundamental to most user feedback
methodologies

3. How to Identify Goals
Notes
You can’t ask the right questions if
you don’t know what needs you are
trying to fill or what questions you
are trying answer.

4. 1. Set & Document Goals
2. Uncover Assumptions

5. Involve team members & get buy in / Ask:
• What questions have come up in meetings? Client calls?
• What requests have come to the team that aren’t verified?
• What don’t we know about a workflow?
• What differences are there between our users? Are there
differences based on client or role?
• What is our focus? Human computer interactions? Content?
Workflows?
Allow for flexibility
Document your goals
Set & Document Goals
Notes
Get buy in
Teams are more open to feedback
from questions they helped define
Look for patterns in what team
members say
Identify the most important goals -
you will have a limited amount of
time and resources
Document your goals.
This will keep you and your team on
the same page - and let you know
when you’re done.
We are not robots - Allow for
feedback you didn’t think to include

6. Don’t discredit a hunch
Document assumptions & biases
Assumptions compound like the game of telephone
Uncover Assumptions
Notes
Don’t discredit a hunch - many of
science’s great discoveries were
based on intuition.
The point of research, and ‘asking
the right questions,’ is to turn those
hunches and assumptions into
theories and fact.
Be honest about assumptions
• how biases impact how we ask
questions
• how weak our current data is &
filling in the gaps
• what others on the team (or even
our clients) are assuming
• get on the same page

7. Uncover Assumptions
Don’t discredit a hunch
Document assumptions & biases
Assumptions compound like the game of telephone
Notes
Assumptions compound
Common flow for client feedback - 4
extra points where inaccuracy can
happen
Even if we ask the right questions,
we may get the wrong answers
because of how many layers of
people and their assumptions and
interpretations we have to go
through
User-centered research goal - take
out as many translation points as
possible

8. 1. Pair off.
2. Pick a project that you are working on now.
3. Write out 1 goal & 1 assumption you would like to address. (Make them user-centered)
3. Share it with your partner.
Group Activity 1: Goals & Assumptions
Sample Questions:
What questions have come up in meetings?
What un-verified requests have come to the team?
What don’t we know about a workflow?
What is our focus? Human computer interactions?
Content? Workflows?
Sample Questions:
What assumptions are made based on gaps in data?
What info do you use that has not come directly from an end
user or has not been documented?
What assumptions for a proposed solution would you like to
validate?

9. How to Engage the Right People
Notes
Even if you have the right questions,
asking the wrong people will still get
you bad information.

10. Determine Participants
Representative users
How many
Notes
Determine differences across your
user base - use several dimensions:
jobs they hold, tasks they do,
location, or specialty
If you identify differences, they
should be represented in the
research
Qualitative research – start with 2-3
users at 2-3 different locations for
each user type or primary variable

11. Determine Participants
Representative users
Notes
Distinguish end users from
stakeholders - often not the same
thing
• Business stakeholders concerns
- more efficient, increase quality
and sales, and abide by
government regulations
• End users concerns – enjoyable
workflows, information not data,
and less time in documentation
C-suite and managers - helpful in
understanding most vocalized or
systemic issues, rarely have entire
“front line” picture
Distinguish end users from stakeholders
How many

12. Determine Participants
Representative users
Across the spectrum
Product Familiarity
Subject Matter / Job Experienc
Techology Familiarity
Notes
Find representation across
spectrums from newbie to expert
• Are they familiar with the product
and use it frequently or are they
new?
• How much experience do they
have at their job, or with the
subject matter?
• What is their familiarity with the
technology you are using?
Participants can be at the top of one
spectrum and the bottom or middle
of the others
Distinguish end users from stakeholders
How many

13. 1. List 1 or 2 examples of how your questions and responses might differ
between business or development stakeholders and your end users?
Group Activity 2: Participants

14. How to Ask Questions

15. 1. Formulate Good Questions
2. Follow-up & Probing Questions
3. Consider Format
4. Answer Questions with Questions

16. Formulate | Ask open ended questions
No “yes/no” response
Typically begin with: What? How? Who? Where? When?
Do not suggest choices
Give control to the user

17. Open vs Closed
Closed Open
Do you like this new feature? What are your thoughts regarding this
new feature?
Do you prefer to enter the time in a text
field or choose from a dropdown menu?
How do you prefer to choose the time?
For what reason(s)?
Do you use this option? Describe to me how you use this option.
Formulate | Ask open ended questions
Notes
1 Answered with yes or no
2 Suggests choice
3 Answered with yes or no,
“describe” or “show” will lead to
much more in depth responses

18. Formulate | Ask open ended questions
However… closed-ended have their place
Notes
Most common use case - closed-
ended question followed by open-
ended question
Closed: Are there instances in
which you use this feature?
Open: What are those instances?
Describe to me how the feature
works for you in these instances?

19. 1. Write out 1 close-ended questions based on your research goals.
2. Write out 1 corresponding open questions.
3. Share!
Group Activity 3: Open Ended Questions
Closed Open
Do you like this new feature? What are your thoughts regarding this
new feature?
Do you prefer to enter the time in a text
field or choose from a dropdown menu?
How do you prefer to choose the time?
For what reason(s)?
Do you use this option? Describe to me how you use this option.

20. Formulate | Avoid leading questions
Imply that a particular response is more correct
Can influence user responses
Lead to false or inaccurate information
Notes
Most people want to please or go
along with the flow - may influence
the user’s response & gather
inaccurate/false information

21. Leading Alternative
It seemed like it was easy for you to go through that workflow.
Was it?
How was your experience with that workflow?
How much do you like this new feature? Tell me your thoughts about this new feature.
A lot of clinicians have problems with this screen. What do you
think about it?
What are your thoughts about this screen?
Formulate | Avoid leading questions
Leading Questions
Notes
1 Injecting your perceptions &
making it easy for the user to say it
was easy
3 Research shows implying a peer
consensus radically influence
answers

22. 1. Write out 1 leading questions based on your research goals.
2. Write out 1 corresponding non-leading questions.
Group Activity 4: Avoid leading questions
Leading Alternative
It seemed like it was easy for you to go through that workflow.
Was it?
How was your experience with that workflow?
How much do you like this new feature? Tell me your thoughts about this new feature.
A lot of clinicians have problems with this screen. What do you
think about it?
What are your thoughts about this screen?

23. Formulate | Avoid Double-Barrels
Two (or more) topics are covered in one question
Double-Barreled Separated
Do you think this screen is visually
appealing and easy to understand?
What are your thought about the visual
appearance of this screen?
What does the information on the
screen mean to you?
Double-Barreled vs Separated
Leads to confusion & unclear responses
Notes
Unclear to the participant which
topic they should respond to

24. Formulate | Alternatives to “why”
May put the user on the defensive
Requires the user to justify his/her actions
Implies that there is a “right answer”
Often results in little useful information
Notes
Alternatives to “why” lead to more
fruitful responses

25. Why Alternatives
Why did you make that decision? How did you go about making that
decision?
Explain/describe what went into making
that decision.
Help me understand your thoughts
when making that decision.
Formulate | Alternatives to “why”
Why Questions

26. Formulate | Avoid future or hypothetical
Users are not good at prediction
The best predictor of future behavior is past behavior
Use temporally-bounded questions
Use their experience not another’s
Notes
Users may appear confident when
reporting on future behaviors or
hypothetical situations - research
shows they are generally not good
at predictions
Best predictor – observe & ask
about current behaviors
Similarly, people are not good at
understanding other’s feelings and
behaviors - have them stick to
their own experiences

27. Formulate | Avoid future or hypothetical questions
Hypothetical Grounded
Do you think you would use this feature in the future? In the past month, how often have you used this feature
(similar feature, performed behaviors for which the feature
would be useful, etc)?
How frequently do you use this medication? In the past seven days, how often have you used this
medication?
Hypothetical vs Grounded
Notes
1 Try linking the use of the feature
to past similar behaviors or the
behavior outside of the system
2 Define a time range - get a
discrete, objective value

28. 1. Write out 1 hypothetical or future questions based on your research goals.
2. Write out 1 corresponding grounded questions.
Group Activity 5: Avoid hypothetical questions
Hypothetical Grounded
Do you think you would use this feature in the future? In the past month, how often have you used this feature
(similar feature, performed behaviors for which the feature
would be useful, etc)?
How frequently do you use this medication? In the past seven days, how often have you used this
medication?

29. Ask follow up & probing questions
Encourage user to expand and provide details
Follow up
Tell me more about…
What do you mean by…
Can you provide an example of…
What are your thoughts about…
Describe/Explain…Elaborate/Expand…
Follow Up Questions
Notes
Participants have a tendency to
abbreviate answers
• Automated behaviors
• Unsure how much info is needed
Asking for specific examples
• Forces participants to come to
terms with how frequently or
recently an issue actually arose
• The specificity of a singular
experience decreases the
likelihood they tell you what they
think they are supposed to do or
think, and instead exposes what
they actually did and thought

30. Consider format
General to specific
Allow for additional comments
Easy to hard or sensitive
Notes
Guided conversations, logical
transitions, not rigid Q&A sessions
Start with easy, general question
e.g., “Could you tell me about your
current job?”
Save sensitive or controversial
questions for later on - once rapport
has been established and the
participant has opened up
Broad to specific questions
• less likely to lead the user
• more likely to better understand
the context
End with open question
• Allow the participant time to
process
• highlight what was most
important
• Bring up topics not yet included

31. 1. Write out a set of 3-4 questions that progress from broad to specific based on
your research goals.
Group Activity 6: Consider format
Remember:
Easy to hard or sensitive
General to specific
Use follow up questions
Follow up
Tell me more about…
What do you mean by…
Can you provide an example of…
Describe/Explain…
Elaborate/Expand…

32. Answer Questions with Questions
Answering stops information gathering
Use their question as a springboard
Question Answer Question
Why does this screen look like this? Because we thought that was the best
way to do it.
Tell me how you would expect this
screen to look.
Why can’t I pick the option that I need? Your IT department built it that way. What option are you looking for?
Answer vs Question
Notes
Don't immediately answer questions
or jump to problem solving
• shuts down ability to gather more
information
• main goal is to get useful
feedback
Use question as springboard
If something was important enough
to bring up with a question - it is
probably important enough to gather
more information

33. How to Be an Effective Listener
Notes
We might still taint what might
otherwise be good information.

34. 1. Engage with the participant
2. Remain neutral & open
3. Don’t be afraid of silence
Notes
Listening is an active process

35. Engage with the participant
Express interest
Provide cues that you are listening
Don’t just take notes
Avoid interrupting
Adopt their language
Notes
You will get much more fruitful information if the user thinks you
are engaged and interested with what they have to say.
Engage
• make eye contact
• Maintain open, inviting posture
• Be empathetic
• Express interest
Listening cues
• slight nod of head
Don’t just take notes
• it is difficult to show you are engaged if you are constantly
taking notes
• consider audio recording or a note taker
Avoid interrupting the user
• cue to the participant - what they were saying is not important
• If they veer off - let him/her finish the sentence or find a
natural, say something like “Earlier you mentioned
{something} I’d like to ask you more about that.” - will make
the user feel heard while allowing you to get back on track
Adopt the user’s language
• shows you understand them
• free to continue using the language they are comfortable with

36. Remain neutral & open
Suspend judgement
Watch your behavior & body language
Document scientifically
Notes
Be receptive to all the information
you receive.
Suspend your judgement
even if not what expected or wanted
• avoid words like “good” or “right”
• leads the user to thinking there is
a correct response and they may
change their response to gain
your approval
Be careful of your behavior and
body language
Avoid expressions of surprise or
immediately jumping to take notes
Document
Be neutral & specific
• this is not the time for
conclusions
• document as accurately and fully
as possible

37. Don’t be afraid of silence
Give time to reflect
Use silence
Notes
Give time to reflect - some people
take longer to process questions
and when hurried may give a less
accurate response
Effective in expanding on a topic
• culture’s rhythm of conversation
• subtle cue to the other party that
it is their turn to talk or keep
talking

38. How To Ask
The Right Questions
Hannah Fiechtner, Sr. Human Factors Researcher
hannahfdesign@gmail.com | hannahfdesign.com