1. Research Quality in
Qualitative Methods
Class Session 6
ADLT 673, March 17, 2014

2. Assumptions of Qualitative
Design
• Are concerned with process over outcomes
• Are interested in the meaning of experience –
how people make sense of their lives
• Act as the primary instrument for data collection
and interpretation
Qualitative Researchers ….

3. Assumptions of Qualitative
Design
• Involves fieldwork – the researcher physically
goes to the setting or site to observe or record
behavior
• Is descriptive—the researcher seeks to
understand in terms of words or pictures
• Is inductive—the researcher builds abstractions,
concepts, hypotheses, and theories from details
Qualitative Research ….

4. Characteristics of a Qualitative
Research Problem
The concept is “immature” due to a lack of theory or
previous research
The sense that the available theory may be inaccurate,
inappropriate, incorrect, or biased
A need exists to explore and describe the phenomena
and develop theory
The nature of the phenomenon may not be suited to
quantitative measures

5. In a paper or proposal, the
researcher should
• Specifically describe the type of design and its
approach to data collection, analysis, and report
writing*
• Narrative Inquiry
• Ethnography
• Case Studies
• Phenomenology
• Grounded Theory
* Not an inclusive list – these are some of the more common

6. In a paper or proposal, the researcher
should describe design characteristics
The discipline or field where it originated
A good definition of the design
The typical unit of analysis for the design
Types of problems investigated by the design
Types of data collection
Data analysis processes & formats for reporting
Any other special characteristics of the design

7. The Researcher’s Role
Researcher biases, background, steps taken re: reflexivity
Why this site? What was done at the site?
Will it be disruptive? How minimize? What effects might that
have on the quality of the data?
How will results be reported? What will the “gatekeeper”
gain from the study?
Indicate steps taken to obtain IRB permission
Comment on sensitive / ethical issues: confidentiality of
data, anonymity of participants, and intentions to use
research for intended purposes

8. Data Collection Steps
• Setting the boundaries for the study
• Parameters for data collection
• Purposefully select participants who are best suited to
provide insight into the research question
• Collecting information through observations, interviews,
documents, and visual materials
• Establishing the protocol for recording information
• Indicate the type of data to be collected and provide a
rationale for it
• Review Table 9.3 on pp. 150-151 of Qual Procedures article
for advantages and disadvantages of different methods

9. Thoughts on Interviewing
• NOT to get answers, test hypotheses, or evaluate
• In-depth interviewing is a means to understand the
experience of other people, and the meaning that
they make of their experience
• “Tell me about…”
• “Can you describe for me…”
• What, how, and why questions
• Interviewer: fewer words (short questions)
• Interviewee: many words (allow for expansive
answers)
• Use probes
• Watch for markers and follow them up with a probe

10. Conducting Focus Groups
• Requires a highly skilled facilitator to draw people out,
listen carefully, and encourage others to build on the topic
• Goal is to obtain in-depth, “rich, thick” descriptive data
• Important to build rapport with group – food helps, but
finish it before you start to record
• Pose question to the group, then wait for first response.
Build off that response by asking, “Can anyone else relate
to that experience” or “Has anyone else had a different
experience?”
• Encourage as many diverse responses as possible
• If the participant gives a short, one or two word answer,
ask for an example, and keep asking for examples

11. Tricks of the Trade
Use digital recorder(s) – need 2 or 3 for a table of 12 participants
No more than 4-5 questions per hour/ 6 to 8 in 1.5 hours
Round robin number your participants (no names) and have them
use their number before speaking
If you conduct the intervention, ask another (qualified) person to serve
as your facilitator
Use a transcriptionist familiar with qualitative data ; decide on degree of
clean-up in transcription (verbatim / pauses and hesitancies)
Always listen to your data; do not rely on transcripts alone

12. Characteristics: Narrative Studies
Stories from individuals and groups (can include
documents) about individuals’ lived and told
experiences
Stories are either told to the researcher, or co-
constructed between the researcher and participant
Strong collaborative feature in all narrative inquiry
Types of narratives: Autoethnography, life history, oral
history

13. Example: Seasons of a Man’s Life (1978)
• Developed the biographical interview method for
narrative studies
• Explored the male life cycle: his 1978 book has
remained the most comprehensive psychosocial
understanding of male adult development to date
• Enabled him to propose a theory of the life structure
as a series of stages, marked by the “mid-life” crisis
• Research Question: What does it mean to be an
adult?
• Levinson and his colleagues at Yale studied 40 men
from a sample drawn from four different
occupational groups

14. Example: Seasons of a Man’s Life (1978)
• Unique features of the biographical interview
method: following, rather than leading
• Interviewer is sensitive to feelings expressed by
interviewees and attempts to explore themes in ways
that have meaning for each study participant
• Interviewer and interviewee have defined roles, yet the
relationship between them is one of equality, both
being able to comment on the researcher on the basis
of personal experience.
• Not simply an interviewing technique or procedure, but
development of a relationship that permits greater
understanding of the interviewee’s experience

15. Triangulation: An Important Feature in
Qualitative Studies
Use of multiple measures of the same variable to
increase confidence that the data reflect the
phenomenon under study
• Source triangulation: use of a variety of data sources
• Investigator triangulation: Seven different
researchers contributed to the research team in
Levinson’s work; four co-authored the book that
resulted
• Methodological triangulation: use of multiple
methods to study a problem

16. Characteristics: Ethnography
Purpose is to study cultural phenomena in a natural
setting
Originated in anthropology
Researcher spends long periods of time living in the
host culture to study it
Methods are primarily direct observations of activities
of group studied; formal and informal interviews
Quality often depends upon time spend in observing
and interacting with informants
Use of extensive field notes (taken or recorded and
transcribed immediately after observations)

17. Quality in Ethnography (Patton,1990)
Comprehensiveness of field notes
Variety in information gathered from different perspectives
Cross-validation and triangulation of different data sources
Representation of participants in their own terms; capture
experiences in their own words
Selection of key informants wisely
Ability to synthesize vast amounts of data to compose a complete
picture
Provide carefully timed feedback as part of the verification process of
fieldwork; observe its impact
Field notes should include your own experiences, thoughts, and
feelings as the researcher

18. Examples from Medicine
• Boys in White (1961) by Becker, Geer,
Hughes, & Strauss – an early ethnography
• Original problem framed as a question:
What does medical school do to medical
students other than give them a technical
education?
• Evolved to become a study of medical
school culture; reflective of emergent
design

19. Examples from Medicine
• Getting Rid of Patients (1985) by Terry
Mizrahi – (based on dissertation research)
• Study of Internal Medicine residents’
experience in GME at SAMS (southern
area medical school); suspected to be here
at VCU
• Cultural study – 3 years; involved extensive
participant observation and interviews

20. Characteristics: Case Study
Best used to study “how” and “why” questions within
a bounded system
Ideal approach when the variables are inextricably
linked to the context in which they occur
The goal of a case study is exploratory, does not
include evaluation of an intervention
Tells a story, with intricate details, with special
attention paid to specifics about the setting, culture,
and the participants in the study
Individual can be the unit of analysis; can then conduct
a cross-case analysis of multiple individuals studied

21. Case Study of Eight Physicians
at MCV earning the M.Ed.
1. What impact has participating in the M.Ed. in Adult
Learning program had on how you teach?
2. What changes have occurred in your beliefs, values,
and assumptions about teaching and learning that
you can attribute to your participation in the
program?
3. To what extent has participating in the program
helped you to realize both personal and professional
goals as a medical educator?

22. Case Study of Eight Physicians
at MCV earning the M.Ed.
• Bounded system: cohort of physician learners
within one institution
• Methods included
• Interviews (beginning of program, mid-point, upon
completion)
• Focus group (someone else conducted)
• Inductive analysis of data from class learning
products: papers written, presentations made; CV
analysis; projects, etc.

23. Characteristics: Phenomenology
Purpose is to identify phenomena as they are perceived
by the study participants
What is the meaning, structure, and essence of the lived
experience of this phenomenon by individual(s)?
Direct investigation and description of phenomena
without a priori theories about causal explanations
Has many derivatives, such as phenomenological
heuristic inquiry (Moustakas): researcher must have
experienced the phenomena him/herself

24. Example:
Research Question:
How do midcareer women perceive and describe
transformative learning through developmental
relationships?
(data collection: participants kept journals for 6 months; in-
person and telephone interviews; focus group session)
Research Question:
What do medical students experience to be the essence or
essential structure of empathy? (interviews)
• Focus on the lived experiences of participants

25. Characteristics: Grounded Theory
Involves the discovery of theory through the analysis of
data
Involves both inductive and deductive thinking through
a constant comparison of data at different levels of
abstraction
Not a descriptive qualitative method
Researcher does not formulate hypotheses in advance
since preconceived hypotheses result in theory
ungrounded in data
Goal is to generate explanatory concepts: the unit of
analysis is the incident as reported by individuals

26. Key Concepts in Grounded Theory
Continual questioning of gaps, omissions, inconsistencies,
and incomplete understandings – this informs the need for
additional data on the situation being studied
Open processes in conducting research, rather than fixed
methods and procedures
Generate theory and data from interviewing, rather than
observation practices
Data collection, coding and analysis occur simultaneously,
not as separate components of research design
Inductive: theory must grow out of the data and be
grounded in the data

27. How does grounded theory work?
Initial (or open) coding and categorization of data
• Identify important words, or groups of words in the data and
then label them
• In vivo codes, words taken verbatim from participants
Concurrent data collection and data analysis
• Researcher collects data with an initially purposive sample
• These data are coded before more data are collected
• Researcher constructs a theoretical proposition and then
collects data to test the hypothesis
• Engages in a constant comparison analysis of incident to
incident, incident to codes, codes to codes, codes to
categories, and categories to categories
• Result is theory built up from the data themselves

28. Data Coding
Coding is analysis – Reviewing field notes or transcripts and
dissecting them meaningfully while keeping the relationships
among the parts intact
Codes are labels for assigning meaning
Codes are attached to “chunks” of data: either words, phrases,
sentences, or paragraphs
It is not the words that matter, but their meaning
Codes are used to organize and categorize data into themes and
patterns
Coding can be done manually or with the aid of qualitative data
software

29. Two Methods for Coding
• Can begin with a provisional “start list” of codes based on
conceptual framework prior to fieldwork. This is called a
priori coding
• List comes from list of research questions, literature review,
problem areas studied, key variables to create an initial codebook
which is modified through successive iterations of data analysis
• Coding can be done using inductive methods solely (grounded
theory)
• Initial data are transcribed and reviewed line by line, typically
within a paragraph. In the margins next to the paragraph,
categories or labels are generated, and the list grows. The list is
reviewed, modified, and continuously examined.

30. Want to learn more about
software for coding qualitative
data?
• Available YouTube Videos from manufacturers:
• Check out Nvivo 10 Tutorial
• http://www.youtube.com/watch?v=oelXFnJ-7Ms
• Another option is Atlas.ti
• http://www.youtube.com/watch?v=0skIbvmScsE (intro)
• http://www.youtube.com/watch?v=_f1jO3B4Z18 (coding)
• http://www.youtube.com/watch?v=YibqDB0iO-0 (coding and
analysis)

31. Rigor and Quality in
Qualitative Research Methods
Quantitative Qualitative
• Internal Validity
• External Validity
• Reliability
• Objectivity
• Credibility (truth value)
• Transferability
(applicability)
• Dependability or
Trustworthiness
(consistency)
• Confirmability (neutrality)

32. Credibility
Addressed by three issues: (1) the techniques and
methods used to ensure integrity and accuracy of the
findings; (2) the qualifications, experience, and
perspective that the primary researcher brings to the
study, and (3) the paradigm orientation and assumptions
that undergird the study (Patton, 1990)
Trustworthiness When researcher describes in
detail how successive interpretations of the data are
carried out and makes this available for public scrutiny in
publications; primary data should also be made available
to participants for their verification (Reissman, 1993)

33. You Try
Take a marker and use it as you review the three pages of
transcript material if you want to use it
Create codes in the margin as labels you assign codes to “chunks”
of meaning
Review codes with others at your table
Calculate inter-rater reliability of the first 10 codes you
(collectively) determine
Discuss discrepancies as a group and see if you can come to
agreement in meaning of a passage or group of words
What kind of coding guidelines can you, as a group, make to help
you do better as a coding team next time?

34. Next Week’s Meeting Times
• Monday, March 24
• 5 pm
• 6 pm
• 7 pm
• Thursday, March 27
• 5:15
• 6:30
• Blue (Fulco, Paletta,
Zulfiqur
• Red (McIntosh, Lee,
Ferrada
• Purple (DiGiovanni,
McGinn, Carlyle, Marko)
• Orange (Dragoescu,
Vinnikova, Woleben, Call)
• Green (Sabo, Massey,
deWit, Uram-Tuculescu