1. Question Design and
Methodological
Considerations
ADLT 673 – Teaching
as Scholarship in
Medical Education
Class Session 4

2. Agenda for February 24
Feedback on
Your Research
Questions
Formation of
Project
Discussion
Groups
Any changes
needed?
Team-Based
Learning
Exercise
3 TBLs for
remainder of
course
Grades
counted on two
(5 points each)

3. To move from opinionbased curricula to
evidence-based as a
result of research
findings
What do you perceive as
the goal of SoTL in Medical
Education?

4. Create a
clear
question
Disseminatio
n of results
Reporting
Medical
Education
research is a
highly
structured
process
Data
analysis
Choice of
methodolog
y and design
Subject
recruitment

5. What is a conceptual framework?
A model that organizes different parts of a study
into a coherent whole
System of
concepts, assumptions, expectations, beliefs, a
nd theories that support and inform your
research (Miles and Huberman, 1994)
A visual or written product that explains the main
things to be studied: key factors, concepts, or
variables

6. 4 common conceptual frameworks
Kirkpatrick model (modified by Barr et al., 2005)
• Learner satisfaction, Change in attitudes, Change in
knowledge/skill
• Change in behavior, Change in professional practice
• Change in patient condition , Cost Effectiveness
Translational Science Model
• Medical knowledge translates to practice – 3 levels
3-P Model: Presage-Process-Product
Cook classification – hierarchical, based on purpose of
study (description, justification, clarification)

7. Research Questions
Quantitative methods
• Does? Is (are)? Can? Should?
Qualitative methods
• What?
• Why?
Quantitative, qualitative, or mixed methods
• How?

8. Quantitative Approaches
Post-course
designs
Longitudinal
studies
Randomized
controlled
trials (RCTs)
Before and
after studies
Controlled
before and
after studies

9. Qualitative Approaches
Ethnography
Action
Research
Case Studies
Phenomenology
Grounded
Theory

10. Terminology in Qualitative
Methodology
Inductive Approach
Iterative Approach
Thematic Analysis / Emergent Design
Reflexivity
Representativeness
Saturation
Triangulation
Resonance or member checks

11. Rigor and Quality in Research
Methods
Quantitative

Internal Validity
Qualitative

Credibility (truth value)

Transferability
(applicability)

External Validity

Reliability

Dependability
(consistency)

Objectivity

Confirmability
(neutrality)

12. Models for Mixed Methods
Research in Medical Education
Instrument Development
(questionnaires, checklists)
Explanatory Model (to complement or clarify)*
Triangulation Model (data collected
simultaneously)*
Longitudinal transformation (multiple data
points, often more than one population)

13. Issues for Consideration
Availability of resources to conduct the research
(time, money, and personnel with requisite strengths)
Access to tools and programs (software, data security and
storage)
Challenges in publishing mixed methods studies
• Need to have expertise in both qualitative and quantitative
• Need to be able to address alternative (pragmatic) paradigm
Decide on prominence of each data type; formulate questions
appropriate for each aspect of study
Decisions on how and when data are collected, analyzed and
integrated