1. APPLICATION OF ASSESSMENT AND
EVALUATION DATA TO IMPROVE A DYNAMIC
GRADUATE MEDICAL EDUCATION
CURRICULUM
TIFFANY L. SMITH, PATRICK B. BARLOW,
ROBERT E. HEIDEL, & WILLIAM METHENY

2. A LAYOUT OF THE PRESENTATION
Setting the Scene
– The Task
– The Complexities
Explanation of the tools created
and the process of creating them
1. Assessment Tool
2. Precourse Tool
3. Classroom Activities and
Homework
4. An Evaluation Tool
5. Fine-Tuning
The Result of Our Efforts
– How this process informs
our task
– A lesson
– Suggestions
Questions

3. SETTING THE
SCENE
Systems thinking and
complexity concepts
as frameworks for
conceptualizing
interventions
(Patton, 2010)

4. THE TASK
The creation of a blanket syllabus and set of learning
objectives for research design and statistics that can be
worked into any lecture series.

5. THE COMPLEXITIES
• Target Audience
– Time constraint
– Intrinsic motivation toward research
• Accreditation and Accountability
• Absence of Materials
• Stakeholder Buy-In

6. THE PROCESS
• What did we start with?
• What is it like to be without a formal set of
objectives or a syllabus?
• What do we do to change?
Where do we go from here?...

7. FIRST: AN ASSESSMENT TOOL
• Implementation of an initial assessment for
the purposes of understanding:
– Student baseline knowledge
– Student progress at the completion of the series

8. SECOND: A PRE-COURSE TOOL
• The initiation of a pre-course survey
– What do our students want to get out of the course?
– Who are they?
– What are their interests?
– Perceptions of the usefulness of the course

9. THIRD: CLASSROOM ACTIVITIES AND
HOMEWORK
• New classroom activities related to applying what
they’ve learned
• New homework assignments to assess their
ability to apply course content outside of the
classroom
• A final research proposal project
– Incorporating the course content into an oral
presentation

10. FOURTH: AN EVALUATION TOOL
• Development of a post-course survey to assess:
– Change in perceptions
– Change in attitudes
– Evaluation of instructor contribution
– Utility of the modules

11. A Constant Process of Fine-Tuning

12. THE RESULT OF OUR EFFORTS TO
UNDERSTAND AND OPERATE IN A DYNAMIC
SYSTEM
We implemented all of these sweeping
curriculum and measurement changes almost
exactly one year after our first experience
teaching the material, and it was universally the
most successful and rigorous workshop to date.

13. HOW THIS PROCESS
INFORMS OUR TASK
• As this academic year’s workshops get
underway, there will be a constant turning of the
feedback wheel
• Utilization of data
• Constant adaptation is KEY
• Constant, critical reflection on our own work
(Stevahn, King, Ghere, & Minnema, 2005)
• Must remember the complex nature of the task
• This was not only an implementation of a
course. Not just the implementation of a new
assessment.
• It was also the implementation of evaluation
into the process of course development and
modification.

14. A LESSON

Learning objectives, like many things,
cannot be looked at like a logic model,
with rigidity and fixedness, but instead
like a process that will
change over time.

15. SUGGESTIONS

16. Promote feedback and utilization of results

17. Remember the nature of the organization

18. Collaborate with colleagues

19. Use reflection for improvement and
development

20. Think outside the box

21. THANK YOU!
QUESTIONS?