1. Procedural Assessment:
Where do we stand?
March 2013
Walter J. Coyle MD, FACG, FASGE

2. Objectives
 The apprentice model
– The way we were
 Competency based education
– Where we are now
 Outcomes based learning
– Procedural focus
– Milestone development

3. Apprenticeship
 Successfulfor thousands of years
 Key properties:
– Good mentor
– Motivated student
– Adequate exposure AND hands on time
 Problems:
– Consistency
– Objective measures of success
– Low ceiling for promotion
The Mystery of Mastery. Psychology Today 1986;20:32

4. Apprenticeship
 Works well for very sub-specialized areas
and few centers of excellence
 Still model for advanced endoscopy
– AEF match
– Variation in level of exposure and mastery
– Who monitors the mentors?
– How do the graduates do?
 Medicine resents outside monitoring
– Better if we did it ourselves

5. Competency based training
 ACGME initiative from the 1990s
– Applied to all aspects of training
– Knowledge, professionalism, procedures
 Ineffective for procedural training
– GI procedures still in apprentice model
– Little consensus on assessment and outcomes
– Little data to define milestones
 How should a 2nd year fellow scope?
Lurie. Med Educ 2012;46:1365

6. Competency based training
 Diverse training methods and assessment
techniques
 Small programs vs large; research fellows
 Silo mentality: no consistent standard
 Explains why we have this problem now

7. Outcomes based learning
 More process oriented
 Focus on the process not the problem
 ACGME wants us to move here
 Starting point: 1st year fellow
 Ending point: Staff GI
 Milestone development: easier for
knowledge core vs procedures
N Engl J Med 2012;3686:1051-56

8. Milestone development
 Final milestone: Colonoscopy
– >95% cecal intubation rate
– >25% ADR
– Low complication
– Patient satisfaction
 Stepwise milestones: None with great data
or evidence
– 1st year vs 2nd vs 3rd
Gastrointest Endosc 2010;71:319-24

9. Procedural Education: initial
focus on process
 Intense didactic
– FYF course, DVDs, local resources
 Intense hands on training with scope
– ? Simulators
– Training box/tool
– Standardized patient
– Example of pilot training?

10. Procedural Education:
subsequent focus on process
 Ongoing, continuous assessment:
– Mentor feedback; patient feedback
– Objective outcomes based assessment tool
– Universal tool ?
 Development of outcomes based, data
driven milestones that apply throughout
fellowship
– How???

11. Procedural Education: A
Proposal
 Universal
assessment tool agreed upon
 Web-based submission of assessments
– Collection and development of milestone
– Feedback to fellow and program
– Fellow compared to peers nationally
 Progressionthrough milestones will be
fellow driven, not fixed year driven

12. Data on Fellow
Sedlack, GIE 2010;72:1125-33

13. Procedural Education: A
Proposal
 Requirements of system
– Ease of use: minutes, APP for phone, link on
desktop
– Secure
– Can provide data back to program and fellow in
real time
– Dynamic and progressive

14. Procedural Education: A
Proposal
 Cost:GI programs, GI societies, ACGME
 Web site location and maintenance
– ACGME
– CORI database like initiative
– GIQuik
– Endoscopic report generating systems
 Provation initiative with Mayo Clinic
 Time frame

15. ASGE Proposed Forms

16. Integrated Assessment
 Are we ready and committed??
 Resource Commitment
 Staff Commitment
 Barriers breaking silos
 Only definite: change is here

17. Summary
 Prior models and procedural mentoring are
probably inadequate
 Classic competency based assessment is
flawed for procedures
 Outcomes and milestones are a next step
 GI directed development of milestones and
tools is critical

18. Summary
Need to think of the process NOT
the problem