1. Pan-Canadian Practice Ready Assessment for IMG Physicians:
A competency-based assessment for provisional
licensure in family medicine
Cindy Streefkerk, Dan Faulkner, Lauren Copp, Sydney Smee, PhD, André De Champlain, PhD, Timothy Allen, MD
Ottawa Conference
27 April 2014 - Ottawa

2. Disclosure Statement
I have no actual or potential conflict of interest in relation to this
presentation.


3. © NAC / MCC 2014
1. Background
2. Practice-Ready Assessment as a
process
3. Family Medicine PRA standards
3
Overview

4. © NAC / MCC 2014
25 per cent of practising physicians in
Canada are International Medical
Graduates (IMGs)
Rural and remote areas are the most
underserviced
13 jurisdictions govern licensure
4
Background

5. © NAC / MCC 2014
Ensure public protection
Credentials and experience are often
“unknowns”
No equivalent processes across
jurisdictions
• Complex pathways
Limited capacity to integrate IMGs
5
IMG Physician Situation & Challenges

6. © NAC / MCC 2014
1. Entry into Canadian Residency
2. Entry into Practice through
provisional licensure
a) Direct – credentials only
b) Practice-ready Assessment
6
IMG Physician Routes to Practice

7. Today: Current Processes for Entry-to-Practice

8. © NAC / MCC 2014 8* Miller’s pyramid of competence
Miller’s Pyramid & PRA – Assessing
Clinical Competence
NAC PRA Type
In Practice Assessment
Over-Time
Assessment
Selection
(Interactions with trained patients and
assessors - OSCE)
Point-in-
Time
Assessment
Selection
(Therapeutics, CDM, short-answer)
Screening
(MCQ – MCCEE)
SHOWS
HOW
DOES
KNOWS
HOW
KNOWS

9. © NAC / MCC 2014
Medical Regulatory Authorities (9) and
Federation of Medical Regulatory
Authorities
PRA Programs (8)
Certifying College – College of Family
Physicians of Canada
Provincial Funders (8-9)
Federal Government (Health Canada as
a project funder)
9
Stakeholders – In Collaboration

10. © NAC / MCC 2014
Standards as a starting point
• Precluded judgment of what was currently happening
• Set a common goal and expectations of each other’s
processes
• Focused on “what” not “how”
Ultimately, to trust in each others rigour for
provisional licensure purposes
10
Approach to pan-Canadian PRA

11. © NAC / MCC 2014
Baseline of current practices and
processes
Reviewed and synthesized possible
processes and tools into standards
Integrated existing practices into the
common standards, with a view to
improve
11
An AIterative Process

12. © NAC / MCC 2014
Follow a sampling framework
Sample as many observations as possible
More assessors are always better
Target assessment tools to competencies of
interest
Provide ongoing, structured feedback to
candidates
12
Results – Over-Time Assessment
Process

13. 13
Results: What a Family Physician Does
Sentinel habits define essential, priority skills that
are comprehensive and easily recognizable in busy
clinical settings
1: Incorporates the patient’s experience
and context into problem identification
and management
5: Uses generic key features when
performing a procedure
2: Generates relevant hypotheses
resulting in a safe and prioritized
differential diagnosis
6: Demonstrates respect and/or
responsibility
3: Manages patients using available best
practices
7: Verbal or written communication is
clear and timely
4: Selects and attends to the appropriate
focus and priority in a situation
8: Seeks out and responds appropriately
to feedback

14. 14
Results: Who they see
Clinical domains define the various populations and
activities that physicians encounter in clinical
settings
1: Behavioural medicine/mental health
5: Care of the vulnerable and
underserviced
2: Care of adults 6: Maternity/newborn care
3: Care of children and adolescents 7: Palliative care
4: Care of the elderly 8: Procedural skills

15. © NAC / MCC 2014
Assessment occurs in a practice
environment (community-based)
• Rich in patient care opportunities
Allow candidates time to acclimatize
Allow adequate time to assess response
to feedback
Should not take longer than 12 weeks to
determine practice-readiness for
provisional licensure
15
Results: Assessment place & time

16. © NAC / MCC 2014
Experienced, competent family
physicians
Hold a licence to practise medicine &
be in good standing
At least three years of practice in
Canada
Receive
• Ongoing support & feedback
• Training & orientation
16
Results: Who is the Assessor

17. 17
Results: Over-Time Assessment Toolkit
Multi-Source Data
Chart-Based
Components
Continuous Clinical
Assessment
DEFINED
Focus is on communicator,
collaborator & professional
roles
• Chart stimulated recall
• Chart audits
• Case-based discussions
• Mini-CEX
• DOPS
• CBAS
• Field notes
STANDARD
• Feedback comes from
patients & professional
colleagues
• Feedback is documented
• Demonstrates ability to
meet regulatory standards
for charting
• Observation of chart-based
assessments are
documented
• Observations cover all sentinel
habits across all clinical domains
• Observations occur across time &
patient problems
GUIDELINE
Ideally, feedback comes from:
• Minimum of 15 patients
sampled as broadly as
possible across
demographics & problems
• 5-8 professional colleagues
(MD & non-MD)
Assessor judgement
determines the number of
charts for review
• More than one clinical setting
may be required to ensure
appropriate sampling
• Ideally,
• If field notes only, one/day
totaling 40-80
• If mini-CEX (or equivalent),
one/week totaling 8-12

18. © NAC / MCC 2014
Future standards as a first step
• In a complex environment – varying processes
• With multiple stakeholders with a perspective
• Opportunity to bring subject matter expertise to the
table
To ultimately ensure
• Jurisdictions trust each others rigour
• Candidates have a fair/consistent process
18
Summary

19. Thank you!
Questions, comments, concerns?
Cindy Streefkerk - cstreefkerk@mcc.ca