1. BLUEPRINT PROJECT
MCC Annual General Meeting
Beyond the Point-in-time
Assessments for
High-Stakes Decision
Making
September 14, 2014
1
2. Plenary Objectives
• Outline the progress of the Blueprint project
since AGM 2013 – Claire Touchie
• Review assessment and validity frameworks
for high-stakes decision making – Dame
Lesley Southgate
• Propose models for combining formative and
summative assessments for high-stakes
decision making – André De Champlain
2
4. Purpose of the Blueprinting
• … is to assure the public that physicians
licensed to practice medicine have the
required knowledge, skills and attitudes
for safe and effective patient care.
• Only those who meet this standard are
qualified to enter professional practice
4
5. Project Objectives
• ensure that critical core competencies,
knowledge, skills and behaviors required of a
physician entering supervised and unsupervised
practice are being appropriately assessed
The process will
• ensure that MCC assessments continue to fulfill all the
requirements and standards for credentialing examinations
• provide a clearly documented and deliberate process to
• update exam specifications
• respond to ongoing developments in the profession
5
7. Recommended Common
Blueprint
Dimensions of Care
Health
Promotion and
Illness
Prevention
Acute Chronic
Psychosocial
Aspects
Physician Activities
Assessment/
Diagnosis
Management
Communication
Professional
Behaviors
7
8. External Consultations
Governance Board July 5, 2013
UGME Meeting June 20, 2013
CaRMS August 22, 2013
Annual General Meeting
Sept. 15-17, 2013
AFMC – CACMS September 23, 2013
(11h45 – 15 mins)
AFMC – Council of Deans October 30, 2013
(in am – 15 mins)
Collège des médecins du Québec
(CMQ)
October 31st, 2013
RCPSC
Assessment Committee
November 7, 2013
UBC November 12, 2013
Student & Residents
(2 meetings)
November 18, 2013
(Meeting 1 of 2)
November 25, 2013
(Meeting 2 of 2)
FMRAC November 26, 2013
(9:00 a.m. – 2:00 p.m.)
CFPC
November 27th, 2013
(a.m.)
RCPSC
Education Committee
November 27, 2013
1:00 – 2:00 p.m.
PGME
November 27th, 2013
(7 – 9 p.m.)
IMG Symposium January 14, 2014
8
10. What have we heard?
• Overall support for the Blueprint dimensions and
definitions as proposed
• Importance of language:
– supervised means different things to different folks
– Residents do certain activities unsupervised
• Need to make explicit certain terms:
– Teamwork, patient safety, inter- and intra-professionalism,
leadership
– Concerns about the term for and placement of the dimension
“Psychosocial Aspects”
• Relevance of the D2 (and part II) when certification is the
final step to licensure
• All CanMEDS roles can be mapped to each dimension
• Focused on challenges and opportunities
10
12. Recommended Common
Blueprint
Dimensions of Care
Health
Promotion and
Illness
Prevention
Acute Chronic
Psychosocial
Aspects
Physician Activities
Assessment/
Diagnosis
Management
Communication
Professional
Behaviors
12
13. Definitions
Dimensions of Care
Focus of care for the patient, family, community
and/or population.
Health Promotion
and Illness
Prevention
The process of enabling people to increase control over their health and its determinants,
and thereby improve their health. Illness prevention covers measures not only to prevent
the occurrence of illness such as risk factor reduction but also to arrest its progress and
reduce its consequences once established. This includes but is not limited to screening,
periodic health exam, health maintenance, patient education and advocacy, and
community and population health.
Acute
Brief episode of illness, within the time span defined by initial presentation through to
transition of care. This dimension includes but is not limited to urgent, emergent, and life-threatening
conditions, new conditions, and exacerbation of underlying conditions.
Chronic Illness of long duration that includes but is not limited to illnesses with slow progression.
Psychosocial Aspects
Presentations influenced by the social and psychological determinants of health and how
these can impact on wellbeing or illness. The determinants include but are not limited to
life challenges, income, culture, and the impact of the patient’s social and physical
environment.
14. Definitions
Physician Activities
Reflects the scope of practice and behaviors of a physician practicing in
Canada
Assessment/
Diagnosis
Exploration of illness and disease through gathering, interpreting and synthesizing relevant information
that includes but is not limited to history taking, physical examination and investigation.
Management
Process that includes but is not limited to generating, planning, and organizing safe and effective care in
collaboration with patients, families, communities, populations, and other professionals (e.g., finding
common ground, agreeing on problems and goals of care, time and resource management, roles to arrive
at mutual decisions for treatment, working in teams).
Communication
Interactions with patients, families, caregivers, other professionals, communities and populations.
Elements include but are not limited to relationship development, intraprofessional and interprofessional
collaborative care, education, verbal communication (e.g.: using the patient-centered interview and active
listening), non-verbal and written communication, obtaining informed consent, and disclosure of patient
safety incidents.
Professional
Behaviors
Attitudes, knowledge, and skills based on clinical and/or medical administrative competence,
communication ,ethics, as well as societal and legal duties . The wise application of these behaviors
demonstrates a commitment to excellence, respect, integrity, empathy, accountability and altruism within
the Canadian health-care system. Professional behaviors also include but are not limited to self-awareness,
reflection, life-long learning, leadership, scholarly habits and physician health for sustainable
practice.
15. Assessment leading up to
Decision 1: Entry into Residency
Dimensions of Care
Health
Promotion
and Illness
Prevention
Acute Chronic
Psychosocial
Aspects
Row
Percent
Physician Activities
Assessment/
Diagnosis 30±5
Management 20±5
Communication 30±5
Professional
Behaviors 20±5
Column Percent 20±5 30±5 30±5 20±5 100
15
16. Assessment leading up to
Decision 2: Entry into Independent Practice
Dimensions of Care
Health
Promotion
and Illness
Prevention
Acute Chronic
Psychosocial
Aspects
Row
Percent
Physician Activities
Assessment/
Diagnosis 25±5
Management 35±5
Communication 20±5
Professional
Behaviors 20±5
Column Percent 20±5 25±5 35±5 20±5 100
16
18. Classification of MCC Content
3 days and 26 physicians
• 3514 Multiple-choice items
• 1321 Clinical Decision Making items
• 103 OSCE cases
18
19. Gap Analysis Results
MCCQE Part I
19
DIMENSIONS OF CARE
Health
promotion and
Illness
Prevention
Acute Chronic
Psychosoci
al Aspects
TOTAL
PHYSICIAN ACTIVTIES
Assessment/
Diagnosis
151 1654 715 33 2555
3.1% 34.2% 14.8% 0.7% 52.9%
Management
235 1068 467 27 1797
4.9% 22.1% 9.7% 0.6% 37.2%
Communication
34 53 22 28 137
0.7% 1.1% 0.5% 0.6% 2.8%
Professional
Behaviours
60 82 130 57 329
1.2% 1.7% 2.7% 1.2% 6.8%
TOTAL
480 2857 1335 145 4834
9.9% 59.1% 27.6% 3.0% 100%
20. Gap Analysis Results
MCCQE Part II
20
DIMENSIONS OF CARE
Health
promotion and
Illness
Prevention
Acute Chronic
Psychosoci
al Aspects
TOTAL
PHYSICIAN ACTIVTIES
Assessment/
Diagnosis
0.2% 37.2% 8.4% 2.5% 48.3%
Management
2.8% 7% 2.3% 0.6% 12.7%
Communication
6.5% 23% 2.9% 2.0% 34.4%
Professional
Behaviours
0.6% 3.2% 0.3% 0.5% 4.5%
TOTAL
10.1% 70.4% 13.9% 5.6%
21. Transition of Exams
• MCC Part I
– MCQs and CDM items
– Spring 2017
• MCC Part II
– New station types being piloted
– Focus on communication and professional
behaviors within a clinical context
– Still will expect components of history-taking and
physical examinations
– Fall 2018
21
22. Our Work Going Forward
Assessment Evolution –
Challenging the Comfort Zone
23. To meet overall Blueprint
• Will need additional assessments
– Opportunities
– Innovations
– Collaborations
23
24. Consultation Input - Opportunities
0
1
1
2
5
10
15
19
23
28
45
0 5 10 15 20 25 30 35 40 45 50
Harmonize with specialty examinations at the
Royal College
Standardize faculty of medicine OSCEs
E-portfolio rollout
Standardize ITER/FITER to include as
assessment tool
Common technical skills assessment tools
Encourage discussions with both colleges and
related initiatives
Standardize mini-CEX across schools
Self assessment
Influence accreditation standards wrt
assessment expectations
A UG ITER
Harmonize with surgical foundations exam
24
29. Workshop 1
An Assessment E-Portfolio across the
Continuum: A Discussion of the “What”
• Discuss the types of assessments to be
included in an e-portfolio for the two MCC
decision points
• Propose mechanisms for this to be feasible
and acceptable for schools, candidates and
regulators
29
30. Workshop 2
Feeding Forward: From A Student-Centered
to Patient-Centered Approach
• Define the meaning and understanding of
“feeding forward”
• Discuss opportunities and barriers to sharing
of assessment information within and
between stakeholder organizations for the
purpose of quality improvement and learning
30
31. Blueprint Project Team
The Blueprint Project Team
Dr. Claire Touchie Chief Medical Education Advisor – Project co-lead
Cindy Streefkerk Consultant – Project co-lead
Tanya Rivard Senior Test Development Officer and Special Projects
Anna Di Medio Project Administrator
Dr. Sydney Smee Manager, Strategic Initiatives and Clinical Skills Assessment
Jessica Hertzog-Grenier Director of Communications
Ingrid de Vries Associate Director – Evaluation Bureau
Becca Carroll Manager - Computer-Based Testing
Dr. Andrea Gotzmann Research Psychometrician
Dr. Marguerite Roy Medical Education Researcher
Sasha Papayanis Business Analyst – Information Technology
Alex Clay Project Coordination Manager
31
And initial discussions have identified some opportunities along the physician assessment continuum – from undergraduate education to Physician performance enhancement – revalidation
Walkthrough slide
Opportunity to come back once we know what we are doing for potential business/assessment opportunities and making use of existing MCC structures like TAC to help move these agendas forward.
What role would you see your committee playing?
To develop a collaborative process to identify assessment tools that can be shared nationally through shared processes which would need to be developed
To ensure that medical schools are assessing the core fundamental competencies that all physicians should demonstrate through mapping of their assessments to the MCC Blueprint dimensions
To explore, and if feasible, develop common shared content (items) platform for Cdns med schools
