Assessment in CBME Competency Based Medical Education by Dr Girish .B, Associate Professor, Department of Community Medicine, Chamarajanagar Institute of Medical Sciences (CIMS), Chamarajanagar, Karnataka

1. Assessment in CBME
By
Dr Girish .B
M.B.B.S, M.D, M.B.A
Associate Professor
Department of Community Medicine
Chamarajanagar Institute of Medical Sciences (CIMS)
An Autonomous Institute of Govt of Karnataka,
Chamarajanagar

2. Objectives
At the end of the session the participant should be
able to:
Understand the differences between the traditional
assessment & Competency Based Assessment (CBA).
Understand the changes in assessment as per new
competency based curriculum .
Understand the components & tools of competency
based assessment.
Understand the role of feedback in assessment.
Plan, develop and implement CBA.

3. Flaws in current methods of assessment

4. Subjective
More of knowledge based
More of summative, norm referenced
Not reliable or repeatable
Unidirectional
No scope for feedback
No much importance to affective domain
Precipitates students dissatisfaction
Flaws in current methods of assessment

5. How does assessment differ ?
Conventional curricula
Competency
based curricula
Fragmented,
mostly summative,
norm-references
Integrated,
mostly formative,
criterion-referenced

6. Models & Assessment

7. Features of Competency Based Assessment
(CBA)
• CBA operates within the framework of
competencies. Assessment tools should align
with competencies/objectives.
• CBA should help to acquire competencies /
objectives (assessment for learning) & their
certification (assessment of learning)

8. Features of Competency Based Assessment
(CBA) ( 2 )
• CBA is continuous and ongoing process with
opportunities for providing developmental
feedback.
• Direct observation of students improves utility
of CBA and feedback.
• Multiple assessors, multiple tools & multiple
assessments improve the validity & reliability
of CBA.

9. Basic quality of Assessment methods
• Objectivity
• Relevance
• Reliability
• Validity

10. Miller’s Pyramid

11. Opportunities for assessment
A. Informal
B. Formal
• Internal assessment
• University examinations
–

12. Informal opportunities
• Provide the much needed feedback to the learner,
helping her to improve
• Dissociate assessment and decision making,
allowing students to ‘open up’
• Help to take away the stress of assessment

13. Types of Assessment

14. Formative assessment
• An assessment conducted during the
instruction with the primary purpose of
providing feedback for improving learning.

15. Purpose of Formative assessment
• Diagnose difficulties in students
• motivating students
• feedback to teachers

16. Summative assessment
(University examination)
• An assessment conducted at the end of
instruction to check how much the student
has learnt.

17. Paradigm of medical student assessment

18. Formal Assessments
• Internal assessment: How he learnt
• University examinations: How much he learnt
• Both test different aspects
• One is not a replacement for other

19. Internal assessment
• Range of assessments conducted by the
teachers teaching a particular subject with
the express purpose of knowing what is
learnt and how it is learnt.
• Internal assessment can have both formative
and summative functions.

20. Internal assessment: Extract from GMER
• Regular periodic examinations shall be conducted
throughout the course.
• There shall be no less than 3 internal assessment
examinations in each Pre-clinical / Para-clinical
subject & no less than two (2) examinations in each
clinical subject in a professional year.
• An end of posting clinical assessment shall be
conducted for each clinical posting in each
professional year.

21. Components of Internal assessment:
• Theory IA : Written tests, should have essay
questions, short notes and creative writing
experiences.
• Practical / Clinical IA : OSPE, OSCE, DOPS, mini-CEX
• Assessment of Log-book : record all activities like
seminar, symposia, quiz, etc. upto 20% of IA.
• Internal Assessment for Professional development
programme (AETCOM) : a. Written tests
b. OSCE based clinical scenarios

22. Internal assessment:
• In subjects that are taught at more than one phase,
proportionate weightage must be given for internal
assessment for each Phase.
For example,
• General Medicine must be assessed in
second Professional, third Professional Part I &
third Professional Part II, independently.

23. Internal assessment: (cont)
• Day to day records and log book should be given
importance in internal assessment.
• Internal assessment should be based on
competencies and skills.

24. Internal assessment: Eligibility
• Made mandatory before a student is allowed to
appear for the final university examination.
• Need at least 35% separately in theory and
practicals.

25. Internal assessment: 2nd Year

26. Aligning assessment with competencies

27. Aligning assessment with competencies

28. Feedback in Internal assessment:
• Feedback should be provided to students
throughout the course so that they are aware of
their performance and remedial action can be
initiated well in time.
• The feedbacks need to be structured & the faculty
and students must be sensitized to giving &
receiving feedback.

29. Feedback in Internal assessment:
• The results of IA should be displayed on notice board
within two (2) weeks of the test and an opportunity
provided to the students to discuss the results and get
feedback on making their performance better.
• It is also recommended that students should sign with
date whenever they are shown IA records in token of
having seen and discussed the marks.
• Internal assessment marks will not be added to
University examination marks and will reflect as a
separate head of passing at the summative examination.

30. University examinations
• University examinations are to be designed with a
view to ascertain whether the candidate has
acquired :
– the necessary knowledge,
– minimal level of skills,
– ethical and professional values,
– with clear concepts of the fundamentals,
• which are necessary for him / her to function
effectively & appropriately as a physician of first
contact.

31. MBBS Examination Schedule

32. Marks distribution for various subjects in
University examinations

33. Marks distribution for various subjects in
University examinations (2)

34. Theory examinations
• Use multiple tools like
–Long essay questions ( structured )
–Short notes
–Reasoning questions
–Short notes Applied aspects
–Short notes AETCOM
–Multiple choice questions (MCQs)
–MCQs not more than 20%

35. MCQs
• MCQs can test well in the cognitive domain at all
levels.
• It tends to assess only recall of knowledge,
however, it can test any higher level of the cognitive
domain.
• It can also discriminate well between students, if it
is well constructed.
• It improves the validity of tests.

36. MCQs

37. MCQs
• MCQs should be scenario based, single response
with 4 options in answers.
• Avoid one liner and negative terms in stem of
question.
• Avoid ‘all of above’ and ‘none of above’ in options.

38. Blueprinting in Knowledge domain

39. Practical / Clinical examinations
• To assess proficiency in skills, data interpretation
and logical conclusions
• Clinical cases should match what a practitioner is
likely to see in actual practice
• Avoid rare cases / syndromes.
• Focus on data gathering, physical examination,
writing records and management plans.

40. Assessing Skills in Competency
Based Assessment
Practical skills
OSPE
Clinical skills
OSCE
OSLER
Work place based assessment methods

41. What is OSPE / OSCE?
It is an assessment tool for
assessment of practical / clinical skill.

42. Characteristics of the OSPE / OSCE
It is an assessment approach primarily used to
measure practical/clinical competence.
Should be planned or structured (predetermined
practical/clinical competences).
Different types of test method can be
incorporated into it.

43. In most stations students are observed
(by one or more examiners).
Scored as they carry out the task or interpret
clinical materials (e.g. Laboratory data, X-
rays), write notes or answer question.

44. Harden’s 12 Tips for Organizing an OSPE/ OSCE
What is to be
assessed?
Duration of station
Number of stations
Use of examiners
Range of approaches
New stations
Organization of the
examination
Assigning priority
Resource requirements
Plan of the examination
Change signal
Records

45. Advantages of the OSPE/OSCE
Valid examination
The examiners can control the complexities
of the examination
Used as summative as well formative
Can be used with larger number of students
Reproducible

46. Advantages of the OSPE / OSCE (cont)
The variable of the examiner and the patient
are to a large extent removed
Fun activity within the department or college,
which promotes team work

47. Disadvantages of the OSCE/ OSPE
Knowledge and skills are tested in
compartments
The OSCE may be demanding for both
examiners and patients
More time in setting it up
Shortage of examiners
Might be quite distressing to the student

49. Cover wide range of skill
Communicative
Decision
Knowledge
Manual performance

50. Superiority of OSPE/ OSCE
Same content
Same criteria
Same examiner
High degree of validity & reliability

51. Station
What is a station?
Station organization
Types of station
Number of stations
Timing in one station

53. Disadvantage of OSPE
Time consuming
Space
Materials
Subjects

54. Training for OSPE
Examiner
Other teacher
Examinee
Staff

55. Spotter & OSPE
 Basic difference?

56. Simulation
• Going to occupy major place in T-L and
assessment.
• Assess students - Psychomotor domain
including soft skills (Communication ).
• Team skills also can be assessed - using Tele-
simulation.
• Skill Lab – Mandatory for all medical colleges (
GMER – 2019 ).

57. Simulation (2)
Human
– Standardized / Simulated Patient
(Communication & Clinical skills )
Technological / Non-human
– Mannequin - based simulation
( Part-task Trainers to virtual reality)

58. Objective Structured Long case Examination and
Record - OSLER
I. History taking (4)
 Pace & clarity of presentation
 Communication process
 Systematic approach
 Establishment of core facts
II. Physical examination (3)
 Systematic approach
 Examination technique
 Establishment of correct physical findings
III. Formulation of appropriate investigations in a logical sequence (1)
IV. Appropriate management (1)
I. Clinical acumen (1)
- Draws on previous nine items to assess candidates ability to identify and
solve problems

59. Advantages
• Involves examiner from the stage of History
taking till completion of case.
• Valid
• Reliable due to structured checklist
• Student becomes more methodical in his
approach to the students.
• Higher student satisfaction in assessment.

60. Work Place Based Assessment

61. What is WPBA?
• Assessment of performance of the
student at real life situation
– work place.
–Direct observation
–Conducted at workplace
–Followed by feedback

62. Learning to Drive
Driving Lesson
Assessment for Learning
Driving Test
Assessment of Learning

63. Rationale for WPBA
 Conforms to the highest level of Miller’s Pyramid
 Focus on clinical skills including the necessary soft skills
(communication, behavior, professionalism, ethics,
attitude)
 Context and content specificity
 Alignment of learning with actual working

64. Miller’s Pyramid

65. Methods of WPBA

66. Mini Clinical Evaluation Exercise (Mini CEX)
• A faculty member observes a trainee as he/she interacts
with a patient around a focused clinical task.
• Each aspect of the clinical encounter is scored by a faculty
member using a 9–point rating scale
– 1–3 is unsatisfactory
– 4–6 is satisfactory
– 7–9 is superior.
• Each patient encounter takes roughly 15 minutes followed
by 5– 10 minutes of feedback.
• Trainees are assessed several times throughout the year of
training with different faculty and in different clinical
situations

68. Direct Observation of Procedural Skills (DOPS)
• Trainee is assessed through out the day ,as he /she
performs the routine procedures on real patients
– Technical skill is observed,
– Way the trainee behaves,
– Way he interacts with fellow colleague, patients and
their bystanders,
– Professional behavior is also observed
• Feedback so received everyday can be used by the trainee
to better himself as days progress

70. Case Based Discussions (CBDs)
It is similar to the one practiced by us..
Ideal CBD includes patient-student
interaction
Student describes his encounter with the
patient with the facilitator
Facilitator gives feedback to the student

72. Mini Peer Assessment Tool (mPAT)
Trainee
Peer
1 Peer
2
Peer
3
Peer
4Peer
5
Peer
6
Peer
7
Peer
8

73. Checklist with 16 aspects will be used for
assessment
Diagnosis and appropriate application of
available investigative tools
Management of time
Management of stress, fatigue, and workload
Effective communication
Knowledge of one’s own limitations
Qualitative feedback..
Needs 15 minutes to complete the
assessment

74. Patient Satisfaction Questionnaire
Patient feedback tool to gather
patients assessment of residents –
communication skill / listening skills
or ability to answer queries, etc.

75. Challenges associated with WPBA tools
Standardization of tools
Time constraints
Busy OPDs
Students strength in medical colleges
Less number of trained faculty
Students are not oriented for these methods

76. MCI Website Modules

77. Covid 19 Pandemic situation
When Covid 19 Pandemic situation is going to end ?
When does students return back ?
Online assessment may be reality ?

78. Take Home Message
Prepare multiple assessment tools in department &
use according to the need.
Help students develop critical thinking & clinical
reasoning skills while acquiring competencies.
This compliments high state examination.

79. References :
 Medical Council of India. Assessment Module for Undergraduate
Medical Education Training Program, 2019: pp 1-29.
 Sharma R, Bakshi H, Kumar P. Competency-based undergraduate
curriculum: A critical view. Indian J Community Med 2019;44:77-80.

80. Thank you