1. Multimedia-based Script
Concordance Testing as a method
of testing Clinical Decision Making
Skills
Todd Chang, MD
dr.toddchang@gmail.com

2. Background
• Assessing Clinical Decision Making Skills
(CDMS) is difficult but necessary
• Script Concordance Testing (SCT)
– Has been validated as a method of assessing
trainees in many subspecialties
– Is currently being studied for infant lumbar punctures
(LP) to assess infant LP management

3. Background
• Text-based questions and team-based simulations may
not isolate individual’s CDMS
– Multimedia questions have lower scores than text-
based
Holtzman KZ, Swanson DB, Ouyang W, Hussie K, Allbee K. Use of Multimedia on the Step 1 and Step 2
Clinical Knowledge Components of USMLE: A Controlled Trial of the Impact on Item Characteristics. Acad
Med 2009; 84(10s): s90-s93

4. SCT
A 1-month-old male has a rectal temperature of
40.3 Celsius. There is mild rhinorrhea.
Does the following change your likelihood to
perform an LP: He is RSV+
-2 Much less likely
-1 Less likely
0 No change
+1 More likely
+2 Much more likely

5. PICO Question
• Population
– In pediatric residents, subspecialty fellows, and
attendings
• Intervention
– Does the use of Multimedia depictions of clinical
scenarios
• Comparison
– Compared with text-based depictions
• Outcome
– Affect SCT scores negatively?

6. Approach
• Create 2 versions of every SCT question:
– text-based case scenario
– multimedia-based (screen-based simulation)
case scenario
Q1. An alert 2-week-old infant is
RSV+ and has a respiratory rate of
70 and subcostal retractions
Q1. 2-week-old RSV+

7. Approach
• Optimize a 15-question SCT set to test 2 CDMS:
– Infant Lumbar Puncture
– Infant/Pediatric/Adolescent Intubation
• Randomize multimedia vs. text-based &
administer questions
• Evaluate score differences against training
status, (sub)specialty status, and self-reported
experience

8. 3 Questions
• How do we create the optimal multimedia
element – VR vs. true patients?
• Should randomization be per question or per
student?
• Are there more optimal methods of validating the
SCT question set?
Todd Chang, MD
dr.toddchang@gmail.com