1. ALERT PRESENTATIONS
Consult Group Breakouts
IMSH 2012
Presentations: 7:00-7:45pm
Breakouts: 7:45-9:30pm

2. Format
• Discuss project for 30 minutes
– Presenter stays at table
– Consultants rotate tables x 3
• Transition for 5 minutes
– Summarize state of consultation
– Presenter highlights key questions

3. 1. Chang- Script Concordance LP
2. Barry- BVM training
3. Kummett- Neonatal Skills
4. Mehta- Health literacy
5. Sherzer- Epi pen community
6. Levy- PALS tool validation
7. Maa- PALS performance tool
8. Maa- Hybrid-simulator
9. Meyer- Donation after Cardiac Death
10.Overly- Structured-patient encounter

4. Multimedia-based Script
Concordance Testing as a method
of testing Clinical Decision Making
Skills
Todd Chang, MD
dr.toddchang@gmail.com

5. Background
• Assessing Clinical Decision Making Skills
(CDMS) is difficult but necessary
• Script Concordance Testing (SCT)
– Has been validated as a method of assessing
trainees in many subspecialties
– Is currently being studied for infant lumbar punctures
(LP) to assess infant LP management

6. Background
• Text-based questions and team-based simulations may
not isolate individual’s CDMS
– Multimedia questions have lower scores than text-
based
Holtzman KZ, Swanson DB, Ouyang W, Hussie K, Allbee K. Use of Multimedia on the Step 1 and Step 2 Clinical Knowledge
Components of USMLE: A Controlled Trial of the Impact on Item Characteristics. Acad Med 2009; 84(10s): s90-s93

7. SCT
A 1-month-old male has a rectal temperature of
40.3 Celsius. There is mild rhinorrhea.
Does the following change your likelihood to
perform an LP: He is RSV+
-2 Much less likely
-1 Less likely
0 No change
+1 More likely
+2 Much more likely

8. PICO Question
• Population
– In pediatric residents, subspecialty fellows, and
attendings
• Intervention
– Does the use of Multimedia depictions of clinical
scenarios
• Comparison
– Compared with text-based depictions
• Outcome
– Affect SCT scores negatively?

9. Approach
• Create 2 versions of every SCT question:
– text-based case scenario
– multimedia-based (screen-based simulation)
case scenario
Q1. An alert 2-week-old infant is RSV+
and has a respiratory rate of 70 and
subcostal retractions
Q1. 2-week-old RSV+

10. Approach
• Optimize a 15-question SCT set to test 2 CDMS:
– Infant Lumbar Puncture
– Infant/Pediatric/Adolescent Intubation
• Randomize multimedia vs. text-based &
administer questions
• Evaluate score differences against training
status, (sub)specialty status, and self-reported
experience

11. 3 Questions
• How do we create the optimal multimedia
element – VR vs. true patients?
• Should randomization be per question or per
student?
• Are there more optimal methods of validating the
SCT question set?
Todd Chang, MD
dr.toddchang@gmail.com

12. Improving the Effectiveness of Bag and
Mask Ventilation Training in an
Academic Center NICU
Jim Barry
Medical Director, University of
Colorado Hospital NICU

13. Background
• BMV is a simple skill that is simply done wrong
frequently
• Simulation and Learning (and not forgetting)
Theory
– Retrieval based testing/learning improves short and
long term memory- THE WAY WE TEST MATTERS
– Partial task trainers and deliberate practice can
improve skill attainment in trainees (LP)
• Knowledge Gaps
– Can the combination of deliberate practice and
retrieval testing improve skill acquisition and retention
for BMV ?

14. PICO Question
• Population
– Primary- 2011- 2012 Pediatric Residents from Univ Co
– Secondary- other NICU staff: RNs, RTs, NNPs, MDs
• Intervention
– Formal training: deliberate practice, knowledge (retrieval vs
recognition), spacing
• Comparison
– Subjects in 3 groups varied by testing: Recognition, Retrieval,
Retrieval+Practice
• Outcome
– Changes in BMV Knowledge and Skill at 1 month and 6-12
months later

15. Approach
• Randomized controlled study
1. Randomize teams monthly into 1 of 3 study groups
2. Knowledge evaluation with pre/post-test (Retrieval or
Recognition) beginning/end of month and 6-12 months
3. Pretest, questionnaire to determine BMV experience and
career choice
4. Evaluate BMV skill and equipment knowledge with video-
recorded session using apneic neonatal partial task
trainer
5. BMV scoring tool and mastery learning applied
6. Pre/PostData collected: Knowledge, BMV skill

16. 3 Questions
• What would be the best format/time to evaluate
long(er) term retention? 3,6,9 months
• Currently at single center, could this be
replicated at other sites?
• How could this education/intervention be applied
to patient outcomes?

17. ALERT Presentation:
Maintaining Proficiency in Bag
Mask Ventilation, a two month
recommendation
Gary Kummet, Julie Lindower

18. Background
• 60 second delay of effective ventilation in adults
after cardiopulmonary arrest decreased survival
by 9%.
• Bag mask ventilation training
– Improves skills immediately post training
– Requires significant exposure to achieve proficiency
– Skills decay rapidly with time (6-7 months)
• Knowledge gaps
– Does early establishment of ventilation improve
pediatric outcomes?
– What is the training interval to maintain proficiency?

19. PICO Question
• Population
– Pediatric and Family Practice residents at a tertiary medical
center
• Intervention
– will receive bag mask ventilation training every two months
• Comparison
– compared to conventional training (NRP/PALS at orientation)
• Outcome
– Improved procedural skills competency six months after initial
training

20. Approach
• Randomized controlled study
1. Provide initial training to all residents to achieve
baseline proficiency
2. Randomize residents into one of two study groups
3. Re-training at two month intervals to study group
4. Respiratory arrest scenario at 6 months (videotaped
with objective mannequin feedback)
5. Feedback and debriefing

21. 3 Questions
• What are the technical requirements of the
simulator for this study?
– Taking the test to the tester/simulator portability
– Obtaining objective data
• What issues do we need to consider when
standardizing the research across study sites?
– Initial training, testing, re-training, re-testing
• What are clinically significant differences in
performance?

22. Appropriate Health Literacy
Communication study
Renuka Mehta

23. Background
• Poor health literacy (HL) is associated with adverse
patient outcomes, poor patient satisfaction and
possible litigation.
• This may be related, in part, to communication
mismatches with providers and the healthcare system.
• Thus we seek to improve upon this gap by developing a
multi-modal, interprofessional communication training
method that can be utilized to enhance providers’
communication skills.

24. P.I.C.O
1. Participants will be health care students (medical, nursing,
Pharmacy and allied health care providers).
2. Online lecture focusing on HL and communication skills,
interaction with the VP software, learning strategies for clear
and sensitive ways to communicate with parents, and
Interprofessional team training simulation workshop where
subjects will communicate with a Standardized Patient (SP)
acting as a parent .
3. Pre and post intervention comparison of appropriate HL
communication between control and intervention groups.
4. Improvement in low health Literacy sensitive communication
between pre and post intervention using standardized parent.

25. Approach
Development
of Multi
stakeholder
case scenario
Interdisciplin Health literacy
ary team communicatio Online lecture
training
n curriculum
Virtual parent
software

26. Questions
1. What is the best way to set up a
interdisciplinary team training
2. What are practical outcome
3. What assessment tools will be helpful.

27. ALERT Presentation:
Using an Epipen educational
module to improve Food
Anaphylaxis Recognition and
Response
D.J. Scherzer

28. Background
• Incorrect outpatient epinephrine-injector usage
leads to preventable mortality among food allergy
children in the U.S.
– The devices are prescribed with inadequate teaching and
follow-up.
• Succinct educational modules improve competence
when directed towards specific goals (eg. AED).
• Competency Gaps
– Incomplete knowledge of indications
– Complexity of a multi-step process
– Lack of confidence, concern for treatment risks

29. PICO Question
• Population
– Parents, patients, school personnel, healthcare staff
– Prescribers
• Intervention
– Succinct educational module comprised of MCQ, practicum and
video
• Comparison
– Historical control – before and after
• Outcome
– Improve ability to know when and how to use epinephrine
injector and to be prepared to actually do it.
– Improve ability to teach others to do above.

30. Approach
• Prospective longitudinal study; learner outcomes
• Subjects serve as their own controls.
• Pre- assessment survey of subjects:
• Experience and confidence questions
• Knowledge and case management questions
• 1:1 practical performance assessment.
• Post- assessment:
• Review practicum and repeat until success with individuals
• Group presentation of edu-video c Q&A.
• Immediate resurvey of individuals with confidence and case
management questions
• Follow-up with confidence question and case
management questions in 3-6 months. Mock scenario on
site?

31. 3 Questions
• Do the case management questions and
practicum get at the active ingredients of food
anaphylaxis competency?
• How can we make the practicum feel more
realistic?
• Can we roll this out in a way that is easier
logistically? More eLearning? Mock scenarios
on site rather than 1:1 practicum.

32. Development and validation of a generalizable tool to
assess pediatric resuscitations
Yasaman Shayan and Arielle Levy
Pediatric Emergency Department
Sainte-Justine Hospital, Montreal, Canada

33. Background
• Resuscitation of a pediatric patient presents many challenges
• Competency in pediatric resuscitation skills gained by:
– Formal training (PALS)  Rapid decline in skills
– Experience  Rarity of pediatric cardiopulmonary arrest
• These points highlight the importance of simulation as an essential
teaching tool
– Essential to have a valid and reliable assessment tool

34. Objectives
• To develop a short, objective, easy to use and
generalizable scoring tool to assess trainees
during simulated pediatric resuscitations
scenarios
– To determine its validity
– To analyse its inter-rater reliability

35. Tool development
Content Validity
Identification
AHA 2010 of specific Review of
objective scoring items
PALS elements for by subject-
curriculum each domain matter experts
(C-A-B)

36. Tool validation
• Convenience sample of residents rotating through ped ED
First week During the rotation Last week
5 video-taped 12-16 ER shifts 5 video-taped
simulated + simulated
resuscitation Simulation-based resuscitation
scenarios courses scenarios
- Asystole/PEA
Scored - Arrhythmias Scored
- Status epilepticus/ asthmaticus
- Shock

37. Discussion
• How to assure content validity
• Type of scoring system
– Checklist vs. GRS
• Inclusion of crisis resource management skills
– Communication, leadership…

38. ALERT presentation:
Code team leader assessment tool
and
correlation of event leader performance with
team performance
Tensing Maa, Ada Lin, Samantha Gee, Aaron Calhoun

39. Background
• Pediatric code blue events are rare = poor
experiential learning for trainees
• Code team leader simulation training may be
helpful.
• Knowledge gap/needs
– Generalizable scoring tool to assess code team
leader competency
– Does code leader performance correlate with
team performance and event outcome?

40. PICO question
• Population
1. Pediatric healthcare providers (APNs) or trainees
(residents, PICU, NICU, EM and anesthesia fellows)
who are expected to act as code team leaders.
2. Pediatric healthcare providers who are potential code
team members.
• Intervention
– Evaluation of event leaders’ and code teams’
performance during standardized simulated pediatric
codes using our scoring tools.
• Comparison
– Performance of event leader with that of the whole
team
• Outcome
– Assess interrater reliability and validity of team leader
scoring tool
– Correlate event leader performance with team
performance and event outcome

41. Approach
Prospective observational pilot study
1. Fine tune content and determine initial psychometric
benchmarks of event leader scoring tool
2. Perform simulations and collect data on event leaders
and team performance
– Scenarios will be standardized in terms of outcome to
assure reproducibility of results
– Separate raters will be used for the teamwork and
leadership tools to minimize potential biases
– Rating will be done based on live and video taped
performance
– Score results will be statistically compared with each other
as well as with the final outcomes of the session to look for
correlations

42. Questions
1. Suggestions on the content (domains or
behavioral anchors) of the team leader rating
tool?
2. Can you accurately separate team leader
performance from team performance when
you have other “experts” (ex. more
experienced fellows) on the team?
3. What about bias from inexperience with or
anxiety from simulation?

43. Hybrid-learning: a model for a comprehensive
curriculum incorporating online self-directed
modules and augmented by high fidelity
patient simulations
Samantha Gee, Ada Lin, and Tensing Maa
Nationwide Children’s Hospital, The Ohio State University
Section of Pediatric Critical Care

44. Background
• “Traditional pathway”
– Didactic learning + patient exposure
• Management of pediatric acute and chronic liver failure,
including the liver transplant patient, is complex:
– Acutely decompensated
– Multiorgan system dysfunction
– Life-threatening sequelae
• Knowledge gap:
– Firsthand experience is limited to liver transplant
centers
– High acuity level requires accurate decision-making
to successfully manage this rare patient population
– Competency and comfort level of fellows trained by
the traditional pathway may not be optimized

45. PICO Question
Population: Pediatric ICU and GI fellow trainees
Intervention: Comprehensive liver failure and
transplantation hybrid-learning curriculum:
◦ Core reading materials for background preparation
◦ Online self-learning modules in didactic form
◦ High fidelity simulation with immediate debriefing
sessions for learning recap and feedback
Comparison: Trainees taught by the traditional pathway
Outcome: Improved mastery of learning and ease of
transfer to practice for those who participate in the hybrid
curriculum as compared to traditional

46. Approach
Design: Prospective pilot study involving ICU and GI fellows
 Revolving curriculum: 6 modules over 18 months
 Online self-directed learning, including core readings and quizzes
 A series of interdisciplinary, comprehensive simulations
 Deliberate practice: Each scenario involves a patient in an acute situation,
providing the fellow an opportunity to:
 Practice critical-thinking skills
 Manage the patient accurately in a safe setting
 Tie-in core concepts attained from online learning
 Evaluation:
 Measure knowledge base pre-/post-participation in the hybrid-learning
curriculum
 Systematically scored based on the six core competencies outlined by the
ABP

47. Questions
How to detect a true effect?
 Number of fellows
 Account for test-taking ability
Is there a more accurate way to assess
performance?
 Knowledge base
 Clinical acumen
 Comfort level

48. ALERT Presentation:
Donation after Cardiac Death (DCD): Improving
consent to donate, compliance with institutional
protocols and organ procurement
Elaine C. Meyer, Ph.D., R.N.
Kristen Nelson, M.D.
Elizabeth Hunt, M.D., Ph.D.

49. Background
• Patients awaiting organ transplantation far exceed
availability of healthy organs
• The Institute of Medicine has advocated for
Donation after Cardiac Death (DCD) protocols in
hospitals to expand the potential donor pool
• Staff knowledge and experience with DCD is limited
and may negatively impact the implementation of
protocols
• A simulation-based educational film is available
(focusing on ethics & family-staff conversations)

50. PICO Question
• Population
– Healthcare and organ procurement providers
• Intervention
– DCD educational film
• Comparison
– Each participating site will be compared to itself pre and post
educational intervention; aggregate comparison across
educational conditions
• Outcome
– Knowledge, attitudes, sense of preparation, confidence,
communicative ability, compliance with established DCD
protocols, use & evaluation of film, frequency of family-OPO staff
meetings to initiate and discuss organ donation, frequency of
consent to donate, frequency of organ procurement

51. Approach
• Survey to all INSPIRE sites to determine
absence/presence of DCD, current educational approach
and educational needs
• Among those with DCD, pre-post comparison after
educational intervention with educational arms (film to
hospital educators, film to OPO educators, partnership
and film to hospital & OPO)
• Among those without DCD, pre-post comparison
between film and no film
• Follow natural use, acceptability and efficacy of film as
per outcome measures

52. 3 Questions
• How many INSPIRE sites have DCD protocols? If no,
what is the likelihood that they would be willing or
able to institute DCD protocols in the near future?
• Outcomes such as frequency of family-OPO
meetings, consent to donate, and organ
procurement require partnership with regional
OPOs. How realistic is this?
• What aspect(s) of this proposal would be possible as
part of INSPIRE simulation centers and is external
funding needed?

53. The Structured Patient Encounter:
Improving the pediatric
patient/family experience using a
structure approach to clinical
interactions
Frank Overly, Linda Brown, Adam
Rojek, Linda Dykstra, Lynn Sweeney

54. Background
• Excellence in healthcare is no longer defined merely by the quality
of clinical care offered, but also by the superiority of service provided
to those who seek care.
• With increasing competition between health care systems,
administrators have increased their focus on service and patient
satisfaction as a barometer of how successful the institution is at
providing a high quality patient experience.
• Low satisfaction scores are significantly associated with malpractice
activity (United States).
• New 5 Step Structured Patient Encounter (SPE)
– Incorporated into a larger communication initiative at Rhode
Island Hospital designed to optimize the patient experience
• Knowledge Gaps
– despite a plethora of communication improvement initiatives,
there remains a paucity of published data objectively evaluating
these programs/interventions

55. PICO Question
• Population
– Residents caring for pediatric patients in the outpatient setting
• Intervention
– Exposure to, or training in the Structured Patient Encounter (SPE)
• Comparison
– compare baseline performance with performance after exposure
to, or training in the Structured Patient Encounter (SPE)
• Outcomes
– Families’ perception of the experience with the care provider
– Families’ perception of the overall clinical experience

56. Approach
• Randomized controlled study
1. Randomize groups into one of 4 study arms
1. Routine education, no exposure to SPE
2. Routine education and SPE cognitive aid
3. Routine education Sim enhanced education on SPE
4. Routine education Sim education and cognitive aid for SPE
2. Baseline scores for all individuals (parents’/patient’s
feedback from clinical encounters, sim observation score)
3. Intervention (cognitive aid, sim, sim + cognitive aid)
4. Follow up scores for all individuals (parents’/patient’s
feedback from clinical encounters, sim observation score)

57. Study Design
Standardized Patient Encounter(SPE)
Baseline measurement of individuals’ performances
1)Actual family/patient feedback
2)SPE Checklist evaluation in simulated case
Group 1 Group 2 Group 3 Group 4
Given sim Given sim
Routine Given SPE enhanced SPE enhanced SPE
Education cognitive aid training no training and
cognitive aid cognitive aid
Follow-up measurement of individuals’ performances
1)Actual family/patient feedback
2)Checklist evaluation in simulated case

58. 3 Questions
• Is it necessary to gather information in the
simulated setting?
• What confounding issues might we encounter?
How to overcome them? (ED issues: waiting
times, nursing issues, residents will have normal
progression of skills, acuity level of patient)
• How many encounters would you need to record
for each individual?