1. Create Your Own
Simulations and
Evaluate Them
T EA M STEP PS® N AT I ONAL CON F E R ENCE
DA L L A S, T X
JUN E 1 3 , 2 0 1 3

2. Introductions
o Steve Powell, MS
o Melisa Powell, MS, RN-BC
o Julie Gapstur, RN, BSN
o Daniel Baily, MS

3. Click Image to Follow Link to Resource

4. Objectives
o To know and be able to develop TeamSTEPPS training scenarios
o To understand the process of developing TeamSTEPPS performance
measures for Simulation
o Identify the role debriefs play in promoting team self-correction,
process improvement, and patient safety.
o Recognize the importance of reflection, measurable performance
objectives, and psychological safety during the debrief.
o Recognize the importance of empowering individual team members to
discuss shared team performance and process outcomes during the
debrief.

5. Program Evaluation
of In Situ
Simulation-Based
Team Training
MELISSA POWELL MS, RN -BC

6. Start at the End - Use a Backwards
Planning Model
o Start with a needs assessment. Identify best practices.
o Determine what will be acceptable evidence of current performance.
o Examples:
◦ Practice data - Unit data
◦ Published data about clinical performance
◦ Interviews – Focus groups – Unit request

7. My Story
o Vanderbilt Hospital has a weekly meeting with unit leaders to debrief
all RRT calls that occur on medicine units.
◦ Began to Review all RRT’s called on unit
◦ Appeared to be delays in calling
◦ Nurse were not calling RRT (even when medically appropriate)

8. What Did the Nurses Say?
o
o
o
“He or she {physician} is right here at the bedside with
me, and that feels very uncomfortable as though it says
to the physician that I do not think they are capable.”
“He or she went to medical school. I didn’t. I’m just a
nervous new nurse. They know what they are doing.”
“The nurses just roll their eyes at me and don’t do
anything!”

9. Desired Outcomes for the Project
Rapid Response called:
1. When Appropriate - Accuracy
2. In a Timely Manner - Timeliness

10. What I Did Use as a Measure
How frequently was there a delay to call or failure to call

11. Using Simulation and Video as a Tool for
Evaluating Behaviors

12. Using a Checklist as an Evaluation Tool

13. Frequency Count
Communication
Positive Instances
Check-back
Call-outs
SBAR
Unintelligible Communications
Mutual Support
Two-Challenge
CUS
Task Assistance
Positive Instances

14. Using Debriefing as an Evaluation Tool
o The Facilitator should role model creating a debrief climate with
psychological safety.
o Learners should be asked to express what they will take forward into
their daily practice based on the simulation experience.
o (EXIT TICKET!)

15. Play Video of Debrief

16. Program Evaluation
Post code data
& # of Failure to
Rescue events
Survey
Real event
Intervention
group (100%)
Debrief
Post code
quality survey
Online
Education
Simulation
# Failure to
Rescue events
Source: Moore’s Method

17. Post Training Survey
Beacuase of this simulation event I have learned new communication
techniques that I will employ in my practice.
7%
20%
73%
Strongly Disagree
Please rate the simulation as a method to practice/reinforce your
teamwork skills.
7%
Strongly Agree
14%
79%
Poor
Unit based simulations helped build my confidence in my teammates
for a real life code situation.
7%
0%
Bottom Quartile
Excellent
3rd Quartile
93%
Strongly Disagree
10%
20%
30%
2nd Quartile
40%
50%
Top Quartile
60%
70%
Strongly Agree
80%
90% 100%

18. Simulation Checklist Results
o BLS skills (80%)
o Procedural knowledge (50%)
o Equipment knowledge (unable to assess during due to SimMan3G barriers)
o Assessment and Intervention (99%)
o Recognizing and Initiating RRT immediately (55%)
o Communication skills
◦ SBAR (20%)
◦ CUS (.1%)
◦ Check Back (34%)
◦ Two-Challenge Rule (0%)
◦ Situation Monitoring including Shared Mental Model knowledge (34%)
.

19. Number of Delay and/or Failure to Rescue
18
Control Unit
12
23
Invervention Unit
6
0
5
10
January 2010 to July 2010
15
20
25
January 2011 to July 2011
January 2010 to July 2010
January 2011 to July 2011
Percent Change
Invervention Unit
23
6
-117%
Control Unit
18
12
-40%

20. Performance Outcomes Analysis
o Debriefing nurses post intervention yield stronger self reflective ability
o Using concepts such as authority gradient to communicate what they
experienced
o Staff now often identifies their own barriers and role in early
activation and failure to rescue

21. Role-play Simulation Exercise

22. TeamSTEPPS,
Simulation and the
Power of Debriefs
A POWERFUL COMBINATION
JULIE GAPSTUR, RN, BSN

23. Partnership for Patients Alignment
o Aims:
◦ To reduce patient harm by 40 percent by 2013
◦ To reduce hospital readmissions by 20 percent by 2013
Harm includes OB Adverse Events

24. Our Collaborative Goals
o 95% providers and staff participation in a team-based simulation
module
o Increase Labor and Delivery team debriefs following an unscheduled
cesarean delivery, after handoff to OR, to 95%
o 5% overall improvement in dimension scores or achievement of 5% or
more above the National OB benchmark average on the AHRQ
Hospital Survey on Patient Safety Culture
o + other clinical outcomes

25. Why TeamSTEPPS and Simulation?

26. Simulation and Team Training
Teams experience the difference
between knowing what they are
supposed to do & doing what they
are supposed to know
Courtesy: Paul Preston, MD Kaiser-Permanente

27. What we Did…
Simulation Training
that is focused
on TeamSTEPPS &
DEBRIEFING

28. Why Debrief?
o Teams that debrief effectively
build shared understanding
(mental models) and perform
up to 40% better
o Guided team debriefs improve
performance through selfcritique and problem-solving
o Teams that debrief are able to
uncover and identify problems
earlier than other teams
Smith-Jentsch, et al., 2008
28

29. How we did it….
TeamSTEPPS Essentials Training
Training on HOW TO DEBRIEF
Use multiple levels of Simulation Fidelity
Familiarization with Simulation devices
Faculty Simulation exercise
Participant Simulation exercises
Simulation evaluation

30. Building the Scenario

31. “Show and Tell”

32. The Simulation Event

33. Mama Natalie - Low Fidelity

34. Pre-Simulation Briefing
1. Introduce Staff
2. Review Roles
Pre-Simulation
Briefing by Facilitator
3. Review Equipment
4. Review Goals

35. Shoulder Dystocia –
High Fidelity with SimMom™

36. What is a Debrief?
Debriefs are team events used to:
◦ Review individual and team
performance
◦ Identify errors made
◦ Recognize best practices
◦ Develop a plan to improve
◦ Promote continuous learning and
process improvement
DEBRIEFS ARE NOT JUST
FOR SIMULATION!
36

37. Structured Debrief Checklist
1.
What went well, and why?
2.
What didn’t go so well, and why?
3.
What would we do differently?
FOCUS ON TEAM SKILLS!
Remember, redirect debrief if the focus is on processes, ‘simisms’, or clinical skills

38. Tips for Debriefing
Be a FACILITATOR, not an INSTRUCTOR –
Get Them To Talk
Keep it Simple – Three Questions
Show the Video –
When Needed (if team is unable to self reflect.)
Be Comfortable with Silence
Focus on Team Skills –
NOT Clinical Skills – Redirect the Debrief if Needed.

39. Application
What will it take to implement debriefs with your team?
◦ Identify a facilitator.
◦ Give all team members a voice in the process.
◦ Treat debriefs as learning opportunities, not opportunities for
assigning blame.
◦ Focus on process improvement and future performance.

40. Teamwork Actions
o Conduct debriefs.
o Empower team members to
speak freely and provide
feedback.
o Focus on teamwork and task
work processes.
o Use a tool to evaluate team
performance and facilitate
discussion.
o Record and trend team
performance.

41. Simulation Learnings – ‘Ah-Ha’ Moments
1. Tell them, show them, let them practice
2. Make sure your facilitators are not only content experts,
but well seasoned facilitators
3. Stick to the script in debriefing
4. Plan in advance – but be flexible
5. Make it fun! Don’t be surprised if you see laughing – this
is “serious play!”
6. Don’t be afraid of improvisation

42. Implementation Confidence –
Training Site #4
N = 22

43. Simulation Confidence – Training Site #2
N = 16

44. TEAMS THAT DEBRIEF –
PERFORM BETTTER!
Looking through the Windshield instead of the Rear View Mirror

45. Helps turn your team of experts…
into an expert team

46. Q&A

47. References
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
Bandura. (2001). Social cognitive theory: An agentic perspective. Annual Review of Psychology, 52(1), 1.
Clancy, C. M. (2007). TeamSTEPPS: assuring optimal teamwork in clinical settings. American journal of medical quality, 22(3), 214.
Gaba, D. M. (2004). The future vision of simulation in health care. Quality & Safety in Health Care, 13(suppl 1), i2.
Holzman, R. S. (1995). Anesthesia crisis resource management: real-life simulation training in operating room crises. Journal of clinical
anesthesia, 7(8), 675.
Kolb, D. A. (1983). Experiential Learning: Experience as the Source of Learning and Development (1st ed.). Prentice Hall.
Miller, K. K. (2008). In situ simulation: a method of experiential learning to promote safety and team behavior. The Journal of perinatal
& neonatal nursing, 22(2), 105.
Moore, D. E., Green, J. S., & Gallis, H. A. (2009). Achieving desired results and improved outcomes: Integrating planning and
assessment throughout learning activities. Journal of Continuing Education in the Health Professions, 29(1), 1–15.
doi:10.1002/chp.20001
Nunnink, L. (2009). In situ simulation-based team training for post-cardiac surgical emergency chest reopen in the intensive care unit.
Anaesthesia and intensive care, 37(1), 74–8.
Peberdy, M. A., Callaway, C. W., Neumar, R. W., Geocadin, R. G., Zimmerman, J. L., Donnino, M., Gabrielli, A., et al. (2010). Part 9: Post–
Cardiac Arrest Care 2010 American Heart Association Guidelines for Cardiopulmonary Resuscitation and Emergency Cardiovascular
Care. Circulation, 122(18 suppl 3), S768–S786. doi:10.1161/CIRCULATIONAHA.110.971002
Rogers, E. M. (1995). Diffusion of innovations. Free Press.
Rudolph, Jenny, W. (2006). There’s No Such Thing as “Non-judgmental” Debriefing: A Theory and Method for Debriefing with Good
Judgement. Simulation in Healthcare, 49–55.
Salas, E., DiazGranados, D., Klein, C., Burke, C. S., Stagl, K. C., Goodwin, G. F., & Halpin, S. M. (2008). Does Team Training Improve Team
Performance? A Meta-Analysis. Human Factors: The Journal of the Human Factors and Ergonomics Society, 50(6), 903–933.
doi:10.1518/001872008X375009
Small, S. D. (2008). Demonstration of high‐fidelity simulation team training for emergency medicine. Academic emergency medicine,
6(4), 312.
Vanderbilt Policy Rapid Response Team Activation Policy number CL 30-08.16
Vanderbilt Policy Cardiopulmonary Resuscitation Policy number CL 30-08.21.