1. Evaluating Innovation:
Moving Toward the New Paradigm
of Care Delivery
Jeffrey M. Adams, PhD, RN
Director, Center for Innovations in Care Delivery
Connell Nursing Research Scholar
October 2013
1

2. EVALUATION STEERING COMMITTEE
 Gaurdia Banister
 Annie Kingsley
 Marianne Ditomassi
 Linda Lacke
 Rick Evans
 Bret O’Flaherty
 Brian French
 Robin Lipkis-Orlando
 Amy Giuliano
 Colleen Snydeman
 Dorothy Jones
 Jeff Adams
2

3. Innovation is…?????
Haircuts
Soda
Airlines
Garden Utensils
Hardwood Flooring
Automobiles
Laundry Detergent
Source: Adams JM. (in press) Guest Editorial: How do we know if we’re innovating? A strategy for
innovation evaluation in a practice setting. Journal of Nursing Administration.
3

4. “History of Healthcare Innovating”
•
4
Add photos of Healthcare
Innovation Products

5. A New Context for Innovation
•
5
CMS

6. “Patient Journey” Framework
Before
Preadmission
Care
During
Admission
Process: ED,
Direct Admits,
Transfers
Patient Stay;
Direct Patient Care, Tests,
Treatments, Procedures,
Clinical Support,
Operational Support
Post
Discharge
Process
Support Functions: Finance, Information Systems, HR
Goal: High-performing interdisciplinary teams that deliver safe, effective, timely,
efficient and equitable care that is patient and family centered.
Where Are There Opportunities to Reduce Costs Across These Processes of Care?
6
Post
Discharge
Care

7. INNOVATION UNIT EXPERIENCES SURVEY
“The MGH Innovation Units Program was developed as a frontline
initiative to address the gaps in the continuum of care and utilize the
expertise of all clinical disciplines to improve the patient experience,
quality of care, staff satisfaction and empowerment while decreasing
redundancy, costs, length of stay and readmission rates.”
Source: Ives Erickson J, Ditomassi M, Adams JM. (2012) Innovations in Care Delivery: A Blueprint for
the Future. Nursing Economic$ 30(5): 282-287.
7

8. MGH PCS’s Approach
Innovation Cluster
Focus Areas *
Interventions **
Evaluation
(Pre, During, Post)
Throughout Admission
Relationship-Based Care
Attending Nurse
Handover Rounding Checklist
Patient Engagement
Quantitative
•HCAHPS
Pre-Admission
•Leadership Influence
over Professional
Practice Environments
(LIPPES)
Pre-Admit Data Collection
Welcome Packet
During Admission
Roles & Structures
Education
Communication
Domains of Practice
Interdisciplinary Rounds
Business Cards
Quiet Hours
Hourly Rounding
Electronic White Boards
In Room White Boards
Smart Phones
Hand Held/ Tablets
Post-Discharge
Discharge Follow-up Phone Calls
Others as identified
•Quality Indicators
•Patients Perceptions
of Feeling Known
(PPFKN)
•Readmissions
•Focus Groups
(Staff, Patients,
Families, etc)
•Observations
•Survey of the
Innovation Unit
Expectations
(SIUE-pre)
•Survey of the
Innovation Unit
Experiences
(SIUE-post)
•Revised Perceptions
of Practice
Environment Scale
(RPPE)
•Cost per Case Mix
* The clusters are a lens
with which we gain
perspective on any
particular intervention.
8
•LOS
Qualitative
•Staff Retention
Other measures as identified
** May apply to any or all 3 of the cluster focus areas
June 2013

9. It is in there
9

10. Data, Data, Everywhere
6
10

11. Why is this Innovation Unit Evaluation complex?
•
Aimed at understanding successes & challenges of structures
processes and outcomes of care delivery AND the evaluation process
itself.
•
Telling the story in context of the immediate, while tracking
longitudinally to optimize sustainability.
•
All while the ship is sailing…
•
Different than research
11

12. The Process
Innovation Evaluation (90-Day Cycle)
Day 1-30
Day 31-60
Day 61-90
Source: Institute for Healthcare Improvement (2013)
12

13. So what are we finding?
•
Three examples of how and what we’re finding…
– Idea Books
– Innovation Unit Experiences Survey
– Innovation Unit Interventions Sustainability Checklist
13

14. Idea Books
14

15. INNOVATION UNIT EXPERIENCES SURVEY
What is it like
to work on an innovation unit?
15

16. Analysis of the data yielded five interrelated themes:
1. Feeling prepared: “There was ample support and pre-education.”
2. Innovation: “A complete change of culture.”
3. Managing Challenges and Concerns: “It has been very challenging.”
4. The Attending RN: “The glue for nursing care.”
5. Benefits to patients and families: “The impact on patient care is worth it all.”
16

17. INNOVATION UNIT EXPERIENCES SURVEY
Are the efforts being sustained?
17

18. Innovation Unit Observation
Checklist Phase 1 & 2
18

19. PATIENT/ FAMILY INTERVIEW
Do you have an Attending Nurse?
Phase I Only
Phase II Only
Phase I & II
Yes
59%
57%
58%
No
41%
43%
42%
If YES, was patient/family able to provide a name?
Yes
85%
57%
66%
No
15%
43%
34%
Did you receive an ARN Business Card?
Yes
36%
47%
No
64%
53%
57%
Do the nurses, doctors, and other staff make you feel like you
are part of the team?
Yes
100%
94%
96%
No
0%
6%
4%
Do you have a Discharge Envelope/Checklist?
Yes
47%
49%
48%
No
53%
51%
52%
If YES, is there a discharge date noted in the space provided?
Yes
0%
21%
14%
No
100%
79%
86%
Do you have a copy of the Patient & Family Notebook?
Yes
55%
53%
54%
No
45%
47%
46%
If YES, has it been helpful to you?
Yes
27%
73%
58%
*
No
73%
27%
42%
*
*
43%
*
*

20. Phase I Only
PATIENT/ FAMILY INTERVIEW
Phase II Only
Phase I & II
When someone comes in to check on you: Do they ask
about your pain?
Yes
96%
92%
93%
No
4%
8%
7%
When someone comes in to check on you: Do they ask you
whether you need to go to the bathroom?
Yes
54%
58%
57%
No
46%
42%
43%
When someone comes in to check on you: Do they ask you if
you need to change position and if you are comfortable?
Yes
71%
84%
80%
No
29%
16%
20%
When someone comes in to check on you: Do they ask you if
you need anything else before they leave the room?
Yes
96%
96%
96%
No
4%
4%
4%
Has your call bell been answered promptly?
Yes
92%
86%
88%
No
8%
14%
12%
*
*
*
*

21. Phase I Only
STAFF INTERVIEW OR OBSERVATION
Phase II Only
Phase I & II
Can you tell me the purpose of the Attending Nurse role?
Yes
100%
100%
100%
No
0%
0%
0%
Can you share an example of Relationship-based Care
that has occurred on the unit?
Yes
96%
90%
92%
No
4%
10%
8%
Do Interdisciplinary Rounds occur at a regular scheduled
time on your unit?
Yes
83%
98%
93%
No
17%
2%
7%
The Electronic White Board is utilized on the unit.
(Phase I and Lunder only).
Yes
92%
93%
92%
No
8%
7%
8%
Voaltés are utilized on the unit.
Yes
100%
100%
100%
No
0%
0%
0%
Hourly Rounding (Safety Rounds) occur on the unit.
Yes
100%
90%
93%
No
0%
10%
7%
*
*
*
*
*
*

22. Phase I
Only
OBSERVATION - IN-ROOM WHITE BOARD
Phase II
Only
Phase
I & II
100%
89%
93%
Patient Name
Complete & Accurate
Complete / Not Accurate
0%
0%
0%
Not Complete
0%
11%
7%
Date
Complete & Accurate
92%
79%
83%
Complete / Not Accurate
4%
19%
14%
Not Complete
4%
2%
3%
Nurse Name
Complete & Accurate
92%
82%
85%
Complete / Not Accurate
4%
16%
12%
Not Complete
4%
2%
3%
Physician Name
Complete & Accurate
65%
30%
43%
Complete / Not Accurate
4%
2%
3%
Not Complete
31%
68%
54%
Other Members of the Care Team (s)
Complete & Accurate
50%
76%
68%
Complete / Not Accurate
0%
0%
0%
Not Complete
50%
24%
32%
Estimated Discharge Date
Complete & Accurate
17%
13%
14%
Complete / Not Accurate
0%
2%
2%
Not Complete
83%
85%
84%
Goals for the Day
Complete & Accurate
68%
48%
54%
Complete / Not Accurate
0%
2%
2%
Not Complete
32%
50%
44%
*
*
*
*
*
*
*

23. OBSERVATION - PATIENT FACE SHEET
Phase I Only
Phase II Only
Phase I & II
Pre-Admission Estimated Length of Stay was identified
Yes
20%
26%
24%
No
80%
74%
76%
Estimated Discharge Date was identified
Yes
15%
0%
4% err
No
85%
100%
96%
Discharge Disposition was identified
Yes
9%
0%
3%
No
91%
100%
*
97%
OBSERVATION/STAFF REPORT –
QUIET HOURS
Phase I Only
Phase II Only
*
*
Phase I & II
Do Quiet Hours occur on the unit?
Yes
97%
92%
100%
No
3%
8%
0%
Quiet Hours signage is visible on the unit?
Yes
89%
92%
88%
No
11%
8%
12%
*
*

24. Positive:
Patient
 Do nurse, doctors and others make you feel part of the team?
 When someone comes in to check on you do they ask about – your pain?
 When someone comes in to check on you do they ask if your are
comfortable and want to change position?
 When someone comes in to check on you do they ask if you need anything
else before they leave the room?
 Has your call bell been answered promptly?
Staff
 Can you tell me the purpose of ARN role?
 Can you share an example of Relationship Based Care?
 Do interdisciplinary rounds occur at a regularly scheduled time?
Observation
 Electronic White Boards utilized on unit (Phase 1 and Lunder only)?
 Voaltés are utilized on the unit?
 In Room White Board – Patient Name Complete?
 Quiet Hours signage is visible?
24
Observed or Reported
 Quiet Hours occur on the unit?
 Hourly Rounding occurs on the unit?

25. Opportunities: Observation
 D/C Envelope – if yes is a D/C date noted?
 In Room White Board – Physician Name?
 In Room White Board – Care Team Members Names?
 In Room White Board – Estimated Discharge Date?
 In Room White Board – Goals for the Day?
 Face Sheet – Pre-admit expected LOS?
 Face Sheet – EDD?
 Face Sheet – Discharge disposition?
Aggregate data for Phase 1 and Phase 2
25

26. What is our evaluation telling us?
26

27. Next Steps for Evaluation
Continued evaluation in the 90-Day Cycle
Inclusion of Phase 3 and Phase 4 Units
Adjustment of measures based on Evaluation Steering Committee
Recommendations (i.e., deep dive: Estimated D/C Date, White Boards, Face Sheet)
Increased evaluation of ARN role
Increased transdisciplinary team input (regular meetings/groups soliciting insights)
Coordinate longitudinal data management (research)
Developing an Innovation Evaluation Manual
Publish, present, publish, present, repeat
27

28. MASSACHUSETTS GENERAL HOSPITAL
PATIENT CARE SERVICES
Including
Mechanisms of Evaluations
Data Elements
Instruments
Questions
Timelines
Dashboard examples
Observation Checklist
Number of Resources Needed
28

29. To identify the new paradigm
• Evaluate trends
• Build for research
• Utilize evidence to make
decisions
• Adopt, Adapt & Abandon
• Disseminate work
29

30. Nursing Director Survey Quote:
“Managing the new [innovation] processes is
time consuming. Communication to other staff
is ongoing. I feel I’ve answered the question,
‘What is this all about?’ too many times.
But…the impact on patient care is worth it all.”
30