CER 2016 Srivastava

1. Specialty and Generalist
Collaboration:
Multidisciplinary Teams
Raj Srivastava, MD, FRCP(C), MPH
Assistant Vice President for Research, Intermountain Healthcare
Professor of Pediatrics, University of Utah School of Medicine
Chair, Executive Council, Pediatric Research in Inpatient Settings (PRIS) Network
UCSF Symposium on Comparative Effectiveness Research
San Francisco, CA
February 2, 2016

2. Perform comparative
effectiveness research
aimed at defining best
practices
Implement best
practices and measure
patient/cost outcomes
Disseminate results to
healthcare institutions
Core Principles
Pediatric Research in Inpatient Settings (PRIS)
• PRIS is an independent hospitalist research network founded
through a collaborative effort of three organizations: the
Academic Pediatric Association (APA), the American Academy of
Pediatrics (AAP), and the Society for Hospital Medicine (SHM)
• >800 hospitalists from 100 centers

3. PRIS Mission
Improve the health of and healthcare delivery to
hospitalized children and their families

4. Organizational Structure
Advisory Board
Intermountain Healthcare – Brent James
IHI – Don Berwick
NICHQ – Charlie Homer
Other Research/Network/Pediatric Leaders
Ex-Officio Organizations
CHA – Matt Hall
APA – Mark Schuster
AAP – John Klein
SHM – Andrew Auerbach
Network
Coordinator
Betsy Holm
PRIS Members
PRIS Executive Council
Rajendu Srivastava, MD, MPH, Chair
Chistopher P. Landrigan, MC, MPH, Past Chair
Patrick Conway, MD, MSc
Ron Keren, MD, MPH
Sanjay Mahant, MD, MSc
Samir S. Shah, MD, MSCE
Jay Berry, MD MPH
Karen Wilson, MD, MPH
Theokils Zaoutis, MD, MSCE

5. PRIS Membership

6. 2009
2010
2011
2012
2013
2014
First annual meeting Executive Council
Salt Lake City
• APA, AAP and SHM
Prioritization Project funded by CEOs
• CHA $1.4 million over 3 years
PI: R Srivastava, involves 7 PRIS sites
PHIS+ study funded with ARRA (almost)
• PHIS+ R01 - $9 million over 3 years
PI: R Keren, involves 6 PRIS sites
I-PASS study funded with ARRA
• I-PASS R01 - $3 million over 3 years,
PI: C Landrigan, involves 10 PRIS sites
February 2009 November 2014PRIS Activities

7. What’s Collected on Each Patient Encounter in PHIS
Patient
Abstract
Diagnoses
(ICD-9)
Procedures
(ICD-9)
Patient
Abstract and
ICD-9 Coding
Billed Transaction/
Utilization Data
(all items/services billed to the pt)
Pharmacy
Imaging/
Radiology
Lab
Clinical
Supplies
Other
* Room/Nursing
* Surgical Svcs
* Other misc
Patient
Encounter
Hospital ID Disposition
Patient ID APR-DRG
Dates/LOS MS-DRG
Age, Bw, Gest Age Key Physicians
Principal Diagnosis Payer
Principal Procedure

8. • 39 CEOs
received their
hospital-specific
utilization
reports
• Goal was to
align clinical
leadership with
hospital
administration

9. Standardizing Unit Costs
• Median cost for CBC = $32
• 2 CBC’s: cost = 2 x $32

12. 2009
2010
2011
2012
PIVVOT study funded by PCORI
• $2 million over 3 years
PI: R Keren, involves 40 PRIS sites
GAPPS study funded by CHIPRA grant
• U01 over 5 years
PI: M Shuster, involves 15 PRIS sites2013
2014
February 2009 November 2014PRIS Activities

13. The Pediatric IntraVenous Vs. Oral antibiotic Therapy
(PIVVOT) Study
Ron Keren, MD, MPH
Professor of Pediatrics and Epidemiology
Perelman School of Medicine at the University of Pennsylvania
Vice President of Quality
The Children’s Hospital of Philadelphia

14. Imagine

15. Background
• Some serious bacterial infections (e.g. complicated
pneumonia, perforated appendicitis, osteomyelitis)
require prolonged home antibiotic therapy
• After inpatient improvement with IV antibiotics, choice is
between outpatient parenteral therapy via PICC line or
oral antibiotics
• Scarce evidence showing which treatment option is more
effective

16. 2007
Pediatrics 2009;123:636–642

17. PHIS data from 2000-2005

18. 2012
PHIS data from 2009-2011

19. Why not much change?
• No dissemination and implementation plan
• Study limitations
– Administrative data only
– Questions about ascertainment of osteo diagnosis, exposure,
outcome
– Residual confounding
– Rise of CA-MRSA

20. JAMA Pediatrics 2015 Feb;169(2):120-8.

21. Partnership
• Pediatric Research in Inpatient Settings (PRIS) Network
• Children’s Hospital Association (CHA) and its member
hospitals

22. Personnel
• PI: Ron Keren (Children’s Hospital of Philadelphia, PRIS EC)
• Site PIs:
– Raj Srivastava (University of Utah, PRIS EC Chair)
– Shawn Rangel (Children’s Hospital Boston)
– Samir Shah (Cincinnati Children’s Hospital Medical Center, PRIS EC)
– Matt Hall (Children’s Hospital Association)
• Biostatisticians
– Russell Localio (Children’s Hospital of Philadelphia)
– Xianqun Luan (Children’s Hospital of Philadelphia)
• Other personnel:
– Study coordinators: Rachel deBerardinis and Allison Parker (Children’s Hospital of
Philadelphia)
– Family advocates: Kathryn Conaboy and Darlene Barkman (Children’s Hospital of
Philadelphia)
– PRIS Network Manager: Jaime Blank (University of Utah)

23. Study Aim
• Specific Aim #1: To compare the effectiveness of oral
antibiotics vs. intravenous antibiotics delivered via a PICC
line in children who require prolonged home antibiotic
therapy after hospitalization for complicated pneumonia,
perforated appendicitis, or osteomyelitis
• Specific Aim #2: To compare patient and caregiver reported
quality of life and adherence to therapy for oral antibiotics
vs. IV antibiotics delivered via a PICC in children who require
prolonged home antibiotic therapy after hospitalization for a
serious bacterial infection.

24. Methods
• Retrospective cohort study
• Children hospitalized from January 1, 2009, through
December 31, 2012, at 36 participating children’s hospitals

25. PCORI CER Proposal
• Chart review to confirm diagnosis, exposure, outcomes
• Within and across hospital propensity score-based full
matching
• Stakeholder engagement

26. Treatment Failure
Defined as revisit to the ED or a rehospitalization for:
– change in the antibiotic prescribed or its dosage
– prolongation of antibiotic therapy
– conversion from the oral to the PICC route
– bone abscess drainage
– debridement of necrotic bone
– bone biopsy
– drainage of an abscess of the skin or muscle
– arthrocentesis
– diagnosis of a pathologic fracture

27. Site involvement
• Approximately 5 minutes per chart
• Average of 200 charts per site (range: 50-600)
• Average of 17 hours (range: 4-50)
• Can be completed by:
– Yourself
– Research assistant
– Nurse
– Other trained staff member
• Site compensation:
– Grant money allocated for chart review
– Payment according to number of charts/hours worked
– Paid through a purchase-service agreement

28. Purchase Service Agreement
• Simplest arrangement– fee for service
• No indirects (F&A) costs
• Site submits an invoice
• CHOP approves it and mails a check

29. ! "#$%&' $()*+&' %, "+- . /), "' /. /)0' %1)%+&$2$- &$()34"' %56)7! *, , 038)/&. #6)
!" #$%&' ( ) %
%
* +, -%. /0012%34" 5$%364+%#+, 537%
%%
8 +%, -+%49-6##+5%4: %9, ; +%$: " %<: 6=%" 3%, 3%, %>, -4%: ?%49+%. /0012%34" 5$%4+, @%, =5%, -+%+AB64+5%4: %
C+4%34, -4+5%: =%4963%>-: <+B4D%/=%: -5+-%4: %@, E+%49+%: =F: , -56=C%>-: B+33%, 3%+, 3$%, 3%>: 336F#+7%
G +H; +%B-+, 4+5%, %4: : #E64%?: -%364+3%G 649%6=34-" B46: =3%: =%=+A4%34+>3%, =5%: 49+-%6@>: -4, =4%
6=?: -@, 46: =%, F: " 4%$: " -%364+H3%-: #+%6=%49+%. /0012%34" 5$D%
%
I 3%?, -%, 3%=+A4%34+>3%, -+%B: =B+-=+5%?: -%$: " -%364+7%49+%?6-34%: -5+-3%: ?%F" 36=+33%, -+%4: %B: @>#+4+%
49+%/J K%, >>#6B, 46: =%>-: B+33%, =5%4: %3+4%" >%, %. " -B9, 3+%L+-; 6B+%I C-++@+=4%F+4G ++=%$: " -%364+%
, =5%49+%M96#5-+=H3%N: 3>64, #%: ?%. 96#, 5+#>96, %OMN1. PD%%
%
8 649%-+3>+B4%4: %49+%/J K%, >>#6B, 46: =7%G +%=++5%$: " %4: %4+##%" 3%6?%$: " %, -+%>#, ==6=C%4: %" 3+%MN1. %
, 3%49+%/J K%: ?%-+B: -5%: -%6?%$: " -%6=3464" 46: =%-+Q" 6-+3%49, 4%$: " %3" F@64%, %3+>, -, 4+%/J K%
, >>#6B, 46: =D%%8 +%G 6##%F+%-+, B96=C%: " 4%4: %$: " %6=%49+%=+A4%G ++E%4: %?6=5%: " 4%G 96B9%: ?%49+3+%
: >46: =3%$: " %G 6##%F+%>" -3" 6=CD%
%
/=%4+-@3%: ?%49+%. " -B9, 3+%L+-; 6B+%I C-++@+=47%G +%, -+%6=%49+%>-: B+33%: ?%5-, ?46=C%49: 3+%
B: =4-, B43%, =5%G +%G 6##%39, -+%49+@%G 649%, 3%3: : =%, 3%49+$%, -+%B: @>#+4+D%
%
I 3%, %-+@6=5+-%?: -%49+%46@+#6=+%: ?%4963%>-: <+B47%G +%9: >+%4: %9, ; +%49+%/J K%, =5%. " -B9, 3+%
L+-; 6B+%I C-++@+=43%6=%>#, B+%F$%L+>4+@F+-%, =5%G 6##%F+%B: =5" B46=C%4-, 6=6=C%G +F6=, -3%: =%
B9, -4%-+; 6+G %5, 4, %B: ##+B46: =%49-: " C9: " 4%L+>4+@F+-D%8 +%G 6##%49+=%, 3E%+, B9%364+%4: %B: =5" B4%
49+6-%B9, -4%-+; 6+G 3%F+4G ++=%1B4: F+-%, =5%49+%+=5%: ?%* +B+@F+-D%
%
! "#$%&'() *+, +( -. /012( 3. 4012( 5%6012( 7 &'012( 8 #9012( : %&012(
!! !! !! !! !! !! !!
"#$%&' () *() $+,%) !' - $) ) . ) /0(! !! !! !! !! !! !!
123! !! !! !! !! !! !!
45/6#%0!0$' ,/,/- !7 ) 8,/' $(! !! !! !! !! !! !!
9,0) (!%5/6#%0!6' 0' !) /0$: ! !! !! !! !! !! !!
%
K+#: G %63%, %#634%: ?%49+%6=?: -@, 46: =%6=B#" 5+5%6=%4963%4: : #E64R%
· M: =4, B4%6=?: -@, 46: =%
· . -: <+B4%6=?: -@, 46: =R%
o . /0012%34" 5$%>-: 4: B: #%
o SAB+#%3>-+, 539++4%G 649%, >>-: A6@, 4+%=" @F+-%: ?%B9, -4%-+; 6+G 3%?: -%+, B9%364+%
O?6=, #%=" @F+-%G 6##%#6E+#$%F+%3@, ##+-P%
o . : G +->: 6=4%3#65+3%?-: @%49+%-+B-" 64@+=4%G +F6=, -%
· /J K%6=?: -@, 46: =R%
o /J K%T-+Q" +=4#$%I 3E+5%U" +346: =3%
o V+44+-%: ?%, >>-: ; , #%?-: @%49+%MN1. %/J K%
o /J K%: ?%-+B: -5%?: -@%
o M: @>#+4+%/J K%, >>#6B, 46: =%?-: @%MN1. %
· . " -B9, 3+%L+-; 6B+%I C-++@+=4%6=?: -@, 46: =R%

30. IRB
• Sites will have the option of having CHOP serve as the IRB
of record or submitting an IRB at their own institution
• In order for CHOP to serve as the IRB of record:
– Sites will need to fill out a one-page form signed by an IRB official
at their institution and send it to CHOP
• Some site’s IRBs may require their own IRB submission–
we can provide you a complete protocol.

31. Institutional Review Board, Committees for the Protection of Human Subjects
IRB AUTHORIZATION AGREEMENT
CHOP IRB Authorization Agreement page 1 November 29, 2011
This agreement allows The Children’s Hospital of Philadelphia IRB (CHOP IRB) to act as the IRB of record for
another FWA Institution.
Name of Designated Institution Providing IRB Review
Committees for the Protection of Human Subjects (CHOP IRB)
The Children’s Hospital of Philadelphia
Assurance (FWA): FWA00000459
Name of Institution Relying on the Designated IRB
Name of Relying
Institution:
Assurance (FWA):
The Officials signing below agree that may designate and rely on the CHOP IRB for review and continuing
oversight of its human subjects research described below.
This agreement is limited to the following specific protocol(s):
IRB Number: 13-010086
Title of Study: Pediatric IntraVenous Versus Oral antibiotic Therapy
Principal Investigator: Ron Keren
Sponsor or Funding Agency: Patient-Centered Outcomes Research Institute (PCORI)
By signing this agreement, both institutions have agreed that the CHOP IRB will serve as the IRB of record and
are agreeing to uphold their individual responsibilities as listed on page 2 of this document. The IRB at CHOP will
follow written procedures for reporting its findings and actions to appropriate officials at the relying institution.
Relevant minutes of IRB meetings will be made available upon request. The relying institution remains
responsible for ensuring compliance with the CHOP IRB’s determinations and with the Terms of its OHRP-
approved FWA. This document must be kept on file by both parties and provided to OHRP upon request.
Signature of Signatory Official at CHOP:
Signature Date
Print Full Name Institutional Title
Signature of Signatory Official at the Relying Institution:
Signature Date
Print Full Name Institutional Title

36. Record Status Dashboard

37. Chart Reviews
• Chart reviews conducted October 1, 2013 through
December 31, 2013
• Training will be provided (webinar)
• Number of charts depends on your patient volume
• Data coordination at CHOP
• REDCap database for data entry (web-based)

38. 0
200
400
600
800
Not Reviewed Complete
1001
1003
1005
1007
1008
1010
1011
1012
1014
2001
2002
2003
2004
2005
2006
2007
2008
2009
2010
2011
2-12
2013
2015
2016
2017
2018
2020
2021
2022
2023
2024
2025
2026
2027
2028
2029
2030
2031
2033
2036
2037
2039
3001
HOSPITAL NUMBER
Feedback Loop to Engage the Clinicians

40. Results
• 2060 children with osteomyelitis
• 1005 oral antibiotics, 1055 PICC-administered antibiotics.
• The proportion of children treated via the PICC route
varied across hospitals from 0 to 100%.
• Treatment failure risk difference = 0.3% [95% CI, −0.1% to
2.5%]) (across hospital matched analyses)
• Among children in PICC group, 158 (15.0%) had a PICC
complication that required an emergency department visit
(n = 96), a rehospitalization (n = 38), or both (n = 24).

41. Comments
• Likely to be strongest evidence available to answer
question
• RCT not feasible
• Confirms results of prior study that used only
administrative data
• Results consistent, even with rise in MRSA
prevalence (study period 2009-2012)

42. Secrets to Success
• Funding institute interested in CER
• Availability of data —PHIS —hosted by CHA
• Pediatric Research in Inpatient Settings (PRIS)—research
network to identify site leads and facilitate chart review
• Engaged clinicians

43. Dissemination
• PCORI-organized CME seminar
• JAMA Pediatrics sponsored Twitter Journal Club
• CHA sponsored webinar
• Coverage in dozens of pediatric and lay media

44. Authorship/Attribution

45. Implementation
• Partner with CHA to produce quarterly reports
• We validated admin codes and they have high sens/spec
for case, exposure, outcome ascertainment.
• Audit and feedback reports back to CMOs, CQOs, CSOs.
• Change package-- education, guideline, treatment
recommendations

46. How will PRIS achieve its vision?

47. Demonstrate improvement
in patient outcomes/
impact on cost outcomes
Goal:
Delivery of
High Value Care
Goal:
Delivery of
High Value Care
Reduce Variation
Condition 1

48. Condition 1
Condition 2
Condition 3
Condition 2
Priority Condition
Step 1
Goal:
Delivery of
High Value Care
Goal:
Delivery of
High Value Care
Goal:
Delivery of
High Value Care
Reduce Variation
Condition 1
Demonstrate improvement
in patient outcomes/
impact on cost outcomes

49. Condition 1
Condition 2
Condition 3
Condition 2
Priority Condition
Necessary
Data
(PHIS+)
Evidence/
Evidence-Based
Best Practices
No Evidence
Step 1 Step 2
Goal:
Delivery of
High Value Care
Goal:
Delivery of
High Value Care
Goal:
Delivery of
High Value Care
Reduce Variation
Condition 1
Demonstrate improvement
in patient outcomes/
impact on cost outcomes

50. Condition 1
Condition 2
Condition 3
Condition 2
Priority Condition
Necessary
Data
(PHIS+)
Evidence/
Evidence-Based
Best Practices
Collaboration of
Physician/Nursing
Champions Across
Hospitals
No Evidence
Step 1 Step 2 Step 3
Goal:
Delivery of
High Value Care
Goal:
Delivery of
High Value Care
Goal:
Delivery of
High Value Care
Reduce Variation
Condition 1
Demonstrate improvement
in patient outcomes/
impact on cost outcomes

51. Condition 1
Condition 2
Condition 3
Condition 2
Priority Condition
Necessary
Data
(PHIS+)
Evidence/
Evidence-Based
Best Practices
Collaboration of
Physician/Nursing
Champions Across
Hospitals
No Evidence
Step 1 Step 2 Step 3
Goal:
Delivery of
High Value Care
Goal:
Delivery of
High Value Care
Goal:
Delivery of
High Value Care
Reduce Variation
Condition 1
Step 4
Demonstrate improvement
in patient outcomes/
impact on cost outcomes
Data Tracking System – to Measure and Monitor Patient/Cost Outcomes

52. raj.srivastava@hsc.utah.edu
www.prisnetwork.org