Adding social determinant data to risk adjustment models for pediatric readmissions led to minimal changes in model performance at the discharge level, but resulted in changes to hospital performance rankings. Specifically: - Adding social determinant variables from electronic health records and zip codes to existing clinical risk adjustment models did not meaningfully improve the accuracy or fit of models predicting individual readmissions. - However, accounting for social determinants did change some hospitals' risk-adjusted readmission rates and performance deciles compared to peers. This suggests social determinants may influence hospital performance evaluations and penalties if unadjusted. - Including social determinants in readmissions modeling more fully captures factors influencing readmissions and provides a more accurate assessment of hospital quality.

1. Adding Social Determinant Data Changes
Children’s Hospitals’ Readmissions Performance
Children’s Hospital Colorado
Marion R. Sills, MD, MPH
Matt Hall, PhD
Gretchen J. Cutler, PhD, MPH
Jeffrey D. Colvin, MD, JD
Laura M. Gottlieb, MD, MPH
Michelle L. Macy, MD, MS
Jessica L. Bettenhausen, MD
Rustin B. Morse, MD
Evan S. Fieldston, MD, MBA, MSHP
Jean L. Raphael, MD, MPH
Katherine A. Auger, MD, MSc
Samir S. Shah, MD, MSCE

2. MARION SILLS has documented no financial relationships
to disclose or Conflicts of Interest (COIs) to resolve

3. Background: Readmissions-related policy
• Affordable Care Act

4. Background: Readmissions-related policy
• Affordable Care Act: Centers for Medicare and Medicaid
Services (CMS) must improve quality, as reflected by
readmissions
Quality of
inpatient care
Readmissions

5. Background: Readmissions-related policy
• CMS implemented readmissions penalties nationwide (adults)
• Several state Medicaid agencies implemented readmissions
penalties (currently 8 states)
5
Readmissions Readmission penalty
Quality of
inpatient care

6. Background: Risk adjustment
• Although readmission penalties were designed to incentivize high
quality inpatient care…
Readmissions Readmission penalty
Quality of
inpatient care

7. Background: Risk adjustment
• … pediatric studies: other factors—beyond the hospital’s influence--
are associated with readmission
Readmissions Readmission penalty
Medical factors Social determinants of health (SDH)
Quality of
inpatient care

8. Background: Risk adjustment
• Prior pediatric studies: medical factors and social determinants are
associated with readmission penalties
8
Readmissions Readmission penalty
Medical factors Social determinants of health
Quality of
inpatient care
Sills MR, Hall M, Colvin JD, et al. Association of Social Determinants With Children's Hospitals'
Preventable Readmissions Performance. JAMA Pediatr. 2016 Apr;170(4):350-8.

9. Background: Risk adjustment
• State Medicaid policies: allow risk-adjustment for some medical
factors; not for SDH
9
Readmissions Readmission penalty
Medical factors Social determinants of health
Quality of
inpatient care

10. Background: Risk adjustment
• No prior studies measured impact of social determinants on both
readmissions and readmission penalties in children
Medical factors Social determinants of health
Quality of
inpatient care
Readmissions Readmission penalty

11. Background: Risk adjustment
• No prior studies measured impact of social determinants on both
readmissions and readmission penalties in children
Discharge-level
outcome:
Readmissions
Hospital-level
outcome:
Readmission penalty
Medical factors Social determinants of health
Quality of
inpatient care

12. Objectives
• To determine whether risk adjustment for SDH in models of short term
(30-day) all-condition readmissions
Discharge-level
outcome:
Readmissions
Hospital-level
outcome:
Readmission penalty
• changes hospital-level
performance
• improves fit and accuracy of
discharge-level models
Medical factors Social determinants of health

13. Methods
• Retrospective cohort study
• Data source: 47 children’s hospitals in the Pediatric Health
Information System (PHIS), including non-freestanding
• Study period: calendar year 2014

14. Methods: Readmissions Measure
• Pediatric All Condition Readmissions (PACR)
• Selected because its specifications are publicly available and
NQF-endorsed
• PACR cohort definition = study cohort definition
• Includes acute care hospitalizations for patients <18 years
• Excludes hospitalizations
• of newborns
• for obstetrics
• for psychiatry
• for a planned procedure
14

15. Methods: Readmissions Measure
• Pediatric All Condition Readmissions (PACR)
• PACR risk-adjusts for
• Chronic conditions: uses Chronic Condition Indicators (CCI)
• from Healthcare Cost and Utilization Project (HCUP)
• CCI categorizes chronic conditions into 17 body systems
• PACR measures CCI as 17 indicator variables and a count of number of
body systems involved
• 2 SDH variables: age and sex
1515

16. Methods: Outcome measure
• Discharge-level
outcome: 30-day
PACR
16
Discharge-level
outcome:
Readmissions
Hospital-level
outcome:
Readmission penalty
• Hospital-level outcome:
change in hospital decile
rank on 30-day PACR
Medical factors Social determinants of health

17. Methods: Models
• Sequential models, added risk-adjustment variables:
• Model 1: CCI only
17

18. Methods: Models
• Sequential models, added risk-adjustment variables:
• Model 1: CCI only
• Model 2: Model 1 + age + sex (= the PACR risk adjustment model)
18
Social determinants
of health (SDH)

19. Methods: Models
• Sequential models, added risk-adjustment variables:
• Model 1: CCI only
• Model 2: Model 1 + age + sex (= the PACR risk adjustment model)
• Model 3: Model 2 + electronic health record (EHR)-derived SDH
• race
• ethnicity
• payer
19
Social determinants
of health (SDH)

20. Methods: Models
• Sequential models, added risk-adjustment variables:
• Model 1: CCI only
• Model 2: Model 1 + age + sex (= the PACR risk adjustment model)
• Model 3: Model 2 + EHR-derived SDH
• Model 4: Model 3 + zip-code-linked SDH
• Median household income
• Proportion of housing units vacant
• Proportion of households with children that are single-parent
• Proportion of families below poverty level
• Unemployment rate
• Proportion of adults with less than a high school diploma or equivalent
20
Social determinants
of health (SDH)

21. Methods: Discharge-Level Analysis
• Models: generalized linear mixed effects models
• To assess model accuracy:
• c-statistic for each model
• significance of the improvement in c-statistic between sequential
models
• To test improvement in model fit: Likelihood Ratio test
• Performed for each pair of successive models
• Compares the goodness-of-fit of two models
21
Discharge-level
outcome:
Readmissions

22. Methods: Hospital-Level Analyses
• Calculated hospital-level unadjusted and adjusted readmission
rates for the 4 models
• Calculated change in hospital-level rank-decile
22
Hospital-level
outcome:
Readmission penalty

23. 23
Risk Adjustment Factors: Known determinants of pediatric readmissions
Study OutcomesUnmeasured
Risk-adjustment factors in PACR
• Model 1: Chronic conditions
• Model 2: Model 1 + age + sex
Additional SDH adjustment
• Model 3: Model 2 + EHR-derived SDH
• Model 4: Model 3 + zip-code-linked SDH
Process
measure:
Quality of
hospital’s
inpatient care
Discharge-level
outcome:
Readmission rate
(change in model
accuracy and fit)
Hospital-level
outcome:
Readmission penalty
(readmission rank
decile change)
Methods: Summary of Study Design

24. 24
Results: Study population characteristics
EHR-derived SDH variables
• 30% infants
• 54% male
• 58% white
• 21% Hispanic
• 59% Medicaid-insured
• 39% had conditions involving >2
of CCI’s 17 body systems
Zip-linked SDH variables
• 8% families <poverty level
• median household income: $39,000
• 6% housing units vacant
• 10% unemployment rate
• 14% adults with < high school
diploma
• 27% households with children that
are single-parent
• 458,686 index discharges

25. 25
Results: PACR Rates
Q1: 5.3%
Discharge-level
readmission rate
Hospital-level
readmission rate
Q3: 7.0%
Median: 6.1%
Mean: 6.1%
Max: 8.6%Min: 4.2%
IQR

26. 26
Results: Discharge-level models: Model 4
EHR-derived SDH variables
• Age < 1 year
• Female sex
• 1 of the 17 CCI body
systems (respiratory)
• Having a larger number of
CCI body systems involved
• Medicaid insurance
• White race
Zip-linked SDH variables
• Higher median household
income (unadjusted change:
$64)
• Higher percentage of single
parent households (unadjusted
change: 0.1%)
• Factors increasing risk of 30-day PACR (all p <0.001)

27. 27
Model
C-statistic
Rise in c-
statistic (p
value)
Likelihood Ratio
test
Preferred
model (p value)
Model 1: Readmissions adjusted
for CCI’s
0.704
<0.001
Model 2
(<0.001)Model 2: Model 1 plus age and
sex (the full PACR)
0.707
<0.001
Model 3
(<0.001)Model 3: Model 2 plus EHR-
derived SDH
0.708
0.385
Model 4
(0.011)Model 4: Model 3 plus zip-linked
SDH
0.708
Results: Discharge-level Models: Accuracy Statistics
C-statistic values:
reasonable

28. 28
Model
C-statistic
Rise in c-
statistic (p
value)
Likelihood Ratio
test
Preferred
model (p value)
Model 1: Readmissions adjusted
for CCI’s
0.704
<0.001
Model 2
(<0.001)Model 2: Model 1 plus age and
sex (the full PACR)
0.707
<0.001
Model 3
(<0.001)Model 3: Model 2 plus EHR-
derived SDH
0.708
0.385
Model 4
(0.011)Model 4: Model 3 plus zip-linked
SDH
0.708
Results: Discharge-level Models: Accuracy Statistics
PACR
PACR
PACR + SDHSocial determinants of health

29. 29
Model
C-statistic
Rise in c-
statistic (p
value)
Likelihood Ratio
test
Preferred
model (p value)
Model 1: Readmissions adjusted
for CCI’s
0.704
<0.001
Model 2
(<0.001)Model 2: Model 1 plus age and
sex (the full PACR)
0.707
<0.001
Model 3
(<0.001)Model 3: Model 2 plus EHR-
derived SDH
0.708
0.385
Model 4
(0.011)Model 4: Model 3 plus zip-linked
SDH
0.708
Results: Discharge-level Models: Accuracy Statistics
Minimal
change

30. 30
Model
C-statistic
Rise in c-
statistic (p
value)
Likelihood Ratio
test
Preferred
model (p value)
Model 1: Readmissions adjusted
for CCI’s
0.704
<0.001
Model 2
(<0.001)Model 2: Model 1 plus age and
sex (the full PACR)
0.707
<0.001
Model 3
(<0.001)Model 3: Model 2 plus EHR-
derived SDH
0.708
0.385
Model 4
(0.011)Model 4: Model 3 plus zip-linked
SDH
0.708
Results: Discharge-level Models: Goodness of Fit Statistics

31. 31
Model
C-statistic
Rise in c-
statistic (p
value)
Likelihood Ratio
test
Preferred
model (p value)
Model 1: Readmissions adjusted
for CCI’s
0.704
<0.001
Model 2
(<0.001)Model 2: Model 1 plus age and
sex (the full PACR)
0.707
<0.001
Model 3
(<0.001)Model 3: Model 2 plus EHR-
derived SDH
0.708
0.385
Model 4
(0.011)Model 4: Model 3 plus zip-linked
SDH
0.708
Results: Discharge-level Models: Summary
Summary of discharge-level findings: adding SDH led to minimal changes in model performance

32. 32
Results: Hospital Rank Decile by Readmissions
Rate
• Lower rank decile: lower readmission rate (blue)
• Higher decile rank: higher readmission rate (red)

33. 33
Results: Hospital Rank Decile by Readmissions
Rate

34. 34
Results: Hospital Rank Decile by Readmissions
Rate
8 hospitals: higher decile (worse)
9 hospitals: lower decile (better)
17 hospitals (36%) changed decile rank

35. Discussion: Discharge-level Analysis
• Consistent with prior studies, we found an association between
SDH with readmissions
• However, addition of SDH risk-adjustment: minimal improvement
in the fit and accuracy of prediction models for PACR
• Model performance fair: readmissions hard to predict!
35
Model C-statistic
Model 2: Model 1 plus age and sex (the PACR) 0.707
Model 4: Model 3 plus zip-linked SDH 0.708

36. Discussion: Discharge-level Analysis
• Why such small improvement in model performance?
• High risk SDH (e.g. low income) associated with chronic medical
complexity
• Thus, CCI-adjustment (Model 1) also adjusted for SDH
Readmissions
Medical factors: CCI
Social determinants of health (SDH)

37. Discussion: Hospital-level Analysis
• SDH risk adjustment: substantial changes in rank decile of
hospitals’ readmission performance
• Consistent with our prior hospital-level analysis: SDH-adjustment
changes hospital-level pediatric readmission penalty status
• Consequences for hospitals could be substantial
37

38. Limitations
• PHIS dataset represents only children’s hospitals:
• May limit variation between hospitals
• May limit generalizability to children hospitalized in other settings
• Use of decile rankings can distort the effect of outcome change
• the number of rank changes is related to the number of ranks

39. Conclusions
• SDH risk adjustment using variables common in or linkable
to electronic data
• Substantial impact on readmissions performance at hospital-level
and thus can impact penalties
• Small impact on readmission model performance at discharge-level
• Findings support inclusion of SDH variables in risk-
adjustment of hospital-level pay for performance measures
39
Discharge-level
outcome:
Readmissions
Hospital-level
outcome:
Readmission penalty
Social determinants of health (SDH)

40. Matt Hall, PhD
Gretchen J. Cutler, PhD, MPH
Jeffrey D. Colvin, MD, JD
Laura M. Gottlieb, MD, MPH
Michelle L. Macy, MD, MS
Jessica L. Bettenhausen, MD
Rustin B. Morse, MD
Evan S. Fieldston, MD, MBA, MSHP
Jean L. Raphael, MD, MPH
Katherine A. Auger, MD, MSc
Samir S. Shah, MD, MSCE
Acknowledgments
Citation Co-authors
Sills MR et al. Adding
Social Determinant
Data Changes
Children’s Hospitals’
Readmissions
Performance. J
Pediatrics 2017
Marion.Sills@ucdenver.edu

41. Discharge-level Analysis
• How do we better predict readmission?
• Include other variables
• measures of SDH not available in our dataset
• other measures: e.g., prior utilization patterns, access to care and health
literacy
41

42. 42

43. 43

44. 44