Prevention of Sepsis Through Machine Learning Driven Targeted Early Detection

A Deep-Dive into Real-Time Sepsis Predictions
Zhijie Jet Wang
April 13, 2019
1

COPYRIGHT 2019 PCCI. All Rights Reserved.
PCCI
• Parkland Center for Clinical Innovation (PCCI) is an advanced, not-for-profit
healthcare analytics R&D organization with a mission to create a world of
connected communities where every health outcome is positive. We combine
deep clinical expertise with advanced analytics and artificial intelligence to
enable the delivery of precision medicine at the point of care. PCCI is a
recipient of more than $50 million in grants directed at developing and
deploying patient centric cutting edge technologies connecting communities,
Parkland Health & Hospital System, and beyond.
• PCCI is affiliated with Parkland Health and Hospital System (PHHS)
2

AGENDA
Background Models Evaluation
3

What Is Sepsis?
• Sepsis is a potentially life-threatening condition caused by
the body's response to an infection. The body normally
releases chemicals into the bloodstream to fight an
infection. Sepsis occurs when the body's response to
these chemicals is out of balance, triggering changes that
can damage multiple organ systems (Mayo Clinic).
• There are more than 1 million cases of sepsis each year,
and it kills more than 258,000 Americans annually (CMS).
• Survival probability decreases by 7.6% for each hour of
delay after documented hypotension. (A. Kumar et al)
5

SIRS – The Current Standard
• Systemic Inflammatory Response Syndrome (SIRS)
o Temp >38°C (100.4°F) or < 36°C (96.8°F)
o Heart rate > 90
o Respiratory rate > 20 or PaCO₂ < 32 mm Hg
o WBC > 12,000/mm³, < 4,000/mm³, or > 10% bands
• Suspected or present source of infection
Any 2
Sepsis Criteria
Lactic acidosis, SBP <90 or SBP drop ≥ 40 mm Hg of normal Criteria
Severe Sepsis Criteria
Severe sepsis with hypotension, despite adequate fluid resuscitation Septic Shock Criteria
• The standard keeps changing
• Low PPV and Sensitivity
6

ED Sepsis vs Inpatient Sepsis
• 85% of sepsis cases are treated in Emergency Department (i.e. Present-On-
Admission) and 15% of sepsis cases are Hospital-Acquired (i.e. NPOA).
(SL. Jones et al.)
• ED sepsis cases have 12% mortality rate, and Hospital-Acquired Inpatient
sepsis cases have 35% mortality rate. (SL. Jones et al.)
7

Two Models
• ED Sepsis Model
 Developed from and for the Emergency Department (ED) population
 Evaluates patients only while in ED
 Already deployed!
• Inpatient Sepsis Model
 Developed from and for the inpatient population
 Evaluates patients only while in inpatient status
 To be deployed
9

IP Sepsis Modeling Population
• Inclusion Criteria:
 Age 18 years and older.
 Discharged from PHHS between 10/01/2015 and
09/30/2018.
• Exclusion Criteria:
 No valid diagnosis code.
 LOS > 100 days.
10

Model Outcome
• Predict the risk of post-admission sepsis, severe sepsis, or septic
shock prior to onset, defined by the following discharging ICD-10
codes:
'A02.1', 'A22.7', 'A26.7', 'A32.7', 'A40.0', 'A40.1', 'A40.3', 'A40.8', 'A40.9',
'A41.01', 'A41.02', 'A41.1', 'A41.2', 'A41.3', 'A41.4', 'A41.50', 'A41.51',
'A41.52', 'A41.53', 'A41.59', 'A41.81', 'A41.89', 'A41.9', 'A42.7', 'A54.86',
'B37.7', 'R65.10‘,'R65.20', 'R65.21‘
11

Sepsis Onset Time
• Ideally, the model should fire alerts before sepsis onset.
• No reliable data to identify the exact sepsis onset time currently.
• Start of intravenous antibiotics (IV ABX) treatment by an inpatient team as surrogate
for most likely sepsis onset time for hospital-acquired sepsis:
• ED Inpatient Discharge
IV ABX
12

CANDIDATE PREDICTOR GROUPS
➢Demographics
○ Age
○ Race
○ Other
➢Vital Signs
○ Temperature
○ DBP
○ Other
➢Orders
○ Antibiotics
○ Fluids
○ Other
➢ Lab Tests
○ Albumin
○ aPTT
○ Other
➢Clinical Profile:
○ Oncology (solid organ/heme)
○ Stem Cell Transplant
○ Other
➢Clinical History:
○ Admission source (OP, IP, LTCF/SNF,
Nursing home)
○ Prior sepsis diagnosis
○ Other
200+ candidate predictors were identified.
13

Modeling Data
• 70% for training, and 30% for testing, proportional to the outcome
prevalence.
• Training dataset only considers and uses information available up to
point of sepsis onset time (surrogate defined as the time of first IV ABX
during inpatient stay).
• Because vital signs and labs are strong indicators of sepsis onset, we
only used vital signs and labs collected at least 12 hours prior to any IV
ABX orders.
14

Feature Selection
• Missing value rate < 50%.
• A forward stepwise method with internal 10-fold cross-validations.
• 35 top-ranked predictors out of 100 iterations were selected.
15

Missing Values
• Imputed with the normal value of that feature.
• Last Observation Carried Forward.
16

Candidate Models
• Lasso Logistic Regression (LLR)
• Random Forest (RF)
• Support Vector Machine (SVM)
• XGBoost (XGB)
• Artificial Neural Network (ANN)
17

Model Accuracy in Testing Dataset
Metric LRR RF SVM ANN XGB
Prevalence Rate 2.8%
Sensitivity 45% 42% 34% 38% 50%
Specificity 96% 97% 98% 98% 97%
PPV 25% 21% 23% 24% 35%
NPV 99% 99% 98% 99% 99%
Accuracy 95% 96% 96% 96% 96%
F1 Score 32% 28% 27% 30% 41%
C-stat 91% 92% 92% 93% 93%
19

Model Feature Impact
20

Model Flow in Hospital
21

Conclusions
• We successfully separated IP sepsis cases from ED sepsis cases.
• The XGB model was able to capture 50% of sepsis cases.
• The XGB model achieves a statistical lift of 13.
• The model is able to send out alerts at least 12 hours prior to sepsis onset.
22

Limitations
• PPV and Sensitivity could be further improved.
• Sepsis onset time was not accurate.
23

Work at PCCI
https://pccinnovation.org/careers/
24

THANK YOU
PCCInnovation.org
25

Prevention of Sepsis Through Machine Learning Driven Targeted Early Detection

Empfohlen

Empfohlen

Weitere ähnliche Inhalte

Ähnlich wie Prevention of Sepsis Through Machine Learning Driven Targeted Early Detection

Ähnlich wie Prevention of Sepsis Through Machine Learning Driven Targeted Early Detection (20)

Kürzlich hochgeladen

Kürzlich hochgeladen (20)

Prevention of Sepsis Through Machine Learning Driven Targeted Early Detection