SlideShare a Scribd company logo
CRISMA Center, Department of Critical Care Medicine
Department of Health Policy and Management
McGowan Institute for Regenerative Medicine
Clinical and Translational Science Institute
University of Pittsburgh Schools of the Health Sciences
Derek C. Angus, MD, MPH, FRCP
CRISMA Center, Department of Critical Care Medicine
Department of Health Policy and Management
McGowan Institute for Regenerative Medicine
Clinical and Translational Science Institute
University of Pittsburgh Schools of the Health Sciences
Derek C. Angus, MD, MPH, FRCP
Big Data vs. the RCT?
C  R  I  S  M  A
Evidence-based medicine
“Doctors must base what they do on
randomized clinical trials (RCTs)”
Archie Cochrane
C  R  I  S  M  A
How we know something works ...
Patients with
disease X
C  R  I  S  M  A
How we know something works ...
Patients with
disease X
C  R  I  S  M  A
How we know something works ...
Patients with
disease X
C  R  I  S  M  A
How we know something works ...
Patients with
disease X
C  R  I  S  M  A
How we know something works ...
Patients with
disease X
C  R  I  S  M  A
C  R  I  S  M  A
The good news …
•We now test many ideas with RCTs
- 37,000 started in 2010 …
-All FDA drug and device approvals
•We now conduct RCTs very well
- Methodologic conduct
- Ethical oversight
- Reporting
C  R  I  S  M  A
But …
• RCTs are too narrow
• Cherry-picked population; not the real world
• RCTs are too broad
• No data on treatment effects across patients
• No comparative effectiveness
• Rx A vs. B is not very helpful
• What about A vs. B, vs. C, vs. D … etc.
• Depending on whether I give E or F …
• Having said all that, I just want the answer …
• I don’t want my patient to be a guinea pig …
C  R  I  S  M  A
Clinical care Clinical research
Parallel universes …
C  R  I  S  M  A
Enter the era of ‘Big Data’
• Integration of ‘deep’ personalized data
• Causal inferences on optimal care
• Broad – ‘real-world’ practice
• Narrow – ‘personal’ estimates
• Comparative – considers all options
• Vanderbilt-IBM ‘BioVU’ initiative
• ‘Live’ presentation of information at time
of clinical decision-making
• ‘Just-in-time’ cohort study in EHR
• No guinea pigs
• Longhurst et al. Health Affairs 2013
C  R  I  S  M  A
Feature
Leverage the EHR
Low incremental costs
Real-world ‘effectiveness’
Consider multiple therapies
‘Personalized’ estimates
Offer ‘live’ tailored options
Robust causal inference
‘Big Data’
Analytics
✔
✗
✔
✔
✔
✔
✔
Evidence Generator Report Card
C  R  I  S  M  A
Point-of-care (POC) Clinical Trials
• A clinical moment in the EHR ‘alerts’ the clinical trial machinery
• VA EHR
• In-patient diabetics with poor glucose control
• When physician placed insulin order in CPOE system …
• Opportunity to randomize
• Sliding scale
• Weight-based algorithm
• Fiore et al. Clinical Trials 2011
• Targeting the large ‘pragmatic’ trial arena
• 2 thiazide diuretics in >13k high BP Veterans (NCT02185417)
• 2 aspirin doses in 20k CVD patients (ADAPTABLE) (PCORI)
Feature
Leverage the EHR
Low incremental costs
Real-world ‘effectiveness’
Consider multiple therapies
‘Personalized’ estimates
Offer ‘live’ tailored options
Robust causal inference
‘Big Data’
Analytics
POC
Trials
✔
✗/✔
✔
✔
✔
✗
✗
✗ ✔
✔
✔
✔
✔
✔
Evidence Generator Report Card
C  R  I  S  M  A
Platform Trials
• Adaptive trials
• Focus on disease, not a particular Rx
• Multiple interventions (arms)
• ‘Perpetual’ enrollment
• Often based on Bayes’ theorem
• Tailor choices over time
Berry et al JAMA 2015
• Focus on pre-approval space
• Emphasis on efficiency with (very) small sample sizes
• Different therapies ‘graduate’ to next phase while trial continues
Woodcock and Lavange NEJM 2017
C  R  I  S  M  A
C  R  I  S  M  A
Response-adaptive randomization
Rugo et al. NEJM 2016
C  R  I  S  M  A
The traditional RCT ...
Patients with
disease X
At the start,
50% chance
that A > B
C  R  I  S  M  A
The traditional RCT ...
Patients with
disease X
At the end, >99% sure that A > B
What about in the middle?
C  R  I  S  M  A
A planned trial of A vs. B in 400 patients
The probability that A > B = 78%
Start randomizing MORE patients to A than B …
Alive
Dead
40
20
No. of
patients
A B
After 40 enrolled ….
C  R  I  S  M  A
After 80 patients …
Now, the probability that A > B = 99.9%
Stop the trial!
Alive
Dead
40
20
No. of
patients
A B
C  R  I  S  M  A
Caveats
1. If the ‘second’ 40 was flat or opposite direction …
• Trial continues and the next ‘bet’ swings back closer to 50:50
2. When only 2 groups, power still driven by the smaller group
• So, NOT very helpful if …
• Single homogenous cohort
• Two arms
• But, becomes VERY interesting when …
• Multiple arms
• Multiple subgroups
B
C
Statistical modelRandomization rule
Response-adaptive randomization
A
B
C
Statistical modelRandomization rule
Response-adaptive randomization
A
Odds weighted
towards best RX
B
C
Statistical modelRandomization rule
Response-adaptive randomization
A
D
New arms
activated
A
B
Statistical modelRandomization rule
Response-adaptive randomization
Or dropped
D
C
A
Statistical modelRandomization rule
Response-adaptive randomization
Different weights
for different
patient groups
D
Feature
Leverage the EHR
Low incremental costs
Real-world ‘effectiveness’
Consider multiple therapies
‘Personalized’ estimates
Offer ‘live’ tailored options
Robust causal inference
‘Big Data’
Analytics
POC
Trials
Platform
Trials
✔
✗/
✔
✗
✗
✔
✔
✔
✔
✗
✔
✔
✔
✗
✗
✗ ✔
✔
✔
✔
✔
✔
Evidence Generator Report Card
Feature
Leverage the EHR
Low incremental costs
Real-world ‘effectiveness’
Consider multiple therapies
‘Personalized’ estimates
Offer ‘live’ tailored options
Robust causal inference
‘Big Data’
Analytics
POC
Trials
Platform
Trials ???
✔
✗/✔
✗
✗
✔
✔
✔
✔
✗
✔
✔
✔
✗
✗
✗ ✔
✔
✔
✔
✔
✔
✔
✔
✔
✔
✔
✔
✔
Evidence Generator Report Card
C  R  I  S  M  A
A novel blend of ‘POC’ + platform designs
•REMAP
•Randomized
•Embedded
•Multifactorial
•Adaptive
•Platform trial
REMAP
✔
✔
✔
✔
✔
✔
✔
Angus DC. JAMA 2015
C  R  I  S  M  A
• Funding
• EU FP7 PREPARE WP 5 program (25M euro)
• Australian NHMRC ‘OPTIMISE’ program ($6M)
• New Zealand NHMRC ($2M)
• Simultaneously test
• Different anti-microbial strategies
• Different host immunomodulation strategies
• Different ventilation strategies
• Separate RAR and stopping rules for multiple subgroups
Angus DC. JAMA 2015
C  R  I  S  M  A Angus DC. JAMA 2015
• Patients are preferentially assigned to best performing arm
• Allocation is random, but NOT 50:50
• Odds of assignment proportional to odds of success
• Not a guinea pig!
• Embedded
• ICU admission orders
• Approved in Netherlands and New Zealand
with delayed consent
C  R  I  S  M  A
REMAP-CAP elements
• Domain – an area where a question is asked …
• Domain #1 – choice of antibiotic
• Domain #2 – whether to give steroids or not
• Domain #3 – whether to extend macrolide or not
• Domain #4 – choice of ventilator strategy
• Domain #5 – oxygen titration strategy
• Etc. ….
• Intervention
• Any option within a domain …
• Regimen
• Unique combination of interventions within a domain …
• Stratum
• Baseline subgroup
• Ex. shock or not
Multifactorial intervention assignments
Regimen = set of domain-specific interventions
Effect of an intervention is conditional upon
• Stratum
• Interventions within other domains
Regimen Domain A Domain B Domain C
#1 A1 B1 C1
#2 A1 B1 C2
#3 A1 B2 C1
#4 A1 B2 C2
#5 A2 B1 C1
…..
#n An Bn Cn
Multifactorial intervention assignments
Regimen = set of domain-specific
interventions
Effect of an intervention is conditional upon
• Stratum
• Interventions within other domainsRegimen Domain A Domain B Domain C
#1 A1 B1 C1
#2 A1 B1 C2
#3 A1 B2 C1
#4 A1 B2 C2
#5 A2 B1 C1
…..
#n An Bn Cn
Pre-specified architecture determined by
• Choice of domains, strata, etc.
• Choice of potential interactions
Choices inform a Bayesian inference model
• Pre-trial simulations evaluate performance
Each external adaptation (ex. new domain)
• Modify elements in Bayesian model
• Re-simulate before ‘live’ deployment
Pre-trial design and construction
Pre-specified architecture determined by
• Choice of domains, strata, etc.
• Choice of potential interactions
Choices inform a Bayesian inference model
• Pre-trial simulations evaluate performance
Each external adaptation (ex. new domain)
• Modify elements in Bayesian model
• Re-simulate before ‘live’ deployment
Pre-trial design and construction
Z = g[ ]+ R[ ]+ Time[ ]+ Interv[ ]+ Inter,Shock[ ]+ Interv,Hypox[ ]+ Interv,Interv[ ]
Multifactorial intervention assignments
Regimen = set of domain-specific
interventions
Effect of an intervention is conditional upon
• Stratum
• Interventions within other domainsRegimen Domain A Domain B Domain C
#1 A1 B1 C1
#2 A1 B1 C2
#3 A1 B2 C1
#4 A1 B2 C2
#5 A2 B1 C1
…..
#n An Bn Cn
Pre-specified architecture determined by
• Choice of domains, strata, etc.
• Choice of potential interactions
Choices inform a Bayesian inference model
• Pre-trial simulations evaluate performance
Each external adaptation (ex. new domain)
• Modify elements in Bayesian model
• Re-simulate before ‘live’ deployment
Pre-trial design and construction
Statistical trigger
Steering Committee can
• Add strata, domains & interventions
DSMB can
• Request new external data be incorporated in priors
• Overrule statistical triggers
Multifactorial intervention assignments
Regimen = set of domain-specific
interventions
Effect of an intervention is conditional upon
• Stratum
• Interventions within other domainsRegimen Domain A Domain B Domain C
#1 A1 B1 C1
#2 A1 B1 C2
#3 A1 B2 C1
#4 A1 B2 C2
#5 A2 B1 C1
…..
#n An Bn Cn
Result declared when, within stratum, an intervention is
• Superior >99% likely to be best
• Equivalent >90% likely to be D<3%
• Inferior <1% likely to be best
Pre-specified architecture determined by
• Choice of domains, strata, etc.
• Choice of potential interactions
Choices inform a Bayesian inference model
• Pre-trial simulations evaluate performance
Each external adaptation (ex. new domain)
• Modify elements in Bayesian model
• Re-simulate before ‘live’ deployment
Pre-trial design and construction
Patients
Presumed severe CAP
• Different strata
(ex. shock or not)
• Collected at sites
• Managed at regional data centers
• Merged at central statistical center
Data collection
Re-estimate Bayesian inference model
with new data to update probabilities
Update and adapt
External adaptations
Statistical trigger
Steering Committee can
• Add strata, domains & interventions
DSMB can
• Request new external data be incorporated in priors
• Overrule statistical triggers
Multifactorial intervention assignments
Regimen = set of domain-specific
interventions
Effect of an intervention is conditional upon
• Stratum
• Interventions within other domains
Embedding
Patient identification and enrollment
• Tied to clinical ‘point-of-care’
Randomized interventions
• Issued as ‘order set’ regimen
EHR embedding
• Screen and flag patient
• Consent documentation
• Generate regimen order set
• Flag downstream states
• Data collection
Regimen Domain A Domain B Domain C
#1 A1 B1 C1
#2 A1 B1 C2
#3 A1 B2 C1
#4 A1 B2 C2
#5 A2 B1 C1
…..
#n An Bn Cn
Result declared when, within stratum, an intervention is
• Superior >99% likely to be best
• Equivalent >90% likely to be D<3%
• Inferior <1% likely to be best
Pre-specified architecture determined by
• Choice of domains, strata, etc.
• Choice of potential interactions
Choices inform a Bayesian inference model
• Pre-trial simulations evaluate performance
Each external adaptation (ex. new domain)
• Modify elements in Bayesian model
• Re-simulate before ‘live’ deployment
Pre-trial design and construction
Patients
Presumed severe CAP
• Different strata
(ex. shock or not)
• Collected at sites
• Managed at regional data centers
• Merged at central statistical center
Data collection
Re-estimate Bayesian inference model
with new data to update probabilities
Update and adapt
External adaptations
Statistical trigger
• Launch with initial weights
• Update based on new probabilities
Steering Committee can
• Add strata, domains & interventions
DSMB can
• Request new external data be incorporated in priors
• Overrule statistical triggers
Multifactorial intervention assignments
Regimen = set of domain-specific
interventions
Effect of an intervention is conditional upon
• Stratum
• Interventions within other domains
Embedding
Patient identification and enrollment
• Tied to clinical ‘point-of-care’
Randomized interventions
• Issued as ‘order set’ regimen
EHR embedding
• Screen and flag patient
• Consent documentation
• Generate regimen order set
• Flag downstream states
• Data collection
Regimen Domain A Domain B Domain C
#1 A1 B1 C1
#2 A1 B1 C2
#3 A1 B2 C1
#4 A1 B2 C2
#5 A2 B1 C1
…..
#n An Bn Cn
Result declared when, within stratum, an intervention is
• Superior >99% likely to be best
• Equivalent >90% likely to be D<3%
• Inferior <1% likely to be best
Pre-specified architecture determined by
• Choice of domains, strata, etc.
• Choice of potential interactions
Choices inform a Bayesian inference model
• Pre-trial simulations evaluate performance
Each external adaptation (ex. new domain)
• Modify elements in Bayesian model
• Re-simulate before ‘live’ deployment
Pre-trial design and construction
Patients
Presumed severe CAP
• Different strata
(ex. shock or not)
• Collected at sites
• Managed at regional data centers
• Merged at central statistical center
Data collection
Re-estimate Bayesian inference model
with new data to update probabilities
Update and adapt
Response-adaptive randomization
External adaptations
Statistical trigger
• Launch with initial weights
• Update based on new probabilities
Steering Committee can
• Add strata, domains & interventions
DSMB can
• Request new external data be incorporated in priors
• Overrule statistical triggers
Multifactorial intervention assignments
Regimen = set of domain-specific
interventions
Effect of an intervention is conditional upon
• Stratum
• Interventions within other domains
Embedding
Patient identification and enrollment
• Tied to clinical ‘point-of-care’
Randomized interventions
• Issued as ‘order set’ regimen
EHR embedding
• Screen and flag patient
• Consent documentation
• Generate regimen order set
• Flag downstream states
• Data collection
Regimen Domain A Domain B Domain C
#1 A1 B1 C1
#2 A1 B1 C2
#3 A1 B2 C1
#4 A1 B2 C2
#5 A2 B1 C1
…..
#n An Bn Cn
Result declared when, within stratum, an intervention is
• Superior >99% likely to be best
• Equivalent >90% likely to be D<3%
• Inferior <1% likely to be best
Pre-specified architecture determined by
• Choice of domains, strata, etc.
• Choice of potential interactions
Choices inform a Bayesian inference model
• Pre-trial simulations evaluate performance
Each external adaptation (ex. new domain)
• Modify elements in Bayesian model
• Re-simulate before ‘live’ deployment
Pre-trial design and construction
Patients
Presumed severe CAP
• Different strata
(ex. shock or not)
• Collected at sites
• Managed at regional data centers
• Merged at central statistical center
Data collection
Re-estimate Bayesian inference model
with new data to update probabilities
Update and adapt
Response-adaptive randomization
External adaptations
Statistical trigger
• Launch with initial weights
• Update based on new probabilities
Steering Committee can
• Add strata, domains & interventions
DSMB can
• Request new external data be incorporated in priors
• Overrule statistical triggers
Multifactorial intervention assignments
Regimen = set of domain-specific
interventions
Effect of an intervention is conditional upon
• Stratum
• Interventions within other domains
Embedding
Patient identification and enrollment
• Tied to clinical ‘point-of-care’
Randomized interventions
• Issued as ‘order set’ regimen
EHR embedding
• Screen and flag patient
• Consent documentation
• Generate regimen order set
• Flag downstream states
• Data collection
Regimen Domain A Domain B Domain C
#1 A1 B1 C1
#2 A1 B1 C2
#3 A1 B2 C1
#4 A1 B2 C2
#5 A2 B1 C1
…..
#n An Bn Cn
Result declared when, within stratum, an intervention is
• Superior >99% likely to be best
• Equivalent >90% likely to be D<3%
• Inferior <1% likely to be best
Pre-specified architecture determined by
• Choice of domains, strata, etc.
• Choice of potential interactions
Choices inform a Bayesian inference model
• Pre-trial simulations evaluate performance
Each external adaptation (ex. new domain)
• Modify elements in Bayesian model
• Re-simulate before ‘live’ deployment
Pre-trial design and construction
Patients
Presumed severe CAP
• Different strata
(ex. shock or not)
• Collected at sites
• Managed at regional data centers
• Merged at central statistical center
Data collection
Re-estimate Bayesian inference model
with new data to update probabilities
Update and adapt
Response-adaptive randomization
External adaptations
Statistical trigger
• Launch with initial weights
• Update based on new probabilities
Steering Committee can
• Add strata, domains & interventions
DSMB can
• Request new external data be incorporated in priors
• Overrule statistical triggers
Multifactorial intervention assignments
Regimen = set of domain-specific
interventions
Effect of an intervention is conditional upon
• Stratum
• Interventions within other domains
Embedding
Patient identification and enrollment
• Tied to clinical ‘point-of-care’
Randomized interventions
• Issued as ‘order set’ regimen
EHR embedding
• Screen and flag patient
• Consent documentation
• Generate regimen order set
• Flag downstream states
• Data collection
Regimen Domain A Domain B Domain C
#1 A1 B1 C1
#2 A1 B1 C2
#3 A1 B2 C1
#4 A1 B2 C2
#5 A2 B1 C1
…..
#n An Bn Cn
Result declared when, within stratum, an intervention is
• Superior >99% likely to be best
• Equivalent >90% likely to be D<3%
• Inferior <1% likely to be best
Pre-specified architecture determined by
• Choice of domains, strata, etc.
• Choice of potential interactions
Choices inform a Bayesian inference model
• Pre-trial simulations evaluate performance
Each external adaptation (ex. new domain)
• Modify elements in Bayesian model
• Re-simulate before ‘live’ deployment
Pre-trial design and construction
Patients
Presumed severe CAP
• Different strata
(ex. shock or not)
• Collected at sites
• Managed at regional data centers
• Merged at central statistical center
Data collection
Re-estimate Bayesian inference model
with new data to update probabilities
Update and adapt
Response-adaptive randomization
External adaptations
Statistical trigger
• Launch with initial weights
• Update based on new probabilities
Multifactorial intervention assignments
Regimen = set of domain-specific
interventions
Effect of an intervention is conditional upon
• Stratum
• Interventions within other domains
Embedding
Patient identification and enrollment
• Tied to clinical ‘point-of-care’
Randomized interventions
• Issued as ‘order set’ regimen
EHR embedding
• Screen and flag patient
• Consent documentation
• Generate regimen order set
• Flag downstream states
• Data collection
Regimen Domain A Domain B Domain C
#1 A1 B1 C1
#2 A1 B1 C2
#3 A1 B2 C1
#4 A1 B2 C2
#5 A2 B1 C1
…..
#n An Bn Cn
Result declared when, within stratum, an intervention is
• Superior >99% likely to be best
• Equivalent >90% likely to be D<3%
• Inferior <1% likely to be best
Pre-specified architecture determined by
• Choice of domains, strata, etc.
• Choice of potential interactions
Choices inform a Bayesian inference model
• Pre-trial simulations evaluate performance
Each external adaptation (ex. new domain)
• Modify elements in Bayesian model
• Re-simulate before ‘live’ deployment
Pre-trial design and construction
Patients
Presumed severe CAP
• Different strata
(ex. shock or not)
• Collected at sites
• Managed at regional data centers
• Merged at central statistical center
Data collection
Re-estimate Bayesian inference model
with new data to update probabilities
Update and adapt
Response-adaptive randomization
Statistical trigger
• Launch with initial weights
• Update based on new probabilities
Steering Committee can
• Add strata, domains & interventions
DSMB can
• Request new external data be incorporated in priors
• Overrule statistical triggers
Multifactorial intervention assignments
Regimen = set of domain-specific
interventions
Effect of an intervention is conditional upon
• Stratum
• Interventions within other domains
Embedding
Patient identification and enrollment
• Tied to clinical ‘point-of-care’
Randomized interventions
• Issued as ‘order set’ regimen
EHR embedding
• Screen and flag patient
• Consent documentation
• Generate regimen order set
• Flag downstream states
• Data collection
Regimen Domain A Domain B Domain C
#1 A1 B1 C1
#2 A1 B1 C2
#3 A1 B2 C1
#4 A1 B2 C2
#5 A2 B1 C1
…..
#n An Bn Cn
Result declared when, within stratum, an intervention is
• Superior >99% likely to be best
• Equivalent >90% likely to be D<3%
• Inferior <1% likely to be best
Pre-specified architecture determined by
• Choice of domains, strata, etc.
• Choice of potential interactions
Choices inform a Bayesian inference model
• Pre-trial simulations evaluate performance
Each external adaptation (ex. new domain)
• Modify elements in Bayesian model
• Re-simulate before ‘live’ deployment
Pre-trial design and construction
Patients
Presumed severe CAP
• Different strata
(ex. shock or not)
• Collected at sites
• Managed at regional data centers
• Merged at central statistical center
Data collection
Re-estimate Bayesian inference model
with new data to update probabilities
Update and adapt
Response-adaptive randomization
External adaptations
C  R  I  S  M  A
REMAP severe pneumonia …
• Gets closer to individualized treatment decisions …
• For example, should my patient receive IV hydrocortisone?
• Depends on
• Whether shock is present
• How sick (hypoxic) the patient is
• Whether underlying cause is viral or not
• Whether an anti-viral is being administered
• Whether other strategies are being used that may minimize lung injury and
inflammation (protective vs. ultraprotective ventilation)
• Separate probability estimate for each consideration …
• Trial enrolls until a predefined level of certainty
• As soon as one question hits threshold, answer is announced
C  R  I  S  M  A
Run the trial ‘in silico’ ahead of time …
• Monte-Carlo simulations
• Run 1,000s of times under different scenarios
‘True’
mortality
Average results
from 1,000s of
simulations
80 fewer deaths; higher power
Scenario #1: 2 of 8 regimens are best
‘True’
mortality
Average results
from 1,000s of
simulations
Similar power but 80 fewer deaths
Scenario #1: 2 of 8 regimens are best
‘True’
mortality
Average results
from 1,000s of
simulations
Scenario #2: One regimen is best
‘True’
mortality
Average results
from 1,000s of
simulations
94 fewer deaths; higher power
Scenario #2: One regimen is best
C  R  I  S  M  A
Ok, but …
• EHR data quality
• Institutional commitment
• Ethics
• Statistics and design
• Reporting and dissemination of results
• Funding
• Oversight
• Integration with other clinical research programs
C  R  I  S  M  A
Conclusions
• RCTs remain our strongest ‘truth’ finder
• But, current RCT enterprise LETS US DOWN
• ‘Big Data’ should not be cast as an alternative to the RCT
• This is a false choice
• Instead, the digital age enables novel RCTs designs
• Smarter and safer
C  R  I  S  M  A
Self-learning healthcare is …
fused care and research

More Related Content

What's hot

Sample size
Sample sizeSample size
Sample size
zubis
 
Estimating design effect and calculating sample size for respondent driven sa...
Estimating design effect and calculating sample size for respondent driven sa...Estimating design effect and calculating sample size for respondent driven sa...
Estimating design effect and calculating sample size for respondent driven sa...
Dung Tri
 
Practical Methods To Overcome Sample Size Challenges
Practical Methods To Overcome Sample Size ChallengesPractical Methods To Overcome Sample Size Challenges
Practical Methods To Overcome Sample Size Challenges
nQuery
 
D1 design and analysis approaches to evaluate cardiovascular risk - 2012 eugm
D1   design and analysis approaches to evaluate cardiovascular risk - 2012 eugmD1   design and analysis approaches to evaluate cardiovascular risk - 2012 eugm
D1 design and analysis approaches to evaluate cardiovascular risk - 2012 eugm
therealreverendbayes
 
East ugm-2012-presentation-east-future-mehta
East ugm-2012-presentation-east-future-mehtaEast ugm-2012-presentation-east-future-mehta
East ugm-2012-presentation-east-future-mehta
Cytel
 

What's hot (18)

Sample size for phase ii clinical trials-Simons design and mcp mod case studies
Sample size for phase ii clinical trials-Simons design and mcp mod case studies Sample size for phase ii clinical trials-Simons design and mcp mod case studies
Sample size for phase ii clinical trials-Simons design and mcp mod case studies
 
Sample size determination
Sample size determinationSample size determination
Sample size determination
 
Adaptive Clinical Trials
Adaptive Clinical TrialsAdaptive Clinical Trials
Adaptive Clinical Trials
 
Power and sample size calculations for survival analysis webinar Slides
Power and sample size calculations for survival analysis webinar SlidesPower and sample size calculations for survival analysis webinar Slides
Power and sample size calculations for survival analysis webinar Slides
 
Intro To Adaptive Design
Intro To Adaptive DesignIntro To Adaptive Design
Intro To Adaptive Design
 
5 essential steps for sample size determination in clinical trials slideshare
5 essential steps for sample size determination in clinical trials   slideshare5 essential steps for sample size determination in clinical trials   slideshare
5 essential steps for sample size determination in clinical trials slideshare
 
Sample size
Sample sizeSample size
Sample size
 
Estimating design effect and calculating sample size for respondent driven sa...
Estimating design effect and calculating sample size for respondent driven sa...Estimating design effect and calculating sample size for respondent driven sa...
Estimating design effect and calculating sample size for respondent driven sa...
 
Practical Methods To Overcome Sample Size Challenges
Practical Methods To Overcome Sample Size ChallengesPractical Methods To Overcome Sample Size Challenges
Practical Methods To Overcome Sample Size Challenges
 
Sample size calculation final
Sample size calculation finalSample size calculation final
Sample size calculation final
 
Minimizing Risk In Phase II and III Sample Size Calculation
Minimizing Risk In Phase II and III Sample Size CalculationMinimizing Risk In Phase II and III Sample Size Calculation
Minimizing Risk In Phase II and III Sample Size Calculation
 
Sample size calculation in medical research
Sample size calculation in medical researchSample size calculation in medical research
Sample size calculation in medical research
 
2. Quality Control Notes
2. Quality Control Notes2. Quality Control Notes
2. Quality Control Notes
 
Innovative Strategies For Successful Trial Design - Webinar Slides
Innovative Strategies For Successful Trial Design - Webinar SlidesInnovative Strategies For Successful Trial Design - Webinar Slides
Innovative Strategies For Successful Trial Design - Webinar Slides
 
D1 design and analysis approaches to evaluate cardiovascular risk - 2012 eugm
D1   design and analysis approaches to evaluate cardiovascular risk - 2012 eugmD1   design and analysis approaches to evaluate cardiovascular risk - 2012 eugm
D1 design and analysis approaches to evaluate cardiovascular risk - 2012 eugm
 
East ugm-2012-presentation-east-future-mehta
East ugm-2012-presentation-east-future-mehtaEast ugm-2012-presentation-east-future-mehta
East ugm-2012-presentation-east-future-mehta
 
Repeated events analyses
Repeated events analysesRepeated events analyses
Repeated events analyses
 
Sample size calculation
Sample size calculationSample size calculation
Sample size calculation
 

Similar to Big data vs the RCT - Derek Angus - SSAI2017

Eugm 2012 mehta - future plans for east - 2012 eugm
Eugm 2012   mehta - future plans for east - 2012 eugmEugm 2012   mehta - future plans for east - 2012 eugm
Eugm 2012 mehta - future plans for east - 2012 eugm
Cytel USA
 
Technology Assessment/Outcome & Cost-Effectiveness Analysis 2016
Technology Assessment/Outcome & Cost-Effectiveness Analysis 2016Technology Assessment/Outcome & Cost-Effectiveness Analysis 2016
Technology Assessment/Outcome & Cost-Effectiveness Analysis 2016
evadew1
 
Clinical Validation of Copy Number Variants Using the AMP Guidelines
Clinical Validation of Copy Number Variants Using the AMP GuidelinesClinical Validation of Copy Number Variants Using the AMP Guidelines
Clinical Validation of Copy Number Variants Using the AMP Guidelines
Golden Helix
 

Similar to Big data vs the RCT - Derek Angus - SSAI2017 (20)

The stepped wedge cluster randomised trial workshop: Session 2
The stepped wedge cluster randomised trial workshop: Session 2 The stepped wedge cluster randomised trial workshop: Session 2
The stepped wedge cluster randomised trial workshop: Session 2
 
Clinical trials: quo vadis in the age of covid?
Clinical trials: quo vadis in the age of covid?Clinical trials: quo vadis in the age of covid?
Clinical trials: quo vadis in the age of covid?
 
Eugm 2012 mehta - future plans for east - 2012 eugm
Eugm 2012   mehta - future plans for east - 2012 eugmEugm 2012   mehta - future plans for east - 2012 eugm
Eugm 2012 mehta - future plans for east - 2012 eugm
 
Does clinical research help me take care of my patient?
Does clinical research help me take care of my patient?Does clinical research help me take care of my patient?
Does clinical research help me take care of my patient?
 
Technology Assessment, Outcomes Research and Economic Analyses
Technology Assessment, Outcomes Research and Economic AnalysesTechnology Assessment, Outcomes Research and Economic Analyses
Technology Assessment, Outcomes Research and Economic Analyses
 
Technology Assessment/Outcome & Cost-Effectiveness Analysis 2016
Technology Assessment/Outcome & Cost-Effectiveness Analysis 2016Technology Assessment/Outcome & Cost-Effectiveness Analysis 2016
Technology Assessment/Outcome & Cost-Effectiveness Analysis 2016
 
Theme6 hemming what_is_sw-crt_virginia_conference_may2015
Theme6 hemming what_is_sw-crt_virginia_conference_may2015Theme6 hemming what_is_sw-crt_virginia_conference_may2015
Theme6 hemming what_is_sw-crt_virginia_conference_may2015
 
Radiology Cost Effectiveness - By Jeffrey Shyu
Radiology Cost Effectiveness - By Jeffrey ShyuRadiology Cost Effectiveness - By Jeffrey Shyu
Radiology Cost Effectiveness - By Jeffrey Shyu
 
randomized clinical trials II
randomized clinical trials IIrandomized clinical trials II
randomized clinical trials II
 
To infinity and beyond
To infinity and beyond To infinity and beyond
To infinity and beyond
 
Optimising sepsis treatment with reinforcement learning - Matthieu Komorowski
Optimising sepsis treatment with reinforcement learning - Matthieu KomorowskiOptimising sepsis treatment with reinforcement learning - Matthieu Komorowski
Optimising sepsis treatment with reinforcement learning - Matthieu Komorowski
 
Data Con LA 2019 - Best Practices for Prototyping Machine Learning Models for...
Data Con LA 2019 - Best Practices for Prototyping Machine Learning Models for...Data Con LA 2019 - Best Practices for Prototyping Machine Learning Models for...
Data Con LA 2019 - Best Practices for Prototyping Machine Learning Models for...
 
Optimising sepsis treatment with reinforcement learning
Optimising sepsis treatment with reinforcement learningOptimising sepsis treatment with reinforcement learning
Optimising sepsis treatment with reinforcement learning
 
Cost Effectiveness in Radiology - By Jeffrey Shyu
Cost Effectiveness in Radiology - By Jeffrey ShyuCost Effectiveness in Radiology - By Jeffrey Shyu
Cost Effectiveness in Radiology - By Jeffrey Shyu
 
CNV Annotations: a crucial step in your variant analysis
CNV Annotations: a crucial step in your variant analysisCNV Annotations: a crucial step in your variant analysis
CNV Annotations: a crucial step in your variant analysis
 
1559221358_week 6-MCQ in EBP-1.pdf
1559221358_week 6-MCQ in EBP-1.pdf1559221358_week 6-MCQ in EBP-1.pdf
1559221358_week 6-MCQ in EBP-1.pdf
 
Clinical Validation of Copy Number Variants Using the AMP Guidelines
Clinical Validation of Copy Number Variants Using the AMP GuidelinesClinical Validation of Copy Number Variants Using the AMP Guidelines
Clinical Validation of Copy Number Variants Using the AMP Guidelines
 
To infinity and beyond v2
To infinity and beyond v2To infinity and beyond v2
To infinity and beyond v2
 
Clinical Prediction Rules
Clinical Prediction RulesClinical Prediction Rules
Clinical Prediction Rules
 
The challenge of small data
The challenge of small dataThe challenge of small data
The challenge of small data
 

More from scanFOAM

Scalpels and Stories - rediscoverin narrative in medicinen - Matt Morgan - TBS24
Scalpels and Stories - rediscoverin narrative in medicinen - Matt Morgan - TBS24Scalpels and Stories - rediscoverin narrative in medicinen - Matt Morgan - TBS24
Scalpels and Stories - rediscoverin narrative in medicinen - Matt Morgan - TBS24
scanFOAM
 
Mechanical ventilation in PARDS - same as adults? - Demirakca - TBS24tion_in_...
Mechanical ventilation in PARDS - same as adults? - Demirakca - TBS24tion_in_...Mechanical ventilation in PARDS - same as adults? - Demirakca - TBS24tion_in_...
Mechanical ventilation in PARDS - same as adults? - Demirakca - TBS24tion_in_...
scanFOAM
 
Unmanned aerial systems "drones" - increasing SAR response capability - Will ...
Unmanned aerial systems "drones" - increasing SAR response capability - Will ...Unmanned aerial systems "drones" - increasing SAR response capability - Will ...
Unmanned aerial systems "drones" - increasing SAR response capability - Will ...
scanFOAM
 

More from scanFOAM (20)

Reframing shock physiology - a tale of 3 pressures - Sara Crager - TBS24
Reframing shock physiology - a tale of 3 pressures - Sara Crager - TBS24Reframing shock physiology - a tale of 3 pressures - Sara Crager - TBS24
Reframing shock physiology - a tale of 3 pressures - Sara Crager - TBS24
 
Manual pressure augmentation in OHCA - David Anderson - TBS24
Manual pressure augmentation in OHCA - David Anderson - TBS24Manual pressure augmentation in OHCA - David Anderson - TBS24
Manual pressure augmentation in OHCA - David Anderson - TBS24
 
Scalpels and Stories - rediscoverin narrative in medicinen - Matt Morgan - TBS24
Scalpels and Stories - rediscoverin narrative in medicinen - Matt Morgan - TBS24Scalpels and Stories - rediscoverin narrative in medicinen - Matt Morgan - TBS24
Scalpels and Stories - rediscoverin narrative in medicinen - Matt Morgan - TBS24
 
Whole blood for trauma haemorrhage - UK experience - Laura Green - TBS24
Whole blood for trauma haemorrhage - UK experience - Laura Green - TBS24Whole blood for trauma haemorrhage - UK experience - Laura Green - TBS24
Whole blood for trauma haemorrhage - UK experience - Laura Green - TBS24
 
TBI and CV dysfunction - Flora Bird - TBS24
TBI and CV dysfunction - Flora Bird - TBS24TBI and CV dysfunction - Flora Bird - TBS24
TBI and CV dysfunction - Flora Bird - TBS24
 
POCUS in the Big Sick - Chris Yap - TBS24
POCUS in the Big Sick - Chris Yap - TBS24POCUS in the Big Sick - Chris Yap - TBS24
POCUS in the Big Sick - Chris Yap - TBS24
 
How kissing a frog can save your life - Matt Morgan - TBS24
How kissing a frog can save your life - Matt Morgan - TBS24How kissing a frog can save your life - Matt Morgan - TBS24
How kissing a frog can save your life - Matt Morgan - TBS24
 
Fully Automated CPR - van der Velde - TBS"4
Fully Automated CPR - van der Velde - TBS"4Fully Automated CPR - van der Velde - TBS"4
Fully Automated CPR - van der Velde - TBS"4
 
ECPR at the Roadside - Mamoun Abu-Habsa - TBS24
ECPR at the Roadside - Mamoun Abu-Habsa - TBS24ECPR at the Roadside - Mamoun Abu-Habsa - TBS24
ECPR at the Roadside - Mamoun Abu-Habsa - TBS24
 
Mechanical ventilation in PARDS - same as adults? - Demirakca - TBS24tion_in_...
Mechanical ventilation in PARDS - same as adults? - Demirakca - TBS24tion_in_...Mechanical ventilation in PARDS - same as adults? - Demirakca - TBS24tion_in_...
Mechanical ventilation in PARDS - same as adults? - Demirakca - TBS24tion_in_...
 
Failure is an option - journey of an astronaut candidate - Matthieu Komorowsk...
Failure is an option - journey of an astronaut candidate - Matthieu Komorowsk...Failure is an option - journey of an astronaut candidate - Matthieu Komorowsk...
Failure is an option - journey of an astronaut candidate - Matthieu Komorowsk...
 
Unmanned aerial systems "drones" - increasing SAR response capability - Will ...
Unmanned aerial systems "drones" - increasing SAR response capability - Will ...Unmanned aerial systems "drones" - increasing SAR response capability - Will ...
Unmanned aerial systems "drones" - increasing SAR response capability - Will ...
 
Experience learning - lessons from 25 years of ATACC - Mark Forrest and Halde...
Experience learning - lessons from 25 years of ATACC - Mark Forrest and Halde...Experience learning - lessons from 25 years of ATACC - Mark Forrest and Halde...
Experience learning - lessons from 25 years of ATACC - Mark Forrest and Halde...
 
Groupthink - lessons from the Challenger disaster - Vahé Ender - TBS24
Groupthink - lessons from the Challenger disaster - Vahé Ender - TBS24Groupthink - lessons from the Challenger disaster - Vahé Ender - TBS24
Groupthink - lessons from the Challenger disaster - Vahé Ender - TBS24
 
Precision in neonatal transport - Ian Braithwaite - TBS24
Precision in neonatal transport - Ian Braithwaite - TBS24Precision in neonatal transport - Ian Braithwaite - TBS24
Precision in neonatal transport - Ian Braithwaite - TBS24
 
Mantas Okas - where do we come from and where can we go if we feel like?
Mantas Okas - where do we come from and where can we go if we feel like?Mantas Okas - where do we come from and where can we go if we feel like?
Mantas Okas - where do we come from and where can we go if we feel like?
 
The Shock Continuum - Sara Crager - TBS24
The Shock Continuum - Sara Crager - TBS24The Shock Continuum - Sara Crager - TBS24
The Shock Continuum - Sara Crager - TBS24
 
Shock Continuum - Sara Crager - TBS24.pdf
Shock Continuum - Sara Crager - TBS24.pdfShock Continuum - Sara Crager - TBS24.pdf
Shock Continuum - Sara Crager - TBS24.pdf
 
Fully Automated CPR | Jason van der Velde | TBS24
Fully Automated CPR | Jason van der Velde | TBS24Fully Automated CPR | Jason van der Velde | TBS24
Fully Automated CPR | Jason van der Velde | TBS24
 
The future of the emergency room | Jean-Louis Vincent at TBS23
The future of the emergency room | Jean-Louis Vincent at TBS23The future of the emergency room | Jean-Louis Vincent at TBS23
The future of the emergency room | Jean-Louis Vincent at TBS23
 

Recently uploaded

Alcohol_Dr. Jeenal Mistry MD Pharmacology.pdf
Alcohol_Dr. Jeenal Mistry MD Pharmacology.pdfAlcohol_Dr. Jeenal Mistry MD Pharmacology.pdf
Alcohol_Dr. Jeenal Mistry MD Pharmacology.pdf
Dr Jeenal Mistry
 

Recently uploaded (20)

Final CAPNOCYTOPHAGA INFECTION by Gauri Gawande.pptx
Final CAPNOCYTOPHAGA INFECTION by Gauri Gawande.pptxFinal CAPNOCYTOPHAGA INFECTION by Gauri Gawande.pptx
Final CAPNOCYTOPHAGA INFECTION by Gauri Gawande.pptx
 
TEST BANK For Wong’s Essentials of Pediatric Nursing, 11th Edition by Marilyn...
TEST BANK For Wong’s Essentials of Pediatric Nursing, 11th Edition by Marilyn...TEST BANK For Wong’s Essentials of Pediatric Nursing, 11th Edition by Marilyn...
TEST BANK For Wong’s Essentials of Pediatric Nursing, 11th Edition by Marilyn...
 
US E-cigarette Summit: Taming the nicotine industrial complex
US E-cigarette Summit: Taming the nicotine industrial complexUS E-cigarette Summit: Taming the nicotine industrial complex
US E-cigarette Summit: Taming the nicotine industrial complex
 
Ocular injury ppt Upendra pal optometrist upums saifai etawah
Ocular injury  ppt  Upendra pal  optometrist upums saifai etawahOcular injury  ppt  Upendra pal  optometrist upums saifai etawah
Ocular injury ppt Upendra pal optometrist upums saifai etawah
 
Surat @ℂall @Girls ꧁❤8527049040❤꧂@ℂall @Girls Service Vip Top Model Safe
Surat @ℂall @Girls ꧁❤8527049040❤꧂@ℂall @Girls Service Vip Top Model SafeSurat @ℂall @Girls ꧁❤8527049040❤꧂@ℂall @Girls Service Vip Top Model Safe
Surat @ℂall @Girls ꧁❤8527049040❤꧂@ℂall @Girls Service Vip Top Model Safe
 
DECIPHERING COMMON ECG FINDINGS IN ED.pptx
DECIPHERING COMMON ECG FINDINGS IN ED.pptxDECIPHERING COMMON ECG FINDINGS IN ED.pptx
DECIPHERING COMMON ECG FINDINGS IN ED.pptx
 
linearity concept of significance, standard deviation, chi square test, stude...
linearity concept of significance, standard deviation, chi square test, stude...linearity concept of significance, standard deviation, chi square test, stude...
linearity concept of significance, standard deviation, chi square test, stude...
 
Multiple sclerosis diet.230524.ppt3.pptx
Multiple sclerosis diet.230524.ppt3.pptxMultiple sclerosis diet.230524.ppt3.pptx
Multiple sclerosis diet.230524.ppt3.pptx
 
Couples presenting to the infertility clinic- Do they really have infertility...
Couples presenting to the infertility clinic- Do they really have infertility...Couples presenting to the infertility clinic- Do they really have infertility...
Couples presenting to the infertility clinic- Do they really have infertility...
 
"Central Hypertension"‚ in China: Towards the nation-wide use of SphygmoCor t...
"Central Hypertension"‚ in China: Towards the nation-wide use of SphygmoCor t..."Central Hypertension"‚ in China: Towards the nation-wide use of SphygmoCor t...
"Central Hypertension"‚ in China: Towards the nation-wide use of SphygmoCor t...
 
Is preeclampsia and spontaneous preterm delivery associate with vascular and ...
Is preeclampsia and spontaneous preterm delivery associate with vascular and ...Is preeclampsia and spontaneous preterm delivery associate with vascular and ...
Is preeclampsia and spontaneous preterm delivery associate with vascular and ...
 
Presentació "Advancing Emergency Medicine Education through Virtual Reality"
Presentació "Advancing Emergency Medicine Education through Virtual Reality"Presentació "Advancing Emergency Medicine Education through Virtual Reality"
Presentació "Advancing Emergency Medicine Education through Virtual Reality"
 
Alcohol_Dr. Jeenal Mistry MD Pharmacology.pdf
Alcohol_Dr. Jeenal Mistry MD Pharmacology.pdfAlcohol_Dr. Jeenal Mistry MD Pharmacology.pdf
Alcohol_Dr. Jeenal Mistry MD Pharmacology.pdf
 
Why invest into infodemic management in health emergencies
Why invest into infodemic management in health emergenciesWhy invest into infodemic management in health emergencies
Why invest into infodemic management in health emergencies
 
Temporal, Infratemporal & Pterygopalatine BY Dr.RIG.pptx
Temporal, Infratemporal & Pterygopalatine BY Dr.RIG.pptxTemporal, Infratemporal & Pterygopalatine BY Dr.RIG.pptx
Temporal, Infratemporal & Pterygopalatine BY Dr.RIG.pptx
 
Impact of cancers therapies on the loss in cardiac function, myocardial fffic...
Impact of cancers therapies on the loss in cardiac function, myocardial fffic...Impact of cancers therapies on the loss in cardiac function, myocardial fffic...
Impact of cancers therapies on the loss in cardiac function, myocardial fffic...
 
Retinal consideration in cataract surgery
Retinal consideration in cataract surgeryRetinal consideration in cataract surgery
Retinal consideration in cataract surgery
 
Antiplatelets in IHD, Dose Duration, DAPT vs SAPT
Antiplatelets in IHD, Dose Duration, DAPT vs SAPTAntiplatelets in IHD, Dose Duration, DAPT vs SAPT
Antiplatelets in IHD, Dose Duration, DAPT vs SAPT
 
The History of Diagnostic Medical imaging
The History of Diagnostic Medical imagingThe History of Diagnostic Medical imaging
The History of Diagnostic Medical imaging
 
PT MANAGEMENT OF URINARY INCONTINENCE.pptx
PT MANAGEMENT OF URINARY INCONTINENCE.pptxPT MANAGEMENT OF URINARY INCONTINENCE.pptx
PT MANAGEMENT OF URINARY INCONTINENCE.pptx
 

Big data vs the RCT - Derek Angus - SSAI2017

  • 1. CRISMA Center, Department of Critical Care Medicine Department of Health Policy and Management McGowan Institute for Regenerative Medicine Clinical and Translational Science Institute University of Pittsburgh Schools of the Health Sciences Derek C. Angus, MD, MPH, FRCP
  • 2. CRISMA Center, Department of Critical Care Medicine Department of Health Policy and Management McGowan Institute for Regenerative Medicine Clinical and Translational Science Institute University of Pittsburgh Schools of the Health Sciences Derek C. Angus, MD, MPH, FRCP Big Data vs. the RCT?
  • 3. C  R  I  S  M  A Evidence-based medicine “Doctors must base what they do on randomized clinical trials (RCTs)” Archie Cochrane
  • 4. C  R  I  S  M  A How we know something works ... Patients with disease X
  • 5. C  R  I  S  M  A How we know something works ... Patients with disease X
  • 6. C  R  I  S  M  A How we know something works ... Patients with disease X
  • 7. C  R  I  S  M  A How we know something works ... Patients with disease X
  • 8. C  R  I  S  M  A How we know something works ... Patients with disease X
  • 9. C  R  I  S  M  A
  • 10. C  R  I  S  M  A The good news … •We now test many ideas with RCTs - 37,000 started in 2010 … -All FDA drug and device approvals •We now conduct RCTs very well - Methodologic conduct - Ethical oversight - Reporting
  • 11. C  R  I  S  M  A But … • RCTs are too narrow • Cherry-picked population; not the real world • RCTs are too broad • No data on treatment effects across patients • No comparative effectiveness • Rx A vs. B is not very helpful • What about A vs. B, vs. C, vs. D … etc. • Depending on whether I give E or F … • Having said all that, I just want the answer … • I don’t want my patient to be a guinea pig …
  • 12. C  R  I  S  M  A Clinical care Clinical research Parallel universes …
  • 13. C  R  I  S  M  A Enter the era of ‘Big Data’ • Integration of ‘deep’ personalized data • Causal inferences on optimal care • Broad – ‘real-world’ practice • Narrow – ‘personal’ estimates • Comparative – considers all options • Vanderbilt-IBM ‘BioVU’ initiative • ‘Live’ presentation of information at time of clinical decision-making • ‘Just-in-time’ cohort study in EHR • No guinea pigs • Longhurst et al. Health Affairs 2013
  • 14. C  R  I  S  M  A
  • 15. Feature Leverage the EHR Low incremental costs Real-world ‘effectiveness’ Consider multiple therapies ‘Personalized’ estimates Offer ‘live’ tailored options Robust causal inference ‘Big Data’ Analytics ✔ ✗ ✔ ✔ ✔ ✔ ✔ Evidence Generator Report Card
  • 16. C  R  I  S  M  A Point-of-care (POC) Clinical Trials • A clinical moment in the EHR ‘alerts’ the clinical trial machinery • VA EHR • In-patient diabetics with poor glucose control • When physician placed insulin order in CPOE system … • Opportunity to randomize • Sliding scale • Weight-based algorithm • Fiore et al. Clinical Trials 2011 • Targeting the large ‘pragmatic’ trial arena • 2 thiazide diuretics in >13k high BP Veterans (NCT02185417) • 2 aspirin doses in 20k CVD patients (ADAPTABLE) (PCORI)
  • 17. Feature Leverage the EHR Low incremental costs Real-world ‘effectiveness’ Consider multiple therapies ‘Personalized’ estimates Offer ‘live’ tailored options Robust causal inference ‘Big Data’ Analytics POC Trials ✔ ✗/✔ ✔ ✔ ✔ ✗ ✗ ✗ ✔ ✔ ✔ ✔ ✔ ✔ Evidence Generator Report Card
  • 18. C  R  I  S  M  A Platform Trials • Adaptive trials • Focus on disease, not a particular Rx • Multiple interventions (arms) • ‘Perpetual’ enrollment • Often based on Bayes’ theorem • Tailor choices over time Berry et al JAMA 2015 • Focus on pre-approval space • Emphasis on efficiency with (very) small sample sizes • Different therapies ‘graduate’ to next phase while trial continues Woodcock and Lavange NEJM 2017
  • 19. C  R  I  S  M  A
  • 20. C  R  I  S  M  A Response-adaptive randomization Rugo et al. NEJM 2016
  • 21. C  R  I  S  M  A The traditional RCT ... Patients with disease X At the start, 50% chance that A > B
  • 22. C  R  I  S  M  A The traditional RCT ... Patients with disease X At the end, >99% sure that A > B What about in the middle?
  • 23. C  R  I  S  M  A A planned trial of A vs. B in 400 patients The probability that A > B = 78% Start randomizing MORE patients to A than B … Alive Dead 40 20 No. of patients A B After 40 enrolled ….
  • 24. C  R  I  S  M  A After 80 patients … Now, the probability that A > B = 99.9% Stop the trial! Alive Dead 40 20 No. of patients A B
  • 25. C  R  I  S  M  A Caveats 1. If the ‘second’ 40 was flat or opposite direction … • Trial continues and the next ‘bet’ swings back closer to 50:50 2. When only 2 groups, power still driven by the smaller group • So, NOT very helpful if … • Single homogenous cohort • Two arms • But, becomes VERY interesting when … • Multiple arms • Multiple subgroups
  • 27. B C Statistical modelRandomization rule Response-adaptive randomization A Odds weighted towards best RX
  • 28. B C Statistical modelRandomization rule Response-adaptive randomization A D New arms activated
  • 30. C A Statistical modelRandomization rule Response-adaptive randomization Different weights for different patient groups D
  • 31. Feature Leverage the EHR Low incremental costs Real-world ‘effectiveness’ Consider multiple therapies ‘Personalized’ estimates Offer ‘live’ tailored options Robust causal inference ‘Big Data’ Analytics POC Trials Platform Trials ✔ ✗/ ✔ ✗ ✗ ✔ ✔ ✔ ✔ ✗ ✔ ✔ ✔ ✗ ✗ ✗ ✔ ✔ ✔ ✔ ✔ ✔ Evidence Generator Report Card
  • 32. Feature Leverage the EHR Low incremental costs Real-world ‘effectiveness’ Consider multiple therapies ‘Personalized’ estimates Offer ‘live’ tailored options Robust causal inference ‘Big Data’ Analytics POC Trials Platform Trials ??? ✔ ✗/✔ ✗ ✗ ✔ ✔ ✔ ✔ ✗ ✔ ✔ ✔ ✗ ✗ ✗ ✔ ✔ ✔ ✔ ✔ ✔ ✔ ✔ ✔ ✔ ✔ ✔ ✔ Evidence Generator Report Card
  • 33. C  R  I  S  M  A A novel blend of ‘POC’ + platform designs •REMAP •Randomized •Embedded •Multifactorial •Adaptive •Platform trial REMAP ✔ ✔ ✔ ✔ ✔ ✔ ✔ Angus DC. JAMA 2015
  • 34. C  R  I  S  M  A • Funding • EU FP7 PREPARE WP 5 program (25M euro) • Australian NHMRC ‘OPTIMISE’ program ($6M) • New Zealand NHMRC ($2M) • Simultaneously test • Different anti-microbial strategies • Different host immunomodulation strategies • Different ventilation strategies • Separate RAR and stopping rules for multiple subgroups Angus DC. JAMA 2015
  • 35. C  R  I  S  M  A Angus DC. JAMA 2015 • Patients are preferentially assigned to best performing arm • Allocation is random, but NOT 50:50 • Odds of assignment proportional to odds of success • Not a guinea pig! • Embedded • ICU admission orders • Approved in Netherlands and New Zealand with delayed consent
  • 36. C  R  I  S  M  A REMAP-CAP elements • Domain – an area where a question is asked … • Domain #1 – choice of antibiotic • Domain #2 – whether to give steroids or not • Domain #3 – whether to extend macrolide or not • Domain #4 – choice of ventilator strategy • Domain #5 – oxygen titration strategy • Etc. …. • Intervention • Any option within a domain … • Regimen • Unique combination of interventions within a domain … • Stratum • Baseline subgroup • Ex. shock or not
  • 37. Multifactorial intervention assignments Regimen = set of domain-specific interventions Effect of an intervention is conditional upon • Stratum • Interventions within other domains Regimen Domain A Domain B Domain C #1 A1 B1 C1 #2 A1 B1 C2 #3 A1 B2 C1 #4 A1 B2 C2 #5 A2 B1 C1 ….. #n An Bn Cn
  • 38. Multifactorial intervention assignments Regimen = set of domain-specific interventions Effect of an intervention is conditional upon • Stratum • Interventions within other domainsRegimen Domain A Domain B Domain C #1 A1 B1 C1 #2 A1 B1 C2 #3 A1 B2 C1 #4 A1 B2 C2 #5 A2 B1 C1 ….. #n An Bn Cn Pre-specified architecture determined by • Choice of domains, strata, etc. • Choice of potential interactions Choices inform a Bayesian inference model • Pre-trial simulations evaluate performance Each external adaptation (ex. new domain) • Modify elements in Bayesian model • Re-simulate before ‘live’ deployment Pre-trial design and construction
  • 39. Pre-specified architecture determined by • Choice of domains, strata, etc. • Choice of potential interactions Choices inform a Bayesian inference model • Pre-trial simulations evaluate performance Each external adaptation (ex. new domain) • Modify elements in Bayesian model • Re-simulate before ‘live’ deployment Pre-trial design and construction
  • 40. Z = g[ ]+ R[ ]+ Time[ ]+ Interv[ ]+ Inter,Shock[ ]+ Interv,Hypox[ ]+ Interv,Interv[ ]
  • 41. Multifactorial intervention assignments Regimen = set of domain-specific interventions Effect of an intervention is conditional upon • Stratum • Interventions within other domainsRegimen Domain A Domain B Domain C #1 A1 B1 C1 #2 A1 B1 C2 #3 A1 B2 C1 #4 A1 B2 C2 #5 A2 B1 C1 ….. #n An Bn Cn Pre-specified architecture determined by • Choice of domains, strata, etc. • Choice of potential interactions Choices inform a Bayesian inference model • Pre-trial simulations evaluate performance Each external adaptation (ex. new domain) • Modify elements in Bayesian model • Re-simulate before ‘live’ deployment Pre-trial design and construction
  • 42. Statistical trigger Steering Committee can • Add strata, domains & interventions DSMB can • Request new external data be incorporated in priors • Overrule statistical triggers Multifactorial intervention assignments Regimen = set of domain-specific interventions Effect of an intervention is conditional upon • Stratum • Interventions within other domainsRegimen Domain A Domain B Domain C #1 A1 B1 C1 #2 A1 B1 C2 #3 A1 B2 C1 #4 A1 B2 C2 #5 A2 B1 C1 ….. #n An Bn Cn Result declared when, within stratum, an intervention is • Superior >99% likely to be best • Equivalent >90% likely to be D<3% • Inferior <1% likely to be best Pre-specified architecture determined by • Choice of domains, strata, etc. • Choice of potential interactions Choices inform a Bayesian inference model • Pre-trial simulations evaluate performance Each external adaptation (ex. new domain) • Modify elements in Bayesian model • Re-simulate before ‘live’ deployment Pre-trial design and construction Patients Presumed severe CAP • Different strata (ex. shock or not) • Collected at sites • Managed at regional data centers • Merged at central statistical center Data collection Re-estimate Bayesian inference model with new data to update probabilities Update and adapt External adaptations
  • 43. Statistical trigger Steering Committee can • Add strata, domains & interventions DSMB can • Request new external data be incorporated in priors • Overrule statistical triggers Multifactorial intervention assignments Regimen = set of domain-specific interventions Effect of an intervention is conditional upon • Stratum • Interventions within other domains Embedding Patient identification and enrollment • Tied to clinical ‘point-of-care’ Randomized interventions • Issued as ‘order set’ regimen EHR embedding • Screen and flag patient • Consent documentation • Generate regimen order set • Flag downstream states • Data collection Regimen Domain A Domain B Domain C #1 A1 B1 C1 #2 A1 B1 C2 #3 A1 B2 C1 #4 A1 B2 C2 #5 A2 B1 C1 ….. #n An Bn Cn Result declared when, within stratum, an intervention is • Superior >99% likely to be best • Equivalent >90% likely to be D<3% • Inferior <1% likely to be best Pre-specified architecture determined by • Choice of domains, strata, etc. • Choice of potential interactions Choices inform a Bayesian inference model • Pre-trial simulations evaluate performance Each external adaptation (ex. new domain) • Modify elements in Bayesian model • Re-simulate before ‘live’ deployment Pre-trial design and construction Patients Presumed severe CAP • Different strata (ex. shock or not) • Collected at sites • Managed at regional data centers • Merged at central statistical center Data collection Re-estimate Bayesian inference model with new data to update probabilities Update and adapt External adaptations
  • 44. Statistical trigger • Launch with initial weights • Update based on new probabilities Steering Committee can • Add strata, domains & interventions DSMB can • Request new external data be incorporated in priors • Overrule statistical triggers Multifactorial intervention assignments Regimen = set of domain-specific interventions Effect of an intervention is conditional upon • Stratum • Interventions within other domains Embedding Patient identification and enrollment • Tied to clinical ‘point-of-care’ Randomized interventions • Issued as ‘order set’ regimen EHR embedding • Screen and flag patient • Consent documentation • Generate regimen order set • Flag downstream states • Data collection Regimen Domain A Domain B Domain C #1 A1 B1 C1 #2 A1 B1 C2 #3 A1 B2 C1 #4 A1 B2 C2 #5 A2 B1 C1 ….. #n An Bn Cn Result declared when, within stratum, an intervention is • Superior >99% likely to be best • Equivalent >90% likely to be D<3% • Inferior <1% likely to be best Pre-specified architecture determined by • Choice of domains, strata, etc. • Choice of potential interactions Choices inform a Bayesian inference model • Pre-trial simulations evaluate performance Each external adaptation (ex. new domain) • Modify elements in Bayesian model • Re-simulate before ‘live’ deployment Pre-trial design and construction Patients Presumed severe CAP • Different strata (ex. shock or not) • Collected at sites • Managed at regional data centers • Merged at central statistical center Data collection Re-estimate Bayesian inference model with new data to update probabilities Update and adapt Response-adaptive randomization External adaptations
  • 45. Statistical trigger • Launch with initial weights • Update based on new probabilities Steering Committee can • Add strata, domains & interventions DSMB can • Request new external data be incorporated in priors • Overrule statistical triggers Multifactorial intervention assignments Regimen = set of domain-specific interventions Effect of an intervention is conditional upon • Stratum • Interventions within other domains Embedding Patient identification and enrollment • Tied to clinical ‘point-of-care’ Randomized interventions • Issued as ‘order set’ regimen EHR embedding • Screen and flag patient • Consent documentation • Generate regimen order set • Flag downstream states • Data collection Regimen Domain A Domain B Domain C #1 A1 B1 C1 #2 A1 B1 C2 #3 A1 B2 C1 #4 A1 B2 C2 #5 A2 B1 C1 ….. #n An Bn Cn Result declared when, within stratum, an intervention is • Superior >99% likely to be best • Equivalent >90% likely to be D<3% • Inferior <1% likely to be best Pre-specified architecture determined by • Choice of domains, strata, etc. • Choice of potential interactions Choices inform a Bayesian inference model • Pre-trial simulations evaluate performance Each external adaptation (ex. new domain) • Modify elements in Bayesian model • Re-simulate before ‘live’ deployment Pre-trial design and construction Patients Presumed severe CAP • Different strata (ex. shock or not) • Collected at sites • Managed at regional data centers • Merged at central statistical center Data collection Re-estimate Bayesian inference model with new data to update probabilities Update and adapt Response-adaptive randomization External adaptations
  • 46. Statistical trigger • Launch with initial weights • Update based on new probabilities Steering Committee can • Add strata, domains & interventions DSMB can • Request new external data be incorporated in priors • Overrule statistical triggers Multifactorial intervention assignments Regimen = set of domain-specific interventions Effect of an intervention is conditional upon • Stratum • Interventions within other domains Embedding Patient identification and enrollment • Tied to clinical ‘point-of-care’ Randomized interventions • Issued as ‘order set’ regimen EHR embedding • Screen and flag patient • Consent documentation • Generate regimen order set • Flag downstream states • Data collection Regimen Domain A Domain B Domain C #1 A1 B1 C1 #2 A1 B1 C2 #3 A1 B2 C1 #4 A1 B2 C2 #5 A2 B1 C1 ….. #n An Bn Cn Result declared when, within stratum, an intervention is • Superior >99% likely to be best • Equivalent >90% likely to be D<3% • Inferior <1% likely to be best Pre-specified architecture determined by • Choice of domains, strata, etc. • Choice of potential interactions Choices inform a Bayesian inference model • Pre-trial simulations evaluate performance Each external adaptation (ex. new domain) • Modify elements in Bayesian model • Re-simulate before ‘live’ deployment Pre-trial design and construction Patients Presumed severe CAP • Different strata (ex. shock or not) • Collected at sites • Managed at regional data centers • Merged at central statistical center Data collection Re-estimate Bayesian inference model with new data to update probabilities Update and adapt Response-adaptive randomization External adaptations
  • 47. Statistical trigger • Launch with initial weights • Update based on new probabilities Steering Committee can • Add strata, domains & interventions DSMB can • Request new external data be incorporated in priors • Overrule statistical triggers Multifactorial intervention assignments Regimen = set of domain-specific interventions Effect of an intervention is conditional upon • Stratum • Interventions within other domains Embedding Patient identification and enrollment • Tied to clinical ‘point-of-care’ Randomized interventions • Issued as ‘order set’ regimen EHR embedding • Screen and flag patient • Consent documentation • Generate regimen order set • Flag downstream states • Data collection Regimen Domain A Domain B Domain C #1 A1 B1 C1 #2 A1 B1 C2 #3 A1 B2 C1 #4 A1 B2 C2 #5 A2 B1 C1 ….. #n An Bn Cn Result declared when, within stratum, an intervention is • Superior >99% likely to be best • Equivalent >90% likely to be D<3% • Inferior <1% likely to be best Pre-specified architecture determined by • Choice of domains, strata, etc. • Choice of potential interactions Choices inform a Bayesian inference model • Pre-trial simulations evaluate performance Each external adaptation (ex. new domain) • Modify elements in Bayesian model • Re-simulate before ‘live’ deployment Pre-trial design and construction Patients Presumed severe CAP • Different strata (ex. shock or not) • Collected at sites • Managed at regional data centers • Merged at central statistical center Data collection Re-estimate Bayesian inference model with new data to update probabilities Update and adapt Response-adaptive randomization External adaptations
  • 48. Statistical trigger • Launch with initial weights • Update based on new probabilities Multifactorial intervention assignments Regimen = set of domain-specific interventions Effect of an intervention is conditional upon • Stratum • Interventions within other domains Embedding Patient identification and enrollment • Tied to clinical ‘point-of-care’ Randomized interventions • Issued as ‘order set’ regimen EHR embedding • Screen and flag patient • Consent documentation • Generate regimen order set • Flag downstream states • Data collection Regimen Domain A Domain B Domain C #1 A1 B1 C1 #2 A1 B1 C2 #3 A1 B2 C1 #4 A1 B2 C2 #5 A2 B1 C1 ….. #n An Bn Cn Result declared when, within stratum, an intervention is • Superior >99% likely to be best • Equivalent >90% likely to be D<3% • Inferior <1% likely to be best Pre-specified architecture determined by • Choice of domains, strata, etc. • Choice of potential interactions Choices inform a Bayesian inference model • Pre-trial simulations evaluate performance Each external adaptation (ex. new domain) • Modify elements in Bayesian model • Re-simulate before ‘live’ deployment Pre-trial design and construction Patients Presumed severe CAP • Different strata (ex. shock or not) • Collected at sites • Managed at regional data centers • Merged at central statistical center Data collection Re-estimate Bayesian inference model with new data to update probabilities Update and adapt Response-adaptive randomization
  • 49. Statistical trigger • Launch with initial weights • Update based on new probabilities Steering Committee can • Add strata, domains & interventions DSMB can • Request new external data be incorporated in priors • Overrule statistical triggers Multifactorial intervention assignments Regimen = set of domain-specific interventions Effect of an intervention is conditional upon • Stratum • Interventions within other domains Embedding Patient identification and enrollment • Tied to clinical ‘point-of-care’ Randomized interventions • Issued as ‘order set’ regimen EHR embedding • Screen and flag patient • Consent documentation • Generate regimen order set • Flag downstream states • Data collection Regimen Domain A Domain B Domain C #1 A1 B1 C1 #2 A1 B1 C2 #3 A1 B2 C1 #4 A1 B2 C2 #5 A2 B1 C1 ….. #n An Bn Cn Result declared when, within stratum, an intervention is • Superior >99% likely to be best • Equivalent >90% likely to be D<3% • Inferior <1% likely to be best Pre-specified architecture determined by • Choice of domains, strata, etc. • Choice of potential interactions Choices inform a Bayesian inference model • Pre-trial simulations evaluate performance Each external adaptation (ex. new domain) • Modify elements in Bayesian model • Re-simulate before ‘live’ deployment Pre-trial design and construction Patients Presumed severe CAP • Different strata (ex. shock or not) • Collected at sites • Managed at regional data centers • Merged at central statistical center Data collection Re-estimate Bayesian inference model with new data to update probabilities Update and adapt Response-adaptive randomization External adaptations
  • 50. C  R  I  S  M  A REMAP severe pneumonia … • Gets closer to individualized treatment decisions … • For example, should my patient receive IV hydrocortisone? • Depends on • Whether shock is present • How sick (hypoxic) the patient is • Whether underlying cause is viral or not • Whether an anti-viral is being administered • Whether other strategies are being used that may minimize lung injury and inflammation (protective vs. ultraprotective ventilation) • Separate probability estimate for each consideration … • Trial enrolls until a predefined level of certainty • As soon as one question hits threshold, answer is announced
  • 51. C  R  I  S  M  A Run the trial ‘in silico’ ahead of time … • Monte-Carlo simulations • Run 1,000s of times under different scenarios
  • 52. ‘True’ mortality Average results from 1,000s of simulations 80 fewer deaths; higher power Scenario #1: 2 of 8 regimens are best
  • 53. ‘True’ mortality Average results from 1,000s of simulations Similar power but 80 fewer deaths Scenario #1: 2 of 8 regimens are best
  • 54. ‘True’ mortality Average results from 1,000s of simulations Scenario #2: One regimen is best
  • 55. ‘True’ mortality Average results from 1,000s of simulations 94 fewer deaths; higher power Scenario #2: One regimen is best
  • 56. C  R  I  S  M  A Ok, but … • EHR data quality • Institutional commitment • Ethics • Statistics and design • Reporting and dissemination of results • Funding • Oversight • Integration with other clinical research programs
  • 57. C  R  I  S  M  A Conclusions • RCTs remain our strongest ‘truth’ finder • But, current RCT enterprise LETS US DOWN • ‘Big Data’ should not be cast as an alternative to the RCT • This is a false choice • Instead, the digital age enables novel RCTs designs • Smarter and safer
  • 58. C  R  I  S  M  A Self-learning healthcare is … fused care and research