The document describes a novel platform trial design called REMAP (Randomized, Embedded, Multifactorial, Adaptive Platform) that aims to efficiently test multiple interventions for critical illness. It utilizes a point-of-care embedded design within electronic health records to rapidly enroll patients and assign multifactorial intervention regimens based on a Bayesian statistical model. The model continuously updates probabilities of intervention effectiveness based on accumulating trial data and can trigger results when an intervention is found to be superior, equivalent, or inferior for a given patient subgroup. This allows the trial to efficiently evaluate and adapt multiple treatment options in a real-world intensive care setting.
1. CRISMA Center, Department of Critical Care Medicine
Department of Health Policy and Management
McGowan Institute for Regenerative Medicine
Clinical and Translational Science Institute
University of Pittsburgh Schools of the Health Sciences
Derek C. Angus, MD, MPH, FRCP
2. CRISMA Center, Department of Critical Care Medicine
Department of Health Policy and Management
McGowan Institute for Regenerative Medicine
Clinical and Translational Science Institute
University of Pittsburgh Schools of the Health Sciences
Derek C. Angus, MD, MPH, FRCP
Big Data vs. the RCT?
3. C R I S M A
Evidence-based medicine
“Doctors must base what they do on
randomized clinical trials (RCTs)”
Archie Cochrane
4. C R I S M A
How we know something works ...
Patients with
disease X
5. C R I S M A
How we know something works ...
Patients with
disease X
6. C R I S M A
How we know something works ...
Patients with
disease X
7. C R I S M A
How we know something works ...
Patients with
disease X
8. C R I S M A
How we know something works ...
Patients with
disease X
10. C R I S M A
The good news …
•We now test many ideas with RCTs
- 37,000 started in 2010 …
-All FDA drug and device approvals
•We now conduct RCTs very well
- Methodologic conduct
- Ethical oversight
- Reporting
11. C R I S M A
But …
• RCTs are too narrow
• Cherry-picked population; not the real world
• RCTs are too broad
• No data on treatment effects across patients
• No comparative effectiveness
• Rx A vs. B is not very helpful
• What about A vs. B, vs. C, vs. D … etc.
• Depending on whether I give E or F …
• Having said all that, I just want the answer …
• I don’t want my patient to be a guinea pig …
12. C R I S M A
Clinical care Clinical research
Parallel universes …
13. C R I S M A
Enter the era of ‘Big Data’
• Integration of ‘deep’ personalized data
• Causal inferences on optimal care
• Broad – ‘real-world’ practice
• Narrow – ‘personal’ estimates
• Comparative – considers all options
• Vanderbilt-IBM ‘BioVU’ initiative
• ‘Live’ presentation of information at time
of clinical decision-making
• ‘Just-in-time’ cohort study in EHR
• No guinea pigs
• Longhurst et al. Health Affairs 2013
16. C R I S M A
Point-of-care (POC) Clinical Trials
• A clinical moment in the EHR ‘alerts’ the clinical trial machinery
• VA EHR
• In-patient diabetics with poor glucose control
• When physician placed insulin order in CPOE system …
• Opportunity to randomize
• Sliding scale
• Weight-based algorithm
• Fiore et al. Clinical Trials 2011
• Targeting the large ‘pragmatic’ trial arena
• 2 thiazide diuretics in >13k high BP Veterans (NCT02185417)
• 2 aspirin doses in 20k CVD patients (ADAPTABLE) (PCORI)
18. C R I S M A
Platform Trials
• Adaptive trials
• Focus on disease, not a particular Rx
• Multiple interventions (arms)
• ‘Perpetual’ enrollment
• Often based on Bayes’ theorem
• Tailor choices over time
Berry et al JAMA 2015
• Focus on pre-approval space
• Emphasis on efficiency with (very) small sample sizes
• Different therapies ‘graduate’ to next phase while trial continues
Woodcock and Lavange NEJM 2017
20. C R I S M A
Response-adaptive randomization
Rugo et al. NEJM 2016
21. C R I S M A
The traditional RCT ...
Patients with
disease X
At the start,
50% chance
that A > B
22. C R I S M A
The traditional RCT ...
Patients with
disease X
At the end, >99% sure that A > B
What about in the middle?
23. C R I S M A
A planned trial of A vs. B in 400 patients
The probability that A > B = 78%
Start randomizing MORE patients to A than B …
Alive
Dead
40
20
No. of
patients
A B
After 40 enrolled ….
24. C R I S M A
After 80 patients …
Now, the probability that A > B = 99.9%
Stop the trial!
Alive
Dead
40
20
No. of
patients
A B
25. C R I S M A
Caveats
1. If the ‘second’ 40 was flat or opposite direction …
• Trial continues and the next ‘bet’ swings back closer to 50:50
2. When only 2 groups, power still driven by the smaller group
• So, NOT very helpful if …
• Single homogenous cohort
• Two arms
• But, becomes VERY interesting when …
• Multiple arms
• Multiple subgroups
33. C R I S M A
A novel blend of ‘POC’ + platform designs
•REMAP
•Randomized
•Embedded
•Multifactorial
•Adaptive
•Platform trial
REMAP
✔
✔
✔
✔
✔
✔
✔
Angus DC. JAMA 2015
34. C R I S M A
• Funding
• EU FP7 PREPARE WP 5 program (25M euro)
• Australian NHMRC ‘OPTIMISE’ program ($6M)
• New Zealand NHMRC ($2M)
• Simultaneously test
• Different anti-microbial strategies
• Different host immunomodulation strategies
• Different ventilation strategies
• Separate RAR and stopping rules for multiple subgroups
Angus DC. JAMA 2015
35. C R I S M A Angus DC. JAMA 2015
• Patients are preferentially assigned to best performing arm
• Allocation is random, but NOT 50:50
• Odds of assignment proportional to odds of success
• Not a guinea pig!
• Embedded
• ICU admission orders
• Approved in Netherlands and New Zealand
with delayed consent
36. C R I S M A
REMAP-CAP elements
• Domain – an area where a question is asked …
• Domain #1 – choice of antibiotic
• Domain #2 – whether to give steroids or not
• Domain #3 – whether to extend macrolide or not
• Domain #4 – choice of ventilator strategy
• Domain #5 – oxygen titration strategy
• Etc. ….
• Intervention
• Any option within a domain …
• Regimen
• Unique combination of interventions within a domain …
• Stratum
• Baseline subgroup
• Ex. shock or not
37. Multifactorial intervention assignments
Regimen = set of domain-specific interventions
Effect of an intervention is conditional upon
• Stratum
• Interventions within other domains
Regimen Domain A Domain B Domain C
#1 A1 B1 C1
#2 A1 B1 C2
#3 A1 B2 C1
#4 A1 B2 C2
#5 A2 B1 C1
…..
#n An Bn Cn
38. Multifactorial intervention assignments
Regimen = set of domain-specific
interventions
Effect of an intervention is conditional upon
• Stratum
• Interventions within other domainsRegimen Domain A Domain B Domain C
#1 A1 B1 C1
#2 A1 B1 C2
#3 A1 B2 C1
#4 A1 B2 C2
#5 A2 B1 C1
…..
#n An Bn Cn
Pre-specified architecture determined by
• Choice of domains, strata, etc.
• Choice of potential interactions
Choices inform a Bayesian inference model
• Pre-trial simulations evaluate performance
Each external adaptation (ex. new domain)
• Modify elements in Bayesian model
• Re-simulate before ‘live’ deployment
Pre-trial design and construction
39. Pre-specified architecture determined by
• Choice of domains, strata, etc.
• Choice of potential interactions
Choices inform a Bayesian inference model
• Pre-trial simulations evaluate performance
Each external adaptation (ex. new domain)
• Modify elements in Bayesian model
• Re-simulate before ‘live’ deployment
Pre-trial design and construction
41. Multifactorial intervention assignments
Regimen = set of domain-specific
interventions
Effect of an intervention is conditional upon
• Stratum
• Interventions within other domainsRegimen Domain A Domain B Domain C
#1 A1 B1 C1
#2 A1 B1 C2
#3 A1 B2 C1
#4 A1 B2 C2
#5 A2 B1 C1
…..
#n An Bn Cn
Pre-specified architecture determined by
• Choice of domains, strata, etc.
• Choice of potential interactions
Choices inform a Bayesian inference model
• Pre-trial simulations evaluate performance
Each external adaptation (ex. new domain)
• Modify elements in Bayesian model
• Re-simulate before ‘live’ deployment
Pre-trial design and construction
42. Statistical trigger
Steering Committee can
• Add strata, domains & interventions
DSMB can
• Request new external data be incorporated in priors
• Overrule statistical triggers
Multifactorial intervention assignments
Regimen = set of domain-specific
interventions
Effect of an intervention is conditional upon
• Stratum
• Interventions within other domainsRegimen Domain A Domain B Domain C
#1 A1 B1 C1
#2 A1 B1 C2
#3 A1 B2 C1
#4 A1 B2 C2
#5 A2 B1 C1
…..
#n An Bn Cn
Result declared when, within stratum, an intervention is
• Superior >99% likely to be best
• Equivalent >90% likely to be D<3%
• Inferior <1% likely to be best
Pre-specified architecture determined by
• Choice of domains, strata, etc.
• Choice of potential interactions
Choices inform a Bayesian inference model
• Pre-trial simulations evaluate performance
Each external adaptation (ex. new domain)
• Modify elements in Bayesian model
• Re-simulate before ‘live’ deployment
Pre-trial design and construction
Patients
Presumed severe CAP
• Different strata
(ex. shock or not)
• Collected at sites
• Managed at regional data centers
• Merged at central statistical center
Data collection
Re-estimate Bayesian inference model
with new data to update probabilities
Update and adapt
External adaptations
43. Statistical trigger
Steering Committee can
• Add strata, domains & interventions
DSMB can
• Request new external data be incorporated in priors
• Overrule statistical triggers
Multifactorial intervention assignments
Regimen = set of domain-specific
interventions
Effect of an intervention is conditional upon
• Stratum
• Interventions within other domains
Embedding
Patient identification and enrollment
• Tied to clinical ‘point-of-care’
Randomized interventions
• Issued as ‘order set’ regimen
EHR embedding
• Screen and flag patient
• Consent documentation
• Generate regimen order set
• Flag downstream states
• Data collection
Regimen Domain A Domain B Domain C
#1 A1 B1 C1
#2 A1 B1 C2
#3 A1 B2 C1
#4 A1 B2 C2
#5 A2 B1 C1
…..
#n An Bn Cn
Result declared when, within stratum, an intervention is
• Superior >99% likely to be best
• Equivalent >90% likely to be D<3%
• Inferior <1% likely to be best
Pre-specified architecture determined by
• Choice of domains, strata, etc.
• Choice of potential interactions
Choices inform a Bayesian inference model
• Pre-trial simulations evaluate performance
Each external adaptation (ex. new domain)
• Modify elements in Bayesian model
• Re-simulate before ‘live’ deployment
Pre-trial design and construction
Patients
Presumed severe CAP
• Different strata
(ex. shock or not)
• Collected at sites
• Managed at regional data centers
• Merged at central statistical center
Data collection
Re-estimate Bayesian inference model
with new data to update probabilities
Update and adapt
External adaptations
44. Statistical trigger
• Launch with initial weights
• Update based on new probabilities
Steering Committee can
• Add strata, domains & interventions
DSMB can
• Request new external data be incorporated in priors
• Overrule statistical triggers
Multifactorial intervention assignments
Regimen = set of domain-specific
interventions
Effect of an intervention is conditional upon
• Stratum
• Interventions within other domains
Embedding
Patient identification and enrollment
• Tied to clinical ‘point-of-care’
Randomized interventions
• Issued as ‘order set’ regimen
EHR embedding
• Screen and flag patient
• Consent documentation
• Generate regimen order set
• Flag downstream states
• Data collection
Regimen Domain A Domain B Domain C
#1 A1 B1 C1
#2 A1 B1 C2
#3 A1 B2 C1
#4 A1 B2 C2
#5 A2 B1 C1
…..
#n An Bn Cn
Result declared when, within stratum, an intervention is
• Superior >99% likely to be best
• Equivalent >90% likely to be D<3%
• Inferior <1% likely to be best
Pre-specified architecture determined by
• Choice of domains, strata, etc.
• Choice of potential interactions
Choices inform a Bayesian inference model
• Pre-trial simulations evaluate performance
Each external adaptation (ex. new domain)
• Modify elements in Bayesian model
• Re-simulate before ‘live’ deployment
Pre-trial design and construction
Patients
Presumed severe CAP
• Different strata
(ex. shock or not)
• Collected at sites
• Managed at regional data centers
• Merged at central statistical center
Data collection
Re-estimate Bayesian inference model
with new data to update probabilities
Update and adapt
Response-adaptive randomization
External adaptations
45. Statistical trigger
• Launch with initial weights
• Update based on new probabilities
Steering Committee can
• Add strata, domains & interventions
DSMB can
• Request new external data be incorporated in priors
• Overrule statistical triggers
Multifactorial intervention assignments
Regimen = set of domain-specific
interventions
Effect of an intervention is conditional upon
• Stratum
• Interventions within other domains
Embedding
Patient identification and enrollment
• Tied to clinical ‘point-of-care’
Randomized interventions
• Issued as ‘order set’ regimen
EHR embedding
• Screen and flag patient
• Consent documentation
• Generate regimen order set
• Flag downstream states
• Data collection
Regimen Domain A Domain B Domain C
#1 A1 B1 C1
#2 A1 B1 C2
#3 A1 B2 C1
#4 A1 B2 C2
#5 A2 B1 C1
…..
#n An Bn Cn
Result declared when, within stratum, an intervention is
• Superior >99% likely to be best
• Equivalent >90% likely to be D<3%
• Inferior <1% likely to be best
Pre-specified architecture determined by
• Choice of domains, strata, etc.
• Choice of potential interactions
Choices inform a Bayesian inference model
• Pre-trial simulations evaluate performance
Each external adaptation (ex. new domain)
• Modify elements in Bayesian model
• Re-simulate before ‘live’ deployment
Pre-trial design and construction
Patients
Presumed severe CAP
• Different strata
(ex. shock or not)
• Collected at sites
• Managed at regional data centers
• Merged at central statistical center
Data collection
Re-estimate Bayesian inference model
with new data to update probabilities
Update and adapt
Response-adaptive randomization
External adaptations
46. Statistical trigger
• Launch with initial weights
• Update based on new probabilities
Steering Committee can
• Add strata, domains & interventions
DSMB can
• Request new external data be incorporated in priors
• Overrule statistical triggers
Multifactorial intervention assignments
Regimen = set of domain-specific
interventions
Effect of an intervention is conditional upon
• Stratum
• Interventions within other domains
Embedding
Patient identification and enrollment
• Tied to clinical ‘point-of-care’
Randomized interventions
• Issued as ‘order set’ regimen
EHR embedding
• Screen and flag patient
• Consent documentation
• Generate regimen order set
• Flag downstream states
• Data collection
Regimen Domain A Domain B Domain C
#1 A1 B1 C1
#2 A1 B1 C2
#3 A1 B2 C1
#4 A1 B2 C2
#5 A2 B1 C1
…..
#n An Bn Cn
Result declared when, within stratum, an intervention is
• Superior >99% likely to be best
• Equivalent >90% likely to be D<3%
• Inferior <1% likely to be best
Pre-specified architecture determined by
• Choice of domains, strata, etc.
• Choice of potential interactions
Choices inform a Bayesian inference model
• Pre-trial simulations evaluate performance
Each external adaptation (ex. new domain)
• Modify elements in Bayesian model
• Re-simulate before ‘live’ deployment
Pre-trial design and construction
Patients
Presumed severe CAP
• Different strata
(ex. shock or not)
• Collected at sites
• Managed at regional data centers
• Merged at central statistical center
Data collection
Re-estimate Bayesian inference model
with new data to update probabilities
Update and adapt
Response-adaptive randomization
External adaptations
47. Statistical trigger
• Launch with initial weights
• Update based on new probabilities
Steering Committee can
• Add strata, domains & interventions
DSMB can
• Request new external data be incorporated in priors
• Overrule statistical triggers
Multifactorial intervention assignments
Regimen = set of domain-specific
interventions
Effect of an intervention is conditional upon
• Stratum
• Interventions within other domains
Embedding
Patient identification and enrollment
• Tied to clinical ‘point-of-care’
Randomized interventions
• Issued as ‘order set’ regimen
EHR embedding
• Screen and flag patient
• Consent documentation
• Generate regimen order set
• Flag downstream states
• Data collection
Regimen Domain A Domain B Domain C
#1 A1 B1 C1
#2 A1 B1 C2
#3 A1 B2 C1
#4 A1 B2 C2
#5 A2 B1 C1
…..
#n An Bn Cn
Result declared when, within stratum, an intervention is
• Superior >99% likely to be best
• Equivalent >90% likely to be D<3%
• Inferior <1% likely to be best
Pre-specified architecture determined by
• Choice of domains, strata, etc.
• Choice of potential interactions
Choices inform a Bayesian inference model
• Pre-trial simulations evaluate performance
Each external adaptation (ex. new domain)
• Modify elements in Bayesian model
• Re-simulate before ‘live’ deployment
Pre-trial design and construction
Patients
Presumed severe CAP
• Different strata
(ex. shock or not)
• Collected at sites
• Managed at regional data centers
• Merged at central statistical center
Data collection
Re-estimate Bayesian inference model
with new data to update probabilities
Update and adapt
Response-adaptive randomization
External adaptations
48. Statistical trigger
• Launch with initial weights
• Update based on new probabilities
Multifactorial intervention assignments
Regimen = set of domain-specific
interventions
Effect of an intervention is conditional upon
• Stratum
• Interventions within other domains
Embedding
Patient identification and enrollment
• Tied to clinical ‘point-of-care’
Randomized interventions
• Issued as ‘order set’ regimen
EHR embedding
• Screen and flag patient
• Consent documentation
• Generate regimen order set
• Flag downstream states
• Data collection
Regimen Domain A Domain B Domain C
#1 A1 B1 C1
#2 A1 B1 C2
#3 A1 B2 C1
#4 A1 B2 C2
#5 A2 B1 C1
…..
#n An Bn Cn
Result declared when, within stratum, an intervention is
• Superior >99% likely to be best
• Equivalent >90% likely to be D<3%
• Inferior <1% likely to be best
Pre-specified architecture determined by
• Choice of domains, strata, etc.
• Choice of potential interactions
Choices inform a Bayesian inference model
• Pre-trial simulations evaluate performance
Each external adaptation (ex. new domain)
• Modify elements in Bayesian model
• Re-simulate before ‘live’ deployment
Pre-trial design and construction
Patients
Presumed severe CAP
• Different strata
(ex. shock or not)
• Collected at sites
• Managed at regional data centers
• Merged at central statistical center
Data collection
Re-estimate Bayesian inference model
with new data to update probabilities
Update and adapt
Response-adaptive randomization
49. Statistical trigger
• Launch with initial weights
• Update based on new probabilities
Steering Committee can
• Add strata, domains & interventions
DSMB can
• Request new external data be incorporated in priors
• Overrule statistical triggers
Multifactorial intervention assignments
Regimen = set of domain-specific
interventions
Effect of an intervention is conditional upon
• Stratum
• Interventions within other domains
Embedding
Patient identification and enrollment
• Tied to clinical ‘point-of-care’
Randomized interventions
• Issued as ‘order set’ regimen
EHR embedding
• Screen and flag patient
• Consent documentation
• Generate regimen order set
• Flag downstream states
• Data collection
Regimen Domain A Domain B Domain C
#1 A1 B1 C1
#2 A1 B1 C2
#3 A1 B2 C1
#4 A1 B2 C2
#5 A2 B1 C1
…..
#n An Bn Cn
Result declared when, within stratum, an intervention is
• Superior >99% likely to be best
• Equivalent >90% likely to be D<3%
• Inferior <1% likely to be best
Pre-specified architecture determined by
• Choice of domains, strata, etc.
• Choice of potential interactions
Choices inform a Bayesian inference model
• Pre-trial simulations evaluate performance
Each external adaptation (ex. new domain)
• Modify elements in Bayesian model
• Re-simulate before ‘live’ deployment
Pre-trial design and construction
Patients
Presumed severe CAP
• Different strata
(ex. shock or not)
• Collected at sites
• Managed at regional data centers
• Merged at central statistical center
Data collection
Re-estimate Bayesian inference model
with new data to update probabilities
Update and adapt
Response-adaptive randomization
External adaptations
50. C R I S M A
REMAP severe pneumonia …
• Gets closer to individualized treatment decisions …
• For example, should my patient receive IV hydrocortisone?
• Depends on
• Whether shock is present
• How sick (hypoxic) the patient is
• Whether underlying cause is viral or not
• Whether an anti-viral is being administered
• Whether other strategies are being used that may minimize lung injury and
inflammation (protective vs. ultraprotective ventilation)
• Separate probability estimate for each consideration …
• Trial enrolls until a predefined level of certainty
• As soon as one question hits threshold, answer is announced
51. C R I S M A
Run the trial ‘in silico’ ahead of time …
• Monte-Carlo simulations
• Run 1,000s of times under different scenarios
56. C R I S M A
Ok, but …
• EHR data quality
• Institutional commitment
• Ethics
• Statistics and design
• Reporting and dissemination of results
• Funding
• Oversight
• Integration with other clinical research programs
57. C R I S M A
Conclusions
• RCTs remain our strongest ‘truth’ finder
• But, current RCT enterprise LETS US DOWN
• ‘Big Data’ should not be cast as an alternative to the RCT
• This is a false choice
• Instead, the digital age enables novel RCTs designs
• Smarter and safer
58. C R I S M A
Self-learning healthcare is …
fused care and research