A talk on COVID-19 and causal inference delivered to the CogX2020 web conference on June 8 2020

1. @EpiEllie
COVID-19 and
causal inference
Eleanor Murray, ScD
Department of
Epidemiology
CogX
June 8, 2020

2. What do we want to know?
What will the world look like in the future?
How would things have changed if the world
had been slightly different?
Treat
now
Treat later
@EpiEllie

3. How do we estimate causal
effects?
Miguel Hernàn’s two-step causal algorithm:
1. Ask good questions
2. Answer them with appropriate methods
? !
@EpiEllie

4. What do we want to know?
If we can’t have a time machine, we’d like to
have a randomized trial.
Treat now
Treat later
@EpiEllie

5. Target trial framework for estimating
causal effects
If we can’t have a randomized trial, we’d like to
emulate what would have happened if we could
have done one.
We can do this with:
Imperfect trials
Observational data
Simulation modeling Treat now
Treat later
@EpiEllie

6. What assumptions do we need?
No unmeasured confounding: all common causes of
the treatment and outcome are known and measured in
the data
No open colliders: all common effects of the treatment
and outcome are known and not conditioned on in the
data or analysis
@EpiEllie

7. What assumptions do we
need?
Positivity: there is a non-zero probability of all
levels of treatment for all types of individuals in
our population
@EpiEllie

8. What assumptions do we
need?
 Consistency: our treatment levels are clearly
specified, aka:
 Well-defined interventions
 Well-defined causal questions
@EpiEllie

9. Why are well-defined causal questions
important for complex exposures?
When there are multiple possible ‘interventions’
and we don’t specify one, our answer is a weighted
average of all ‘interventions’ but we don’t know
the weights
Murray, 2016. Agent-based models for causal inference. Harvard
University.
@EpiEllie

10. Why are well-defined causal questions
important for complex exposures?
Worse, if the ‘intervention’ is ill-defined, the
confounding is probably also ill-defined!
Murray, 2016. Agent-based models for causal inference. Harvard
@EpiEllie

11. All communicable disease by nature
involve complex exposures
We cannot define the causal effect of an
intervention on a communicable disease
without accounting for who is infected and
how people come into contact
This is the problem of interference

12. Special challenges for COVID19 RCTs
Identifying an appropriate control group
Can we ethically use placebo? What is ‘standard
of care’?
Identifying an appropriate outcome
Many early studies assessed ‘symptom
improvement’ as a surrogate outcome, but failed
to properly account for ICU admission or death
Many later studies looked at death but only in
hospital. Individuals who were discharged were
‘lost to follow-up’ @EpiEllie

13. Special challenges for COVID19 RCTs
Understanding to whom the results apply
Can a trial run at a hospital in NYC tell us how
to treat patients in Singapore? Kinshasa?
London?
Can a trial run among patients were never on
ventilators tell us how to treat new patients who
may need ventilators?
@EpiEllie

14. Special challenges for COVID19
observational studies
All the same problems as with RCTs apply,
plus:
Identifying an appropriate control group
Who should we compare to when experimental
treatments are preferentially given to the sickest
patients?
We know we have confounding. How do we
control for it?
@EpiEllie

15. Special challenges for COVID19
simulation studies
All the same problems as with RCTs &
observational studies apply, plus:
Identifying parameter inputs
We need external validity for not only our effect
estimates but also our model inputs
Understanding uncertainty
This is a big unanswered question in simulation
modelling that needs more statistical attention
@EpiEllie

16. Uncertainty in simulation models for
causal effects
We cannot generate confidence intervals for
our simulation models because we do not
have a way to capture the full variance.
Modelers recognize three sources of
uncertainty:
Stochastic
Parametric
Structural
 Solution: Increase sample size or number of runs
 Solution: Probabilistic or Bayesian sensitivity analyses
on key parameters.
 Which are key? What distributions should we use? What impact does it have to
assume other parameters don’t have uncertainty?
 Solution: ?? Probably will involve synthesizing across
different model structures but how do we decide which
ones & how do we quantify uncertainty? @EpiEllie

17. Summary
COVID19 urgently needs good causal effect
estimates for identifying effective prevention
and treatment strategies.
Many existing studies are committing basic
statistical and epidemiologic errors.
There are also important gaps in our
statistical methods for quantifying uncertainty
in simulation model estimates.
@EpiEllie