Since the worldwide outbreak of the COVID-19 pandemic, experts all around the globe are working heavily to establish reliable forecasts for the spread of the disease. Hereby they allow decision-makers to roughly plan ahead and inform the population with estimates of what might still lie ahead. Yet, the huge jungle of different models, data and results is confusing and difficult to overlook: what models are reliable, which results can be trusted and what are the secrets behind these models?
OUR SPEAKER
Dr. Martin Bicher is chief developer of the COVID-19 modeling team around Dr. Niki Popper in dwh simulation-services GmbH, which currently supports decision-makers all around Austria with simulated forecasts, scenarios and policy evaluations. Moreover, he is a postdoctoral researcher at the Institute of Information Systems Engineering at TU Wien where he finished his PhD in Technical Mathematics.
1. Modelling the spread of SARS-CoV-2:
How reliable are simulated forecasts?
Dr. Martin Bicher 1,2
1TU Wien, Institute for Information Systems Engineering
2dwh GmbH, dwh simulation
2. Overview
• What is and why do we need
mathematical modelling and
simulation?
• What‘s the secret behind
epidemiological models?
• What‘s the uncertainty of
epidemiological models?
10. Mathematical Modelling and Simulation
Abstract Example
Experiments
5m
Does the paper plane fly
further than 5m?
11. Mathematical Modelling and Simulation
Abstract Example
Experiments
In 80% of all
cases.
5m
Does the paper plane fly
further than 5m?
12. Mathematical Modelling and Simulation
Abstract Example
5m
Does the paper plane fly
further than 5m?
Experiments
In 70% of all
cases.
13. Mathematical Modelling and Simulation
Abstract Example
5m
Does the paper plane fly
further than 5m?
Experiments
In 70% of all
cases.
General Paper Plane Theory:
„Arrow-type paper planes fly ...
dependent on their wing span...“
14. Mathematical Modelling and Simulation
Abstract Example
Does the paper plane fly
further than 5m?
General Paper Plane Theory:
„Arrow-type paper planes fly ...
dependent on their wing span...“
In 90% of all
cases.
15. Mathematical Modelling and Simulation
Abstract Example
I need to know it
in advance
No
arrow-type
Does the paper plane fly
further than 5m?
General Paper Plane Theory:
„Arrow-type paper planes fly ...
dependent on their wing span...“
16. Mathematical Modelling and Simulation
Abstract Example
Mathematical paper
plane model
𝑒 =
𝑢2
2
+
𝑝
𝜌
+ 𝑔𝑧
• Findings from „ General Paper
Plane Theory“
• Data from experiments
• Additional knowledge like
Physikal laws, ...
Does the paper plane fly
further than 5m?
General Paper Plane Theory:
„Arrow-type paper planes fly ...
dependent on their wing span...“
I need to know it
in advance
No
arrow-type
17. Mathematical Modelling and Simulation
Abstract Example
Simulation
Does the paper plane fly
further than 5m?
General Paper Plane Theory:
„Arrow-type paper planes fly ...
dependent on their wing span...“
I need to know it
in advance
No
arrow-type
Mathematical paper
plane model
𝑒 =
𝑢2
2
+
𝑝
𝜌
+ 𝑔𝑧
In 2% of all
cases.
18. Mathematical Modelling and Simulation
Abstract Example
Simulation
Model Uncertainty:
• Previous knowledge cannot
be used directly
• Impossible to model all
influence factors
• ...
Does the paper plane fly
further than 5m?
General Paper Plane Theory:
„Arrow-type paper planes fly ...
dependent on their wing span...“
I need to know it
in advance
No
arrow-type
Mathematical paper
plane model
𝑒 =
𝑢2
2
+
𝑝
𝜌
+ 𝑔𝑧
In 2% of all
cases... probably
19. Mathematical Modelling and Simulation
COVID-19
How is SARS-CoV-2 going
to spread in Austria?
What consequences will
the epidemic of COVID-19
have on the population?
20. Mathematical Modelling and Simulation
COVID-19
• Italy ≠ Austria
General knowledge about
Influenza, other corona
viruses, and the health care
system
• SARS CoV-2 ≠
Influenza
Unintentional „experiments“:
data from countries in which
the virus has started spreading
earlier
How is SARS-CoV-2 going
to spread in Austria?
What consequences will
the epidemic of COVID-19
have on the population?
21. Mathematical Modelling and Simulation
COVID-19
• Italy ≠ Austria
General knowledge about
Influenza, other corona
viruses, and the health care
system
• SARS CoV-2 ≠
Influenza
Unintentional „experiments“:
data from countries in which
the virus has started spreading
earlier
How is SARS-CoV-2 going
to spread in Austria?
What consequences will
the epidemic of COVID-19
have on the population?
COVID-19 simulation
model
e.g.:
𝑆′
= −𝛼𝐼𝑆
𝐼′ = 𝛼𝐼𝑆 − 𝛽𝐼
𝑅′ = 𝛽𝐼
• Information / data from
countries with more
„COVID-19 experience“
• General knowledge about the
spread of infectious diseases
in Austria
• Additional causal knowledge
about the spread of infectious
diseases (SIR)
22. Mathematical Modelling and Simulation
COVID-19
• Italy ≠ Austria
General knowledge about
Influenza, other corona
viruses, and the health care
system
• SARS CoV-2 ≠
Influenza
Unintentional „experiments“:
data from countries in which
the virus has started spreading
earlier
How is SARS-CoV-2 going
to spread in Austria?
What consequences will
the epidemic of COVID-19
have on the population?
COVID-19 simulation
model
e.g.:
𝑆′
= −𝛼𝐼𝑆
𝐼′ = 𝛼𝐼𝑆 − 𝛽𝐼
𝑅′ = 𝛽𝐼
• Information / data from
countries with more
„COVID-19 experience“
• General knowledge about the
spread of infectious diseases
in Austria
• Additional causal knowledge
about the spread of infectious
diseases (SIR)
Model Uncertainty ?
24. What are the secrets behind epidemiological
models?
data based models causal models
• much data
• little logic
• (a little bit) less data
• much logic
• extrapolation of
current trends
• good fit of current
numbers
• statistical methods,
machine learning, ...
• good qualitative
forecasting capabilities
• Status quo more
difficult to fit
• uses meta information
about the system
25. What are the secrets behind epidemiological
models?
data based models causal models
• much data
• little logic
• (a little bit) less data
• much logic
• extrapolation of
current trends
• good fit of current
numbers
• statistical methods,
machine learning, ...
• good qualitative
forecasting capabilities
• Status quo more
difficult to fit
• uses meta information
about the system
data method solution
causalities method solution
data
26. What are the secrets behind epidemiological
models?
data-based
models
causal
models
macroscopic/aggregated models
susceptible
persons
infectious
persons
resistant
persons
new
infections
per day
recoveries
per day
• SIR Strategy
• Most famous model by Kermack and McKendrick (~1927)
27. What are the secrets behind epidemiological
models?
susceptible
new infections
per day
exposed
persons becoming
infectious per day
infectious
resistant
persons recovering
per day
• SEIR Strategy
• „Exposed“ = persons within latency
time
• time-delayed epidemics behaviour
data-based
models
causal
models
macroscopic/aggregated models
28. What are the secrets behind epidemiological
models?
susceptible
exposed
infectious
resistant
asymptomatic
infectious
deceased
home
quarantined
hospitalised
severe
hospitalised
critical
• Compartment model strategy
leaves space for any kind of
model extension
(SIRS,SEIRS,SIR-X,SIS,...)
data-based
models
macroscopic/aggregated models
29. What are the secrets behind epidemiological
models?
macroscopic/
aggregated
models
microscopic models
(microsimulations)
susceptible
person
resistant
person
infectious
person
infection
recovery
• agent-based
models
• Markov Models
• DES Models
data-based
models
causal
models
30. What are the secrets behind epidemiological
models?
macroscopic/
aggregated
models
microscopic
models
• „No-Free-Lunch“
All model types have advantages
and disadvantages
• Every model is developed for a
certain purpose
It should only be used for this
purpose and only within its range
of validity
data-based
models
causal
models
31. DWH/TU Wien
COVID-19 Model
Population Module
Contact
Module
Disease
Module
Policy Module
Responsible for
creating and
managing agents as
well as death and
birth processes
Responsible for
creating agent-
agent contacts
Responsible for
modelling the
disease progression
Responsible for
modelling of
policies
32. DWH/TU Wien
COVID-19 Model
Key concept:
Every real inhabitant is repesented by a statistically representative
virtual inhabitant (a so called agent)
Sampling
33. DWH/TU Wien
COVID-19 Model
Key concept:
Every real inhabitant is repesented by a statistically representative
virtual inhabitant (a so called agent)
Sampling
34. DWH/TU Wien
COVID-19 Model
visit locations
households schools workplaces leisure-time
contacts at locations
who contacts
whom?
contactmodule
• Contacts between agents are
not direct but agent-
location-agent
contacts
• Location types: households,
schools, workplaces
• Additonal
agent-agent
leisure time contacts
35. DWH/TU Wien
COVID-19 Model
• Disease progression is
handled in form of a flow-chart
• Each agent uses is own
discrete-event simulator
age 54
male
age 38
female
infectious
36. How about uncertainty?
• Italy ≠ Austria
General knowledge about
Influenza, other corona
viruses, and the health care
system
• SARS CoV-2 ≠
Influenza
Unintentional „experiments“:
data from countries in which
the virus has started spreading
earlier
How is SARS-CoV-2 going
to spread in Austria?
What consequences will
the epidemic of COVID-19
have on the population?
COVID-19 simulation
model
e.g.:
𝑆′
= −𝛼𝐼𝑆
𝐼′ = 𝛼𝐼𝑆 − 𝛽𝐼
𝑅′ = 𝛽𝐼
• Information / data from
countries with more
„COVID-19 experience“
• General knowledge about the
spread of infectious diseases
in Austria
• Additional causal knowledge
about the spread of infectious
diseases (SIR)
Model Uncertainty ?
37. How about uncertainty?
Model Uncertainty
Model Errors.
• Every model is a simplified version of
reality:
potentially important mechanisms
might not included or insuffiently
known
Data Errors
• All data sources for COVID-19 are
heavily biased
Forecast Uncertainty
Feedback Loop.
• Every forecasts is only valid for a
system that does not change
• Even published forecasts might
change the system behaviour
NEWS
„ ... scientists forecast Millions
of infected persons...“
NEWS
„ ... government reacts on
forecasts and introduces
lockdown measures ...“
NEWS
„ ... all prognoses were wrong!
Can we trust the scientists?“
38. Summary
Though the uncertainty is huge, computer models and
simulations are currently our only way to get an idea of
what lies ahead of us.
Consequently they are and will remain a vital element for
decision support of COVID-19 policy makers
0
2000
4000
6000
8000
10000
12000
14000
16000
2020-02-19T00:00
2020-02-22T00:00
2020-02-25T00:00
2020-02-28T00:00
2020-03-02T00:00
2020-03-05T00:00
2020-03-08T00:00
2020-03-11T00:00
2020-03-14T00:00
2020-03-17T00:00
2020-03-20T00:00
2020-03-23T00:00
2020-03-26T00:00
2020-03-29T00:00
2020-04-01T00:00
2020-04-04T00:00
2020-04-07T00:00
2020-04-10T00:00
2020-04-13T00:00
2020-04-16T00:00
2020-04-19T00:00
2020-04-22T00:00
2020-04-25T00:00
2020-04-28T00:00
2020-05-01T00:00
2020-05-04T00:00
2020-05-07T00:00
2020-05-10T00:00
2020-05-13T00:00
2020-05-16T00:00
2020-05-19T00:00
2020-05-22T00:00
2020-05-25T00:00
2020-05-28T00:00
2020-05-31T00:00
2020-06-03T00:00
2020-06-06T00:00
2020-06-09T00:00
2020-06-12T00:00
2020-06-15T00:00
2020-06-18T00:00
2020-06-21T00:00
2020-06-24T00:00
2020-06-27T00:00
2020-06-30T00:00
2020-07-03T00:00
2020-07-06T00:00
2020-07-09T00:00
2020-07-12T00:00
2020-07-15T00:00
2020-07-18T00:00
2020-07-21T00:00
2020-07-24T00:00
2020-07-27T00:00
2020-07-30T00:00
2020-08-02T00:00
2020-08-05T00:00
2020-08-08T00:00
2020-08-11T00:00
39. Dr. Martin Bicher 1,2
1TU Wien, Institute for Information Systems Engineering
2dwh GmbH, dwh simulation
Thank you for your attention
Modelling the spread of SARS-CoV-2:
How reliable are simulated forecasts?