monitor spec, detector model ● Trace collection: run missions, log detectors ● NCC estimation: - Vary #traces - Vary independence assumptions - Compare to BBC on full data ● Metrics: accuracy, computation time Evaluation of NCC estimates 66 Results: accuracy vs. #traces Evaluation of NCC estimates 67 Results: accuracy vs. assumptions Evaluation of NCC estimates 68 Conclusions ● NCC provides accurate error rate estimates with modest trace data ● Accuracy depends on independence assumptions ● Computation time scales linearly with formula size ● Mixed approach is promising for runtime verification

1. 1
Compositional Probabilistic Analysis
of Temporal Properties
over Stochastic Detectors
Ivan Ruchkin, Oleg Sokolsky, Jim Weimer,
Tushar Hedaoo, Insup Lee
PRECISE Center
Department of Computer and Information Science
University of Pennsylvania
The International Conference on Embedded Software (EMSOFT)
September 22, 2020

2. 2
System example:
Unmanned Underwater Vehicle (UUV)

3. 3
System example:
Unmanned Underwater Vehicle (UUV)

4. 4
Forward sonar:
obstacles
Side sonar:
pipeline
GPS/IMU:
pose
Doppler radar:
seafloor
System example:
Unmanned Underwater Vehicle (UUV)

5. 5
Forward sonar:
obstacles
Side sonar:
pipeline
GPS/IMU:
pose
Doppler radar:
seafloor
“The UVV should eventually re-discover the
pipeline after avoiding an obstacle.”
System example:
Unmanned Underwater Vehicle (UUV)

6. 6
Motivating property
“The UVV should eventually re-discover the
pipeline after avoiding an obstacle.”

7. 7
Motivating property
“The UVV should eventually re-discover the
pipeline after avoiding an obstacle.”
Run-time monitor

8. 8
Motivating property
“The UVV should eventually re-discover the
pipeline after avoiding an obstacle.”
Run-time monitor
Detectors

9. 9
Motivating property
“The UVV should eventually re-discover the
pipeline after avoiding an obstacle.”
Temporal relation
Run-time monitor
Detectors

10. 10
Motivating property
“The UVV should eventually re-discover the
pipeline after avoiding an obstacle.”
Temporal relation
Run-time monitor
– What are the error rates of this property’s
monitor given uncertain, unreliable detectors?
Detectors

11. 11
Motivating property
– What are the error rates of this property’s
monitor given uncertain, unreliable detectors?

12. 12
Motivating property
Data-driven
●
Develop a monitor
●
Sample & label
monitor outputs
●
Empirically estimate
its error rate
– What are the error rates of this property’s
monitor given uncertain, unreliable detectors?

13. 13
Motivating property
Data-driven
●
Develop a monitor
●
Sample & label
monitor outputs
●
Empirically estimate
its error rate
Pro: accurate
Con: expensive data,
effortful development
– What are the error rates of this property’s
monitor given uncertain, unreliable detectors?

14. 14
Motivating property
Data-driven
●
Develop a monitor
●
Sample & label
monitor outputs
●
Empirically estimate
its error rate
Pro: accurate
Con: expensive data,
effortful development
Model-based
●
Specify a monitor
●
Model the system
●
Theoretically estimate
the error rate
– What are the error rates of this property’s
monitor given uncertain, unreliable detectors?

15. 15
Motivating property
Data-driven
●
Develop a monitor
●
Sample & label
monitor outputs
●
Empirically estimate
its error rate
Pro: accurate
Con: expensive data,
effortful development
Model-based
●
Specify a monitor
●
Model the system
●
Theoretically estimate
the error rate
Pro: no data needed
Con: inaccurate, poor
scalability
– What are the error rates of this property’s
monitor given uncertain, unreliable detectors?

16. 16
Motivating property
Data-driven
●
Develop a monitor
●
Sample & label
monitor outputs
●
Empirically estimate
its error rate
Pro: accurate
Con: expensive data,
effortful development
Model-based
●
Specify a monitor
●
Model the system
●
Theoretically estimate
the error rate
Pro: no data needed
Con: inaccurate, poor
scalability
Mixed
●
Specify a monitor
●
Sample & label
detector outputs
●
Estimate the error
rate using the spec
and detector data
– What are the error rates of this property’s
monitor given uncertain, unreliable detectors?

17. 17
Motivating property
Data-driven
●
Develop a monitor
●
Sample & label
monitor outputs
●
Empirically estimate
its error rate
Pro: accurate
Con: expensive data,
effortful development
Model-based
●
Specify a monitor
●
Model the system
●
Theoretically estimate
the error rate
Pro: no data needed
Con: inaccurate, poor
scalability
Mixed
●
Specify a monitor
●
Sample & label
detector outputs
●
Estimate the error
rate using the spec
and detector data
Pro: cheap data,
accurate, scalable,
low effort
– What are the error rates of this property’s
monitor given uncertain, unreliable detectors?

18. 18
Approach outline
Computational
assistant
Detector model
Monitor’s error rate
Detector data
Monitor spec

19. 19
Approach outline
Computational
assistant
Detector model
Monitor’s error rate
Detector data
Monitor spec

20. 20
Our detector model

21. 21
Side sonar input
Front sonar input Obstacle det
Our detector model
Pipeline det
Detection
output
Detection
output

22. 22
Side sonar input
Front sonar input Obstacle det
Our detector model
Pipeline det
Detection
output
Detection
output
Property monitor
Violation

23. 23
Side sonar input
Front sonar input Obstacle det
Our detector model
Pipeline det
Detection
output
Detection
output
Error
Error
Property monitor
Error
Violation

24. 24
Side sonar input
Front sonar input Obstacle det
Our detector model
Pipeline det
Detection
output
Detection
output
Error
Error
Property monitor
Error
Violation
P( ) = ?

25. 25
Side sonar input
Front sonar input Obstacle det
Our detector model
Pipeline det
Detection
output
Detection
output
Obst. ground truth
Pipe ground truth
Error
Error
Property monitor
Error
Violation
P( ) = ?

26. 26
Obstacle det
Our detector model
Pipeline det
Detection
output
Detection
output
Error
Error
Property monitor
Obst. ground truth
Pipe ground truth
Error
Violation
P( ) = ?

27. 27
D2
Our detector model
D1
Detection
output
Detection
output
Error
Error
M
Ground truth
Ground truth
Error
Violation
P( ) = ?

28. 28
D2
Our detector model
D1
Detection
output
Detection
output
Error
Error
M
r.v. GT2
r.v. GT1
Ground truth
Ground truth
Error
Violation
P( ) = ?

29. 29
D2
Our detector model
D1
Detection
output
Detection
output
Error
Error
M
r.v. GT2
r.v. GT1
Ground truth
Ground truth
r.v. GT2
Values:
GT2
= true↔gtt(D2
)
GT2
= false↔gtf(D2
)
Error
Violation
P( ) = ?

30. 30
D2
Our detector model
D1
Detection
output
Detection
output
Error
Error
M
r.v. DO2
r.v. DO1r.v. GT1
Ground truth
Ground truth
r.v. GT2
Values:
GT2
= true↔gtt(D2
)
GT2
= false↔gtf(D2
)
Error
Violation
P( ) = ?

31. 31
D2
Our detector model
D1
Detection
output
Detection
output
Error
Error
M
r.v. DO2
Values:
DO2
= true↔dot(D2
)
DO2
= false↔dof(D2
)
DO2
= unknown↔dou(D2
)
r.v. DO1r.v. GT1
Ground truth
Ground truth
r.v. GT2
Values:
GT2
= true↔gtt(D2
)
GT2
= false↔gtf(D2
)
Error
Violation
P( ) = ?

32. 32
D2
Our detector model
D1
Detection
output
Detection
output
Error
Error
M = φ(D1
, D2
)
r.v. DO2
Values:
DO2
= true↔dot(D2
)
DO2
= false↔dof(D2
)
DO2
= unknown↔dou(D2
)
r.v. GT1
r.v. DO1
Ground truth
Ground truth
r.v. GT2
Values:
GT2
= true↔gtt(D2
)
GT2
= false↔gtf(D2
)
Error
Violation
P( ) = ?
DOM
, GTM

33. 33
D2
Our detector model
D1
Detection
output
Detection
output
Error
Error
Error
M = φ(D1
, D2
)
Violation
P( ) = ?r.v. GT2
Values:
GT2
= true↔gtt(D2
)
GT2
= false↔gtf(D2
)
r.v. DO2
Values:
DO2
= true↔dot(D2
)
DO2
= false↔dof(D2
)
DO2
= unknown↔dou(D2
)
r.v. GT1
FPR: Pr( dot(...) | gtf(...) )
FNR: Pr(¬dot(...) | gtt(...) )
DOM
, GTM
Ground truth
Ground truth
r.v. DO1

34. 34
Three-valued LTL for detectors
● Syntax of LTL3d
Semantics is given by detector compositions
“The UUV should re-discover the pipeline
within d seconds after losing it”
Monitor for this property

35. 35
Three-valued LTL for detectors
● Syntax of LTL3d
●
Semantics is given by detector compositions
“The UUV should re-discover the pipeline
within d seconds after losing it”
Monitor for this property

36. 36
Three-valued LTL for detectors
● Syntax of LTL3d
●
Semantics is given by detector compositions
●
“The UUV should re-discover the pipeline
within d seconds after losing it”
Monitor for this property

37. 37
Three-valued LTL for detectors
● Syntax of LTL3d
●
Semantics is given by detector compositions
●
“The UUV should re-discover the pipeline
within d seconds after losing it”
●
Monitor for this property

38. 38
Approach outline
Computational
assistant
Detector model
Monitor’s error rate
Detector data
Monitor spec

39. 39
Approach outline
Computational
assistant
Detector model
Monitor’s error rate
Detector data
Monitor spec

40. 40
Approach outline
Computational
assistant
Detector model
Monitor’s error rate
Detector data
Monitor spec

41. 41
Three estimators for error rates

42. 42
Three estimators for error rates
●
Black-box calculation (BBC)
– Inputs: labeled monitor traces
– Baseline data-driven estimator
Exact compositional calculation (ECC)
Inputs: detector probabilities, independence
assumptions, reasoning rules
Noisy compositional calculation (NCC)
Inputs: labeled detector traces, independence
assumptions, reasoning rules

43. 43
Three estimators for error rates
●
Black-box calculation (BBC)
– Inputs: labeled monitor traces
– Baseline data-driven estimator
●
Exact compositional calculation (ECC)
– Inputs: detector probabilities, independence
assumptions, reasoning rules
Noisy compositional calculation (NCC)
Inputs: labeled detector traces, independence
assumptions, reasoning rules

44. 44
Three estimators for error rates
●
Black-box calculation (BBC)
– Inputs: labeled monitor traces
– Baseline data-driven estimator
●
Exact compositional calculation (ECC)
– Inputs: detector probabilities, independence
assumptions, reasoning rules
●
Noisy compositional calculation (NCC)
– Inputs: labeled detector traces, independence
assumptions, reasoning rules

45. 45
Three estimators for error rates
●
Black-box calculation (BBC)
– Inputs: labeled monitor traces
– Baseline data-driven estimator
●
Exact compositional calculation (ECC)
– Inputs: detector probabilities, independence
assumptions, reasoning rules
●
Noisy compositional calculation (NCC)
– Inputs: labeled detector traces, independence
assumptions, reasoning rules
Computational assistant: https://github.com/bisc/prob-comp-asst

46. 46
ECC/NCC steps
Step 1: rewrite with
event predicates
Step 2: reduce to known/
estimable probabilities
Step 3: check if an
independence holds
Step 4: numerically
estimate probabilities
Step 5: calculate rate
estimate

47. 47
ECC/NCC steps
●
Step 1: simplify/rewrite
with event predicates
Step 2: reduce to known/
estimable probabilities
Step 3: check if an
independence holds
Step 4: numerically
estimate probabilities
Step 5: calculate rate
estimate

48. 48
ECC/NCC steps
●
Step 1: simplify/rewrite
with event predicates
Step 2: reduce to known/
estimable probabilities
Step 3: check if an
independence holds
Step 4: numerically
estimate probabilities
Step 5: calculate rate
estimate

49. 49
ECC/NCC steps
●
Step 1: simplify/rewrite
with event predicates
Step 2: reduce to known/
estimable probabilities
Step 3: check if an
independence holds
Step 4: numerically
estimate probabilities
Step 5: calculate rate
estimate

50. 50
ECC/NCC steps
●
Step 1: simplify/rewrite
with event predicates
●
Step 2: reduce to known/
estimable probabilities
Step 3: check if an
independence holds
Step 4: numerically
estimate probabilities
Step 5: calculate rate
estimate

51. 51
ECC/NCC steps
●
Step 1: simplify/rewrite
with event predicates
●
Step 2: reduce to known/
estimable probabilities
Step 3: check if an
independence holds
Step 4: numerically
estimate probabilities
Step 5: calculate rate
estimate

52. 52
ECC/NCC steps
●
Step 1: simplify/rewrite
with event predicates
●
Step 2: reduce to known/
estimable probabilities
●
Step 3: check if an
independence holds
Step 4: numerically
estimate probabilities
Step 5: calculate rate
estimate

53. 53
Independence assumptions
●
Assumption 1: detections determined by GTs
●
Assumption 2: detection determined by its GT

54. 54
ECC/NCC steps
●
Step 1: simplify/rewrite
with event predicates
●
Step 2: reduce to known/
estimable probabilities
●
Step 3: check if an
independence holds
Step 4: numerically
estimate probabilities
Step 5: calculate rate
estimate

55. 55
ECC/NCC steps
●
Step 1: simplify/rewrite
with event predicates
●
Step 2: reduce to known/
estimable probabilities
●
Step 3: check if an
independence holds
●
Step 4: numerically
estimate probabilities
Step 5: calculate rate
estimate

56. 56
ECC/NCC steps
●
Step 1: simplify/rewrite
with event predicates
●
Step 2: reduce to known/
estimable probabilities
●
Step 3: check if an
independence holds
●
Step 4: numerically
estimate probabilities
Step 5: calculate rate
estimate

57. 57
ECC/NCC steps
●
Step 1: simplify/rewrite
with event predicates
●
Step 2: reduce to known/
estimable probabilities
●
Step 3: check if an
independence holds
●
Step 4: numerically
estimate probabilities
Step 5: calculate rate
estimate

58. 58
ECC/NCC steps
●
Step 1: simplify/rewrite
with event predicates
●
Step 2: reduce to known/
estimable probabilities
●
Step 3: check if an
independence holds
●
Step 4: numerically
estimate probabilities
●
Step 5: calculate the
rate estimate

59. 59
Approach outline
Computational
assistant
Detector model
Monitor’s error rate
Detector data
Monitor spec

60. 60
Approach outline
Computational
assistant
Detector model
Monitor’s error rate
Detector data
Monitor spec

61. 61
Approach outline
Computational
assistant
Detector model
Monitor’s error rate
Detector data
Monitor spec

62. 62
Evaluation of NCC estimates

63. 63
Evaluation of NCC estimates
●
How accurate are they?
●
How are they affected by the quantity of
trace data?
●
How are they affected by the independence
assumptions?
●
What are their computational costs?

64. 64
●
How accurate are they?
●
How are they affected by the quantity of trace
data?
●
How are they affected by the independence
assumptions?
– Sensitive to accurate assumptions
●
What are their computational costs?
– 0-10 seconds, depending on formula size
Evaluation of NCC estimates

65. 65
Evaluation setup
●
Setup: UUV Gazebo sim, randomized missions
●
Goal: estimate error rates of two monitors
●
73 missions (total 7.7 hrs of sim time), each:
– Two pairs of traces: (DO, GT) for pipe det & monitor
https://github.com/uuvsimulator/uuv_simulator

66. 66
Evaluation variables
●
Independent:
– Mission configuration
– Monitor formula; deadline d for pipe loss (sec)
– Detector/monitor traces, for NCC/BBC resp.
Dependent (error rate estimates):
ECC: the true value based on exact probabilities
NCC: our approach on cheap (detector) data
BBC: data-driven on expensive (monitor) data

67. 67
Evaluation variables
●
Independent:
– Mission configuration
– Monitor formula; deadline d for pipe loss (sec)
– Detector/monitor traces, for NCC/BBC resp.
●
Dependent (error rate estimates):
– ECC: the true value based on exact probabilities
– NCC: our approach on cheap (detector) data
– BBC: data-driven on expensive (monitor) data

68. 68
ECC estimate (true, exact value)
NCC estimate (ours, detector traces)
BBC estimate (baseline, monitor traces)
Interpretation: given enough data,
the estimates are close

69. 69
NCC estimate (ours, detector traces)
BBC estimate (baseline, monitor traces)
Interpretation: less data favors NCC;
more data favors BBC

70. 70
Summary
Logical composition of detectors with LTL3d
Probabilistic estimation of error rates
Rule-based computational assistant
Accuracy on par with data-driven estimates
While using cheaper data & scalable analysis
Preferred when little or no data available
Computational
assistant
Detector model
Monitor’s error rate
Detector data
Monitor spec

71. 71
Summary
● Logical composition of detectors with LTL3d
Probabilistic estimation of error rates
Rule-based computational assistant
Accuracy on par with data-driven estimates
While using cheaper data & scalable analysis
Preferred when little or no data available
Computational
assistant
Detector model
Monitor’s error rate
Detector data
Monitor spec

72. 72
Summary
● Logical composition of detectors with LTL3d
●
Probabilistic estimation of error rates
– Rule-based computational assistant
Accuracy on par with data-driven estimates
While using cheaper data & scalable analysis
Preferred when little or no data available
Computational
assistant
Detector model
Monitor’s error rate
Detector data
Monitor spec

73. 73
Summary
● Logical composition of detectors with LTL3d
●
Probabilistic estimation of error rates
– Rule-based computational assistant
●
Accuracy on par with data-driven estimates
– While using cheaper data & scalable analysis
Computational
assistant
Detector model
Monitor’s error rate
Detector data
Monitor spec

74. 74
Refences
●
The computational assistant and UUV case study data:
https://github.com/bisc/prob-comp-asst
●
The original paper:
https://dx.doi.org/10.1109/TCAD.2020.3012643
●
Supplementary materials:
https://www.researchgate.net/publication/342993188_Suppl
ementary_Materials_for_Compositional_Probabilistic_Analy
sis_of_Temporal_Properties_over_Stochastic_Detectors

75. 75
Additional slides

76. 76
Our framework at glance
Inputs:
– Set of detectors with event/error probabilities
– Logical property over detectors
– Detector independence assumptions
– Labeled traces from detectors
Method: a modeling & analysis framework based on
– Logical composition model
– Algebraic probability calculations
– Axiomatic independence inference
– Probability estimation from data
Output: estimate of an error rate for the property monitor

77. 77
Our framework at glance

78. 78
Detector model as variables
● Given: mutually exclusive H1 and H0
●
Atomic detector D is a pair (DO, GT) of r.v.s:
– Detection outcome DO ∈ {T, F, U}
– Ground truth GT ∈ {T, F}
●
A probability space with marginal events:
dot(D) = (T, *) gtt(D) = (*, T)
dof(D) = (F, *) gtf(D) = (*, F)
dou(D) = (U, *)

79. 79
Detector model as probability space
Given: mutually exclusive H1 and H0
Stochastic detector D – a triple (Ω, 𝓕, Pr):
– Ω: set of six possible outcomes for a pair of variables:
Detection Outcome (DO) ∈ {T, F, U}
Ground Truth (GT) ∈ {T, F}
– 𝓕: sigma-algebra of events over Ω (i.e., the powerset of Ω)
dot = (T, *), dof = (F, *), dou = (U, *)
gtt = (*, T), gtf = (*, F)
– Pr: 𝓕 → [0, 1], a probability measure over 𝓕
D T F U
T
dot dof dou
F
D T F U
T gtt
F gtf
D T F U
T o1
o2
o3
F o4
o5
o6

80. 80
Detector examples
● Coin toss with faulty perception: H1 – heads, H0 – tails
● Detecting obstacles: H1 – obstacle ahead, H0 – no obstacle
● Perfect coin toss: H1 – heads, H0 – tails
OD dot dof dou
gtt 0.2 0.09 0.09
gtf 0.01 0.6 0.01
PCT dot dof dou
gtt 0.5 0 0
gtf 0 0.5 0
FCT dot dof dou
gtt 0.3 0.05 0.15
gtf 0.05 0.3 0.15

81. 81
Composition of detectors (general)
● Operand detectors D1, D2 … DN
– A value pair (DO, GT) for each
●
Composition operator op
– 3-value version op3
– 2-value version op2
●
Composite detector is (DO, GT) such that
– DO = op3(DO1, … DON )
– GT = op2(GT1, … GTN )

82. 82
Composition of detectors (ours)
● Operand detectors D1, D2 … DN
– A trace of value pairs (DO, GT) for each
●
LTL-like formula φ
– 3-value semantics [[[φ]]]
– 2-value semantics [[φ]]
●
Composite detector is (DO, GT) such that
– DO = [[[φ(DO1, … DON )]]]
– GT = [[φ(GT1, … GTN )]]

83. 83
Conjunction composition

84. 84
Strong negation
●
Flips hypotheses/outcomes
D CH1
CH0
NC
H1
0.3 0.05 0.15
H0
0.05 0.3 0.15
¬s
D CH1
CH0
NC
H1
0.05 0.3 0.15
H0
0.3 0.05 0.15

85. 85
Weak negation
●
Flips hypotheses, true on unknown
D CH1
CH0
NC
H1
0.3 0.05 0.15
H0
0.05 0.3 0.15
¬w
D CH1
CH0
NC
H1
0.45 0.05 0
H0
0.2 0.3 0

86. 86
Strong exclusive negation
●
Flips hypotheses, false on unknown
D CH1
CH0
NC
H1
0.3 0.05 0.15
H0
0.05 0.3 0.15
¬se
D CH1
CH0
NC
H1
0.3 0.2 0
H0
0.05 0.45 0

87. 87
Binary composition example
Two independent faulty coin tosses
FCT1
Λ FCT2 dot dof dou
gtt 0.09 0.0475 0.1125
gtf 0.0325 0.53 0.1875
FCT1 dot dof dou
gtt 0.3 0.05 0.15
gtf 0.05 0.3 0.15
FCT2
ch1 dof dou
gtt 0.3 0.05 0.15
gtf 0.05 0.3 0.15
FCT1
V FCT2 dot dof dou
gtt 0.53 0.0325 0.1875
gtf 0.0475 0.09 0.1125

88. 88
Negations, implications,
modalities
Strong negation: ¬s D
Weak negation: ¬w D
Strong exclusive negation: ¬se D
Strong implication: D1→s D2 := ¬s D1 V D2
Weak implication: D1 →w D2 := ¬w D1 V D2
Strong exclusive implication: D1 →se D2 := ¬se D1 V D2
Box modality:
Diamond modality:

89. 89
Operator semantics

90. 90
Error rates of detectors
●
False positive rate:
●
False negative rate:

91. 91
Error rates for examples
● Coin toss with faulty perception: H1 – heads, H0 – tails
● Detecting obstacles: H1 – obstacle ahead, H0 – no obstacle
● Perfect coin toss: H1 – heads, H0 – tails
FCT dot dof dou
gtt 0.3 0.05 0.15
gtf 0.05 0.3 0.15
DO dot dof dou
gtt 0.2 0.09 0.09
gtf 0.01 0.6 0.01
PCT dot dof dou
gtt 0.5 0 0
gtf 0 0.5 0
fpr = 0.05
fnr = 0.2
fpr = 0.01
fnr = 0.18
fpr = 0
fnr = 0

92. 92
Error rates for composition example
Two independent coin tosses
with faulty perception
CT1
Λ CT2 dot dof dou
gtt 0.09 0.0475 0.1125
gtf 0.0325 0.53 0.1875
CT1 dot dof dou
gtt 0.3 0.05 0.15
gtf 0.05 0.3 0.15
CT2 dot dof dou
gtt 0.3 0.05 0.15
gtf 0.05 0.3 0.15
CT1
V CT2 dot dof dou
gtt 0.53 0.0325 0.1875
gtf 0.0475 0.09 0.1125
fpr = 0.05, fnr = 0.2
fpr = 0.0325, fnr = 0.155 fpr = 0.0475, fnr = 0.22

93. 93
Reasoning rules
● 𝓡log – tautologies of LTL3d
● 𝓡ev – tautologies of event predicates
● 𝓡prob – algebraic probability rules
● 𝓡indep – conditional independence rules

94. 94
Related work: combining
detectors
●
Ensembles and cross-validation
– Good performance, weak guarantees
●
Hypothesis tests
– Either too simple or too complex
●
Logics of probability (probability operator)
– No need for explicit statements about probability
●
Probabilistic logics (probability of formulas)
– No need for uncertain statements

95. 95
Related work: predicting error
rates
●
Model-free
– E.g., estimation from data
– Requires extensive data collection & labeling
– Difficulty in estimating rare cases
●
Model-based
– E.g., model checking probabilistic automata
– Requires comprehensive (~ inaccurate) modeling
– Limited scalability