Diese Präsentation wurde erfolgreich gemeldet.
Wir verwenden Ihre LinkedIn Profilangaben und Informationen zu Ihren Aktivitäten, um Anzeigen zu personalisieren und Ihnen relevantere Inhalte anzuzeigen. Sie können Ihre Anzeigeneinstellungen jederzeit ändern.
Where did I go wrong? 
Explaining errors in process models 
Niels Lohmann @nlohmann 
Dirk Fahland @dfahland
Verification of processes and services 
2 
process model 
property 
verification technique 
diagnostic information
Verification of processes and services 
3 
verification technique 
diagnostic information 
BPMN 
Soundness 
domain-specifi...
Verification of processes and services 
4 
CMMN 
Declare 
WS-BPEL 
WS-Policy 
EPC 
BPMN 
YAWL 
Object Life-Cycles 
GSM 
Ru...
Verification of processes and services 
5 
CMMN 
Declare 
WS-BPEL 
WS-Policy 
EPC 
BPMN 
YAWL 
Object Life-Cycles 
GSM 
Ru...
Model checking 
general purpose verification approach: 
1. formalize model and specification* 
2. push a button 
6 
*can b...
Effectiveness and efficiency 
-model checking works in reality 
-successful applications in many domains 
-very fast: “ver...
Diagnosis 
-in case of error: outputs target state and produce a witness path 
-describes how target state can be reached ...
Diagnosis: the bad 
PATH process.s00000823##s00006200.inputCriterion.s00001053 fork.s00001071.activate.s00001072 fork.s000...
This talk: better diagnosis 
PATH process.s00000823##s00006200.inputCriterion.s00001053 fork.s00001071.activate.s00001072 ...
This talk: better diagnosis 
path 
essential path 
distill 
Why useless?
Reasons for useless paths 
12 
detours 
depth-first search 
indisputable parts 
bootstrapping 
interleavings 
concurrency
Running example 
13 
lack of synchronization
Reduction: obvious parts 
-classify transitions 
-only report points of alternative continuations* 
14 
* XOR-gateways, ev...
Reduction: obvious parts 
15 
t1 
t2 
t9 
t10 
t11 
t12 
t14 
t8 
t2 
t3 
t4 
t5 
“down” 
“down” 
“up”
Non-obvious “core” of a path ≈ 10-25% 
16
Reduction: spurious decisions 
-can be found by model checking 
-results: 50%-80% spurious, occasionally no reduction (tim...
Reasons for useless paths 
18 
detours depth-first search 
indisputable parts bootstrapping 
interleavings 
concurrency
Reduction: unorder steps 
-idea: show independence of steps ( partially ordered runs) 
-makes synchronization points (mil...
Reduction: unorder steps 
20 
t1 
t2 
t9 
t10 
t11 
t12 
t14 
t8 
t2 
t3 
t4 
t5 
t9 
t14 
t3
More aid: preserve reference points 
21 
t1 
t2 
t9 
t10 
t11 
t12 
t14 
t8 
t2 
t3 
t4 
t5 
t9 
t14 
t3 
p1 
p6 
p6 
t5 
...
Final: remove obvious/spurious parts 
22 
t1 
t2 
t9 
t10 
t11 
t12 
t14 
t8 
t2 
t3 
t4 
t5 
t9 
t14 
t3 
p1 
p6 
p6 
t5 ...
Essential path: find source of error 
PATH 
process.s00000823##s00006200.inputCriterion.s00001053 
fork.s00001071.activate...
Results: typical reduced paths 
24 
2x lack of synchronization 
improper completion 
deadlock
Summary 
-general purpose verification more user friendly 
-paths  partial order of important decisions 
-applicable to a...
Where did I go wrong? 
Explaining errors in process models 
Niels Lohmann
Nächste SlideShare
Wird geladen in …5
×

Where did I go wrong? Explaining errors in process models

934 Aufrufe

Veröffentlicht am

This presentation shows how to reduce diagnostic information returned by general purpose model checkers (counter example paths) to essential parts that help understanding the error. The presentation has been given at the 12th International Conference on Business Process Management (BPM'14), September 2014 in Eindhoven.

Veröffentlicht in: Daten & Analysen
  • Als Erste(r) kommentieren

Where did I go wrong? Explaining errors in process models

  1. 1. Where did I go wrong? Explaining errors in process models Niels Lohmann @nlohmann Dirk Fahland @dfahland
  2. 2. Verification of processes and services 2 process model property verification technique diagnostic information
  3. 3. Verification of processes and services 3 verification technique diagnostic information BPMN Soundness domain-specific high-quality
  4. 4. Verification of processes and services 4 CMMN Declare WS-BPEL WS-Policy EPC BPMN YAWL Object Life-Cycles GSM Rules Soundness Security Compliance Conformance to … diagnostic information verification technique domain-specific high-quality -moving target -domain-specific approaches too specific to follow
  5. 5. Verification of processes and services 5 CMMN Declare WS-BPEL WS-Policy EPC BPMN YAWL Object Life-Cycles GSM Rules Soundness Security Compliance Conformance to … diagnostic information verification technique high-quality general purpose
  6. 6. Model checking general purpose verification approach: 1. formalize model and specification* 2. push a button 6 *can be hidden from the user
  7. 7. Effectiveness and efficiency -model checking works in reality -successful applications in many domains -very fast: “verify while you model” 7
  8. 8. Diagnosis -in case of error: outputs target state and produce a witness path -describes how target state can be reached -operational semantics: can be simulated 8 witness path target state
  9. 9. Diagnosis: the bad PATH process.s00000823##s00006200.inputCriterion.s00001053 fork.s00001071.activate.s00001072 fork.s00001071.fire.s00001078 merge.s00001061.activate.s00001065 merge.s00001061.fire.s00001069 callToTask.s00006202.inputCriterion.s00001053 callToTask.s00006202.outputCriterion.s00001055 callToTask.s00006211.inputCriterion.s00001053 callToTask.s00006211.outputCriterion.s00001055 callToTask.s00006209.inputCriterion.s00001053 callToTask.s00006209.outputCriterion.s00001055 decision.s00001158.activate.s00001072 decision.s00001158.fire.s00001075 merge.s00001160.activate.s00001064 merge.s00001160.fire.s00001069 callToTask.s00006203.inputCriterion.s00001053 callToTask.s00006203.outputCriterion.s00001055 callToTask.s00006214.inputCriterion.s00001053 callToTask.s00006214.outputCriterion.s00001055 callToTask.s00006213.inputCriterion.s00001053 callToTask.s00006213.outputCriterion.s00001055 decision.s00001840.activate.s00001072 decision.s00001840.fire.s00001075 callToTask.s00006201.inputCriterion.s00001053 callToTask.s00006201.outputCriterion.s00001055 decision.s00001123.activate.s00001072 decision.s00001123.fire.s00001075 merge.s00001161.activate.s00001064 merge.s00001161.fire.s00001069 callToTask.s00006208.inputCriterion.s00001053 callToTask.s00006208.outputCriterion.s00001055 decision.s00001157.activate.s00001072 decision.s00001157.fire.s00001073 fork.s00001071.fire.s00001073 merge.s00001061.activate.s00001064 join.s00001163.activate.s00001062 merge.s00001061.fire.s00001069 join.s00001163.activate.s00001064 merge.s00001162.activate.s00001062 merge.s00001162.fire.s00001069 callToTask.s00006210.inputCriterion.s00001053 callToTask.s00006210.outputCriterion.s00001055 decision.s00001159.activate.s00001072 decision.s00001159.fire.s00001073 join.s00001163.activate.s00001065 join.s00001163.fire.s00001069 fork.s00001071.fire.s00001075 merge.s00001160.activate.s00001065 callToTask.s00006207.inputCriterion.s00001053 merge.s00001160.fire.s00001069 callToTask.s00006207.outputCriterion.s00001055 callToTask.s00006203.inputCriterion.s00001053 decision.s00001126.activate.s00001072 callToTask.s00006203.outputCriterion.s00001055 decision.s00001126.fire.s00001073 callToTask.s00006214.inputCriterion.s00001053 callToTask.s00006202.inputCriterion.s00001053 callToTask.s00006202.outputCriterion.s00001055 callToTask.s00006211.inputCriterion.s00001053 callToTask.s00006211.outputCriterion.s00001055 callToTask.s00006209.inputCriterion.s00001053 callToTask.s00006209.outputCriterion.s00001055 decision.s00001158.activate.s00001072 decision.s00001158.fire.s00001075 callToTask.s00006214.outputCriterion.s00001055 callToTask.s00006213.inputCriterion.s00001053 callToTask.s00006213.outputCriterion.s00001055 decision.s00001840.activate.s00001072 decision.s00001840.fire.s00001075 callToTask.s00006201.inputCriterion.s00001053 callToTask.s00006201.outputCriterion.s00001055 decision.s00001123.activate.s00001072 decision.s00001123.fire.s00001073 callToTask.s00006204.inputCriterion.s00001053 callToTask.s00006204.outputCriterion.s00001055 callToTask.s00003714.inputCriterion.s00001053 callToTask.s00003714.outputCriterion.s00001055 callToTask.s00006215.inputCriterion.s00001053 callToTask.s00006215.outputCriterion.s00001055 callToTask.s00006206.inputCriterion.s00001053 join.s00001163.activate.s00001064 callToTask.s00006206.outputCriterion.s00001055 callToTask.s00006205.inputCriterion.s00001053 callToTask.s00006205.outputCriterion.s00001055 merge.s00001161.activate.s00001062 merge.s00001161.fire.s00001069 callToTask.s00006208.inputCriterion.s00001053 callToTask.s00006208.outputCriterion.s00001055 decision.s00001157.activate.s00001072 decision.s00001157.fire.s00001073 join.s00001163.activate.s00001062 callToTask.s00006212.inputCriterion.s00001053 callToTask.s00006212.outputCriterion.s00001055 merge.s00001162.activate.s00001064 merge.s00001162.fire.s00001069 callToTask.s00006210.inputCriterion.s00001053 callToTask.s00006210.outputCriterion.s00001055 decision.s00001159.activate.s00001072 decision.s00001159.fire.s00001073 join.s00001163.activate.s00001065 join.s00001163.fire.s00001069 callToTask.s00006207.inputCriterion.s00001053 callToTask.s00006207.outputCriterion.s00001055 decision.s00001126.activate.s00001072 decision.s00001126.fire.s00001073 STATE decision.s00001126.output.s00001054 : 2 -paths can become very long -length correlates with size of the model -reports all events equally: disregarding importance
  10. 10. This talk: better diagnosis PATH process.s00000823##s00006200.inputCriterion.s00001053 fork.s00001071.activate.s00001072 fork.s00001071.fire.s00001078 merge.s00001061.activate.s00001065 merge.s00001061.fire.s00001069 callToTask.s00006202.inputCriterion.s00001053 callToTask.s00006202.outputCriterion.s00001055 callToTask.s00006211.inputCriterion.s00001053 callToTask.s00006211.outputCriterion.s00001055 callToTask.s00006209.inputCriterion.s00001053 callToTask.s00006209.outputCriterion.s00001055 decision.s00001158.activate.s00001072 decision.s00001158.fire.s00001075 merge.s00001160.activate.s00001064 merge.s00001160.fire.s00001069 callToTask.s00006203.inputCriterion.s00001053 callToTask.s00006203.outputCriterion.s00001055 callToTask.s00006214.inputCriterion.s00001053 callToTask.s00006214.outputCriterion.s00001055 callToTask.s00006213.inputCriterion.s00001053 callToTask.s00006213.outputCriterion.s00001055 decision.s00001840.activate.s00001072 decision.s00001840.fire.s00001075 callToTask.s00006201.inputCriterion.s00001053 callToTask.s00006201.outputCriterion.s00001055 decision.s00001123.activate.s00001072 decision.s00001123.fire.s00001075 merge.s00001161.activate.s00001064 merge.s00001161.fire.s00001069 callToTask.s00006208.inputCriterion.s00001053 callToTask.s00006208.outputCriterion.s00001055 decision.s00001157.activate.s00001072 decision.s00001157.fire.s00001073 fork.s00001071.fire.s00001073 merge.s00001061.activate.s00001064 join.s00001163.activate.s00001062 merge.s00001061.fire.s00001069 join.s00001163.activate.s00001064 merge.s00001162.activate.s00001062 merge.s00001162.fire.s00001069 callToTask.s00006210.inputCriterion.s00001053 callToTask.s00006210.outputCriterion.s00001055 decision.s00001159.activate.s00001072 decision.s00001159.fire.s00001073 join.s00001163.activate.s00001065 join.s00001163.fire.s00001069 fork.s00001071.fire.s00001075 merge.s00001160.activate.s00001065 callToTask.s00006207.inputCriterion.s00001053 merge.s00001160.fire.s00001069 callToTask.s00006207.outputCriterion.s00001055 callToTask.s00006203.inputCriterion.s00001053 decision.s00001126.activate.s00001072 callToTask.s00006203.outputCriterion.s00001055 decision.s00001126.fire.s00001073 callToTask.s00006214.inputCriterion.s00001053 callToTask.s00006202.inputCriterion.s00001053 callToTask.s00006202.outputCriterion.s00001055 callToTask.s00006211.inputCriterion.s00001053 callToTask.s00006211.outputCriterion.s00001055 callToTask.s00006209.inputCriterion.s00001053 callToTask.s00006209.outputCriterion.s00001055 decision.s00001158.activate.s00001072 decision.s00001158.fire.s00001075 callToTask.s00006214.outputCriterion.s00001055 callToTask.s00006213.inputCriterion.s00001053 callToTask.s00006213.outputCriterion.s00001055 decision.s00001840.activate.s00001072 decision.s00001840.fire.s00001075 callToTask.s00006201.inputCriterion.s00001053 callToTask.s00006201.outputCriterion.s00001055 decision.s00001123.activate.s00001072 decision.s00001123.fire.s00001073 callToTask.s00006204.inputCriterion.s00001053 callToTask.s00006204.outputCriterion.s00001055 callToTask.s00003714.inputCriterion.s00001053 callToTask.s00003714.outputCriterion.s00001055 callToTask.s00006215.inputCriterion.s00001053 callToTask.s00006215.outputCriterion.s00001055 callToTask.s00006206.inputCriterion.s00001053 join.s00001163.activate.s00001064 callToTask.s00006206.outputCriterion.s00001055 callToTask.s00006205.inputCriterion.s00001053 callToTask.s00006205.outputCriterion.s00001055 merge.s00001161.activate.s00001062 merge.s00001161.fire.s00001069 callToTask.s00006208.inputCriterion.s00001053 callToTask.s00006208.outputCriterion.s00001055 decision.s00001157.activate.s00001072 decision.s00001157.fire.s00001073 join.s00001163.activate.s00001062 callToTask.s00006212.inputCriterion.s00001053 callToTask.s00006212.outputCriterion.s00001055 merge.s00001162.activate.s00001064 merge.s00001162.fire.s00001069 callToTask.s00006210.inputCriterion.s00001053 callToTask.s00006210.outputCriterion.s00001055 decision.s00001159.activate.s00001072 decision.s00001159.fire.s00001073 join.s00001163.activate.s00001065 join.s00001163.fire.s00001069 callToTask.s00006207.inputCriterion.s00001053 callToTask.s00006207.outputCriterion.s00001055 decision.s00001126.activate.s00001072 decision.s00001126.fire.s00001073 STATE decision.s00001126.output.s00001054 : 2
  11. 11. This talk: better diagnosis path essential path distill Why useless?
  12. 12. Reasons for useless paths 12 detours depth-first search indisputable parts bootstrapping interleavings concurrency
  13. 13. Running example 13 lack of synchronization
  14. 14. Reduction: obvious parts -classify transitions -only report points of alternative continuations* 14 * XOR-gateways, events, exceptions, … assume progress of flow
  15. 15. Reduction: obvious parts 15 t1 t2 t9 t10 t11 t12 t14 t8 t2 t3 t4 t5 “down” “down” “up”
  16. 16. Non-obvious “core” of a path ≈ 10-25% 16
  17. 17. Reduction: spurious decisions -can be found by model checking -results: 50%-80% spurious, occasionally no reduction (timeout) 17 p1 p3 p2 p4 p5 p6 p1 p3 p5 p6 genuine decision spurious decision = irrelevant for outcome
  18. 18. Reasons for useless paths 18 detours depth-first search indisputable parts bootstrapping interleavings concurrency
  19. 19. Reduction: unorder steps -idea: show independence of steps ( partially ordered runs) -makes synchronization points (milestones) explicit 19 independent steps many paths to same goal state order of steps irrelevant
  20. 20. Reduction: unorder steps 20 t1 t2 t9 t10 t11 t12 t14 t8 t2 t3 t4 t5 t9 t14 t3
  21. 21. More aid: preserve reference points 21 t1 t2 t9 t10 t11 t12 t14 t8 t2 t3 t4 t5 t9 t14 t3 p1 p6 p6 t5 t11 t10 t1
  22. 22. Final: remove obvious/spurious parts 22 t1 t2 t9 t10 t11 t12 t14 t8 t2 t3 t4 t5 t9 t14 t3 p1 p6 p6 t5 t11 t10 t1
  23. 23. Essential path: find source of error PATH process.s00000823##s00006200.inputCriterion.s00001053 fork.s00001071.activate.s00001072 fork.s00001071.fire.s00001078 merge.s00001061.activate.s00001065 merge.s00001061.fire.s00001069 callToTask.s00006202.inputCriterion.s00001053 callToTask.s00006202.outputCriterion.s00001055 callToTask.s00006211.inputCriterion.s00001053 callToTask.s00006211.outputCriterion.s00001055 callToTask.s00006209.inputCriterion.s00001053 callToTask.s00006209.outputCriterion.s00001055 decision.s00001158.activate.s00001072 decision.s00001158.fire.s00001075 merge.s00001160.activate.s00001064 merge.s00001160.fire.s00001069 callToTask.s00006203.inputCriterion.s00001053 callToTask.s00006203.outputCriterion.s00001055 callToTask.s00006214.inputCriterion.s00001053 callToTask.s00006214.outputCriterion.s00001055 callToTask.s00006213.inputCriterion.s00001053 callToTask.s00006213.outputCriterion.s00001055 decision.s00001840.activate.s00001072 decision.s00001840.fire.s00001075 callToTask.s00006201.inputCriterion.s00001053 callToTask.s00006201.outputCriterion.s00001055 decision.s00001123.activate.s00001072 decision.s00001123.fire.s00001075 merge.s00001161.activate.s00001064 merge.s00001161.fire.s00001069 callToTask.s00006208.inputCriterion.s00001053 callToTask.s00006208.outputCriterion.s00001055 decision.s00001157.activate.s00001072 decision.s00001157.fire.s00001073 fork.s00001071.fire.s00001073 merge.s00001061.activate.s00001064 join.s00001163.activate.s00001062 merge.s00001061.fire.s00001069 join.s00001163.activate.s00001064 merge.s00001162.activate.s00001062 merge.s00001162.fire.s00001069 callToTask.s00006210.inputCriterion.s00001053 callToTask.s00006210.outputCriterion.s00001055 decision.s00001159.activate.s00001072 decision.s00001159.fire.s00001073 join.s00001163.activate.s00001065 join.s00001163.fire.s00001069 fork.s00001071.fire.s00001075 merge.s00001160.activate.s00001065 callToTask.s00006207.inputCriterion.s00001053 merge.s00001160.fire.s00001069 callToTask.s00006207.outputCriterion.s00001055 callToTask.s00006203.inputCriterion.s00001053 decision.s00001126.activate.s00001072 callToTask.s00006203.outputCriterion.s00001055 decision.s00001126.fire.s00001073 callToTask.s00006214.inputCriterion.s00001053 callToTask.s00006202.inputCriterion.s00001053 callToTask.s00006202.outputCriterion.s00001055 callToTask.s00006211.inputCriterion.s00001053 callToTask.s00006211.outputCriterion.s00001055 callToTask.s00006209.inputCriterion.s00001053 callToTask.s00006209.outputCriterion.s00001055 decision.s00001158.activate.s00001072 decision.s00001158.fire.s00001075 callToTask.s00006214.outputCriterion.s00001055 callToTask.s00006213.inputCriterion.s00001053 callToTask.s00006213.outputCriterion.s00001055 decision.s00001840.activate.s00001072 decision.s00001840.fire.s00001075 callToTask.s00006201.inputCriterion.s00001053 callToTask.s00006201.outputCriterion.s00001055 decision.s00001123.activate.s00001072 decision.s00001123.fire.s00001073 callToTask.s00006204.inputCriterion.s00001053 callToTask.s00006204.outputCriterion.s00001055 callToTask.s00003714.inputCriterion.s00001053 callToTask.s00003714.outputCriterion.s00001055 callToTask.s00006215.inputCriterion.s00001053 callToTask.s00006215.outputCriterion.s00001055 callToTask.s00006206.inputCriterion.s00001053 join.s00001163.activate.s00001064 callToTask.s00006206.outputCriterion.s00001055 callToTask.s00006205.inputCriterion.s00001053 callToTask.s00006205.outputCriterion.s00001055 merge.s00001161.activate.s00001062 merge.s00001161.fire.s00001069 callToTask.s00006208.inputCriterion.s00001053 callToTask.s00006208.outputCriterion.s00001055 decision.s00001157.activate.s00001072 decision.s00001157.fire.s00001073 join.s00001163.activate.s00001062 callToTask.s00006212.inputCriterion.s00001053 callToTask.s00006212.outputCriterion.s00001055 merge.s00001162.activate.s00001064 merge.s00001162.fire.s00001069 callToTask.s00006210.inputCriterion.s00001053 callToTask.s00006210.outputCriterion.s00001055 decision.s00001159.activate.s00001072 decision.s00001159.fire.s00001073 join.s00001163.activate.s00001065 join.s00001163.fire.s00001069 callToTask.s00006207.inputCriterion.s00001053 callToTask.s00006207.outputCriterion.s00001055 decision.s00001126.activate.s00001072 decision.s00001126.fire.s00001073
  24. 24. Results: typical reduced paths 24 2x lack of synchronization improper completion deadlock
  25. 25. Summary -general purpose verification more user friendly -paths  partial order of important decisions -applicable to any verification goal -keep reference points to aid diagnosis Next steps -error localization vs. explanation -detect useless cycles -How should a good diagnosis for $problem look like? 25
  26. 26. Where did I go wrong? Explaining errors in process models Niels Lohmann

×