This document discusses holistic model-based testing for business information systems. It identifies the problem of test-related information being spread across multiple analysis models. The document proposes solving this by taking a holistic view of the analysis models and reusing them to generate basic test models, which can then be extended and used to generate test cases. It outlines a model-based test specification process and discusses results from applying the approach. The holistic view is shown to improve test case generation and coverage measurement across multiple models.

1. Holistic Model-Based Testing
for Business Information Systems
Michael Mlynarski
08.04.2010, Paris

2. Intro

3. Generating test cases?
- theory -

4. We can generate test cases from models!
Mingsong et al.
Binder
...
Ostrand et al.
Rumpe
Test Cases
Pretschner et al.
Model
Vieira et al.
Hartmann et al.
Offutt et al.
Bertolino et al.
...
Güldali et al.
Weißleder

5. Generating test cases?
- practice -

6. There are several types of models!
M
Requirements O
Acceptance
Testing
DEL
S
MOD
Analysis
System
Testing
ELS
MOD
Subsystem
Design
ELS
Integration
Testing
Component
MOD
ELS
Design
Code
Component
Testing
Legend:
time order
validation
Activity
Artifact

7. Holistic view on analysis models is needed...
Behaviour
Create
course
Entry
data
Search
course
Book
attendee
Structure
Save
data
Course
name
type
date
Attendee
name
age
skill
Customer
name
member
Interaction
Start
Dialog
Text
Text
Text Text
Text
Text
Course
Dialog
Attendee
Dialog
olistic
H
view

8. What is the problem?

9. Problem decomposition
Question:
Holistic view
in testing
Holistic view
in testing
How to use the holistic view on analysis
models for test generation purposes?
Justification:
Test-related information are spread
accross several analysis models.
Subproblem #1
Subproblem #2

10. lem
ob
r
bp
Su
#1
Reuse of analysis models
Behaviour
Create
course
Entry
data
Search
course
Book
attendee
Structure
How to reuse
analysis models for
test generation?
Save
data
Course
name
type
date
Attendee
name
age
skill
Customer
name
member
∆
test info
???
Interaction
Start
Dialog
Text
Text
Text Text
Text
Text
Course
Dialog
Attendee
Dialog
Requirements
Test Models
Code
AnalysisCases
Güldali, B. and Mlynarski, M. and Sancar, Y.
Effort Comparison for Model-Based Testing Scenarios,
Quombat workshop, ICST 2010 (to be published)
Test Model
Test Cases

11. lem
rob
p
ub
#2
Model coverage measurement
S
Behaviour
Create
course
Entry
data
Search
course
Book
attendee
Structure
cov
Save
data
Course
name
type
date
er
Attendee
name
age
skill
How to
measure the model
generate
covercoverage of several
models?
Customer
name
member
ver
co
Interaction
Start
Dialog
Text
Text
Text Text
Text
Text
Course
Dialog
Attendee
Dialog
Test Cases

12. Problem decomposition
Holistic view
in testing
Question:
Reuse of analysis
models
How to reuse analysis models for test
model creation?
Justification:
Test designers need a separate model
to model additional test information.
Question:
Model coverage
measurement
How to measure the model coverage of
several models?
Justification:
Quality of a holistic approach test approach
is measurable through model coverage.

13. Hypothesis
‘‘
‚‚
High model coverage within the test case
generation can be reached by using the holistic
view of models created by business analysts.

14. Related work
Holistic view
in testing
no model
relations
Dias Neto, A. and Subramanyan, R. and Vieira, M. and Tracassos, G.
A Survey on Model-based Testing Approaches: A Systematic Review
Siemens Corporate Research, 2007
Briand, L. C. and Labiche, Y.
A UML-Based Approach to System Testing
UML' '01: Proceedings of the 4th International Conference on The Unified Modeling Language, Modeling Languages,
Concepts, and Tools, Springer Berlin / Heidelberg, 2001, 194-208
Reuse of analysis
models
no test
selection
Dai, Z. R. and Grabowski, J. and Neukirchen, H. and Pals, H.
From Design to Test with UML -- Applied to a Roaming Algorithm for Bluetooth Devices
Proceedings of the 16th IFIP International Conference on Testing of Communicating Systems
LNCS 2978, Springer Verlag, 2004
Torres, A. and Escalona, M. and Mejias, M. and Gutiérrez, J.
A MDA-Based Testing. A comparative study.
Proceedings of 4th international conference on Software and Data Technologies ICSOFT,
INSTICC Press, 2009
no criteria
combination
Model coverage
measurement
Mc Quillan, J. A. and Power, J. F.
A Survey of UML-Based Coverage Criteria for Software Testing
Department of Computer Science, 2005
Andrews, A. A. and France, R. B. and Ghosh, S. and Craig, G.
Test adequacy criteria for UML design models
Software Testing, Verification Reliability, 2003, 13, 95-127

15. What is the solution?

16. Model-Based Test Specification Process
Process
Coverage t
en
measurem
Test
Selection
Analyze and
annotate
test basis
Generate basic
test model
Extend basic
test model
Generate test
cases
M2M
Artifacts
Analysis
model
is_transformed_to
instance of
Analysis
metamodel
UML
Basic
test
model
extends
Extended
test
model
instance of
Test
metamodel
generated_from
Test cases
UML
Testing
Profile
Mlynarski, M. and Güldali, B. and Späth, M. and Engels, G.
From Design Models to Test Models by Means of Test Ideas,
In Proceedings of the 6th International Workshop on Model-Driven Engineering, Verification and Validation,
Springer, 2009, 1-10

17. Current results

18. Results
Mlynarski, M. and Güldali, B. and Späth, M. and Engels, G.
From Design Models to Test Models by Means of Test Ideas
MoDeVVa '09: Proceedings of the 6th International Workshop on Model-Driven Engineering, Verification and Validation,
Springer, 2009, pp. 1-10
Güldali, B. and Wübbeke, A. and Mlynarski, M. and Engels, G.
Model-Based Testing using Visual Contracts
Proceedings of Euromicro SEAA Conference 2009, Special Session on “Model Driven Engineering”
IEEE Computer Society, 2009, 121-124
Beulen, D. and Güldali, B.and Mlynarski, M.
Vergleich von Prozessmodellen für MBT aus Managementsicht,
Softwaretechnik-Trends, 3/2010 (to be published)
Güldali, B. and Mlynarski, M. and Sancar, Y.
Effort Comparison for Model-Based Testing Scenarios,
Proceedings of the 1st Workshop on Quality of MBT
Springer, 2010 (to be published)
Güldali, B. and Jungmayr, S. and Mlynarski, M. and Neumann, S. and Winter, M.
Starterhilfe für modellbasiertes Testen: Entscheidungsunterstützung für Projekt- und Testmanager
Objektspektrum, 3/2010, 2010 (to be published)

19. Summary & Outlook
Process
Holistic view
in testing
✓
Coverage t
en
measurem
Test
Selection
Analyze and
annotate
test basis
Extend basic
test model
Generate basic
test model
Generate test
cases
M2M
Reuse of analysis
models
Artifacts
Analysis
model
transform
Basic
test
model
instance of
instance of
Model coverage
measurement
Problem
extend
Analysis
metamodel
Test
metamodel
Solution
Evaluation @
Extended
test
model
generate
✓

20. Discussion

21. Michael Mlynarski
E-Mail: mmlynarski@s-lab.upb.de
Mobile: 0176 24543671
Twitter: michalmm
Software Quality Lab
Warbuger Str. 100
33098 Paderborn
EMDT Workshop
16.09.2010 Illmenau (DE)
http://proinf.de/emdt2010/