Speaker: Iosif Itkin, co-CEO & co-founder, Exactpro Systems 9th November 2021 Hilton Canary Wharf Exactpro is an independent software testing business focused on mission-critical financial market infrastructures, primarily exchanges and clearing houses. In his presentation, Iosif will give a brief overview of research on the concept of model-based testing and the principal challenges of its application while testing complex distributed systems. He will also outline the broader context of interaction between humans and complex computer models.

1. Automation in Software Testing.
Humans and Complex Models.
Iosif Itkin
CEO and co-founder, Exactpro

2. About Exactpro

4. MANAGED SUPPORT
SERVICES PROVIDER OF
THE YEAR
In this Risk.net interview, Alexey Zverev, Exactpro co-CEO and co-
founder, talks about mitigating risks while driving innovation during the
time of high volatility and explains what made Exactpro stand out among
the contestants. Watch the interview with Risk.net via the QR code above.

5. ● https://www.linkedin.com/in/iosifitkin/
● Exactpro representative at the Zero Outage Industry Standard association
● Head of LOST (Laboratory of Software Testing) at Tomsk Polytechnic University
● Co-chair of FIX Trading Community MOST Working Group (Monitoring, Onboarding and Software Testing)
About me

6. What is Software Testing

7. Art, Craft, Science, Engineering
Where does software testing belongs on this continuum?

8. Science and Engineering
https://fs.blog/2013/07/the-difference-between-science-and-engineering/
Abstract Model
= theory
Concrete
Description
= data
Physical System
= object of study
flow
of
information
measure
compare
Abstract Model
= design concept
Concrete
Description
= specification
Physical System
= useful product
flow
of
information
design
produce
Scientific Inquiry Engineering Design
The Antiparallel Structures of Scientific Inquiry and Engineering Design

9. Software Testing
Software Testing is exploring software with the intent of finding bugs.
Software Testing is an empirical technical investigation conducted to provide
stakeholders with information about the quality of the product or service under test.

10. Software Testing
Abstract Model
= theory
Concrete Description
= data
Software
= object of study
flow
of
information
observe
compare
Abstract Model
= design concept
Concrete Description
= specification
Software
= useful product
flow
of
information
design
produce
Software Testing Development
The Antiparallel Structures of Software Testing and Development

11. Never invest in
a business you cannot
understand
“
Warren Buffett

12. Understanding Your Technology Assets
Models
Abstract Model
= theory
Concrete Description
= data
Software
= object of study
flow
of
information
observe
compare
Abstract Model
= design concept
Concrete Description
= specification
Software
= useful product
flow
of
information
design
produce
Software Testing Development
The Antiparallel Structures of Software Testing and Development

13. All Testing is Model Based Testing
A model is a simplified, formal
representation of a relationship,
process or system. The simplification
makes some aspects of the thing
modeled clearer, more visible, and
easier to work with.
All tests are based on models, but
many of those models are implicit.
When the behavior of the program
“feels wrong”, it is clashing with your
internal model of the program and
how it should behave. Black Box Software Testing
Cem Kaner & James Bach

14. Properties of a Model
Herbert Stachowiak. Allgemeine Modelltheorie. Springer-Verlag, Wien and New York, 1973. (Cited by Kühne, Thomas.
"What is a Model?." In Dagstuhl Seminar Proceedings. Schloss Dagstuhl-Leibniz-Zentrum für Informatik, 2005)
mapping feature reduction feature pragmatic feature

15. BA DEV QA
Behavior Driven Development
Given Everything is Broken
When Time is Up
Then Build Back Better

16. The best material
model of a cat is
another, or preferably
the same, cat.
Arturo Rosenblueth
”

17. Parallel Run / Operational Day Replay

18. Bill Gates
The first rule of any technology used in a
business is that automation applied to an
efficient operation will magnify the efficiency.
The second is that automation applied to an
inefficient operation will magnify the
inefficiency.
”

19. Manual vs. Automated
Manual Using Tools Mechanization Automation
A tool is an object used to extend the ability of an individual to modify features of the surrounding environment
Automation is the technology by which a process or procedure is performed with minimal human assistance

20. State of the Art
Mohd-Shafie M.L., Kadir W.M.N.W, Lichter H., Khatibsyarbini M., Isa M.A. Model-based test
case generation and prioritization: a systematic literature review. 2021, Springer.

21. State of the Art
M. Utting, A. Pretschner, B. Legeard, B., “A
taxonomy of model-based testing”,
Special Issue: Model-based Testing Volume 1:
Foundations and Applications of
Model-Based Testing, volume 22, issue 5.
August, 2012.

22. Narrow Definition of Model Based Testing
Can be both mental representation or
a schema or a code-based artifact
Requirement/System
Specifications
MBT Models
Abstract Test Case
Test Results SUT
System
Adapters
Execution
Environment
Executable
Test Case
Mappings
Test Selection
Criteria
1 2
3
4
5
● more formal models (expressed in some machine-
readable, well-defined, notation)
● more formal test generation (we are interested in test
generation algorithms)
● more formal checks for mechanical execution (the
generated tests must be sufficiently precise to
efficiently use humans’ involvement)

23. A great challenge of life:
Knowing enough to think you are
right,
but not knowing enough to
know you are wrong.
Neil deGrasse Tyson
“

24. Humans and Complex Models

25. George E. P. Box
...all models are
approximations. Essentially,
all models are wrong, but
some are useful. However,
the approximate nature of
the model must always be
borne in mind…
”

26. Snake?
Spear? Wall?
Rope?
Tree?
Cut your problems
down to size
Simplified Models
* From Bernard L. Madoff Investment Securities LLC

27. Simplified Models

28. Defense in Depth

29. Microservices Platform
SUT
DW
Diversity

30. MBT Challenges
1. If complex models have to be completed before testing can start, this induces an unacceptable
delay for the proper test executions.
2. For complex SUT, like systems of systems, test models need to abstract from a large amount of
detail, because otherwise the resulting test model would become unmanageable.
3. The required skills for test engineers writing test models are significantly higher than for test
engineers writing sequential test procedures.

31. Scout mindset:
the motivation to see
things as they are, not
as you wish they were.
Julia Galef
”

32. Processes Platforms People
Managed Software Testing

33. 33 Build Software to Test Software
Confidential
exactpro.com
Build Software to Test Software
exactpro.com
FOLLOW US ON SOCIAL MEDIA:
@exactprosystems
TO
Thank You
#cop26