Robotesting, are you ready for that yet? About testing OF robots and testing WITH robots. Presentation at 7th World Congress for Software Quality in Lima, Peru. March 2017.

1. Robotesting,
are you ready
for that yet?
7th World Congress
for Software Quality
Rik Marselis

2. Robotesting:
Testing of robots &
Testing with robots
7th World Congress
for Software QualityRik Marselis, 21 March 2017, Lima - Peru.

3. 1980
Rik Marselis@rikmarselis
2007 2012 20122008
Workgroup
-member
20142009

4. |
Who has a robot?
© Sogeti 2017 4
Who has a robot? In 10 years all of you will !!

5. |Source: octaneai.com 5
Chatbots are robots too (at least in my definition ;-)

6. |
Towards autonomous robots
© Sogeti 2017 6

7. |
Traditional robots
Low intelligence  for specific tasks
(“dull and/or dangerous”) © Sogeti 2017 7

8. |
Robots in practice
Over 50% of the
dairy-farmers in the
Netherlands uses
one or more robots.
© Sogeti 2017 8

9. |
Manure robot
© Sogeti 2017 9
Low intelligence  for specific tasks
(“dull and/or dangerous”)

10. |
Milking robot
© Sogeti 2017 10

11. |
Social impact of robots
Before the farmer had to milk the cows 6:00 – 8:30 and 17:00 – 19:30
Now the farmer and his family can have dinner together at 18:00 !!
© Sogeti 2017 11
De aardappeleters. Vincent van Gogh

12. |
Robots in healthcare
© Sogeti 2017 12
Is this OK?
Or should we
have people
taking care?
An ethical
question…

13. Testing with robots
Robotesting…
7th World Congress
for Software Quality

14. |
Robot automatically performs regression tests
© Sogeti 2017 14

15. |
Robot automatically performs regression tests
© Sogeti 2017 15

16. |
Use robots for…
© Sogeti 2017 16
Functional testing
• Generate test cases
• Execute test cases
• Analyse the results
Brute-force testing
• Generate a huge number of test cases and
execute them
Non-functional testing
• Intelligent performance testen
(not just 10.000 times the same transaction)

17. |
Use robots for…
© Sogeti 2017 17
Big data analytics (using artificial intelligence)
• Analyse live data
• Generate relevant test data
Colaboration between human and machine
• Robot  repetative work
• Human creative work
Regression testing
• Robots are very good at repetative work!!
And so much more!!!!!

18. Robotesting…
Testing of robots
7th World Congress
for Software Quality

19. |
Different angles of quality for robotics
© Sogeti 2017 19
Mechanical
Electrical
Information
Processing
Machine
Intelligence
Business
Impact
Social
Impact
For these “traditional” tests we already
have the knowledge and experience
for decades.

20. |
Artificial intelligence… (from the film “Ex Machina” 2015)
© Sogeti 2017 20The rise of artificial intelligence
has long been inevitable.
The question is
when, not if…

21. |
Different angles of quality for robotics
© Sogeti 2017 21
Mechanical
Electrical
Information
Processing
Machine
Intelligence
Business
Impact
Social
Impact
new
new
new

22. |
Testing still has the well-know foundation
© Sogeti 2017 22
Quality
Risks
Confidence
Quality attributes
Analysis and mitigation

23. |
Machine learning (in the film “Short circuit” 1986)
© Sogeti 2017 23
<<< vervangen door filmclip >>>

24. |
Testing machine learning: it’s about the input!!
© Sogeti 2017 24
The tester as “psychologist” of the intelligent machine

25. |
Different angles of quality for robotics
© Sogeti 2017 25
Mechanical
Electrical
Information
Processing
Machine
Intelligence
Business
Impact
Social
Impact
Controlling the input

26. |
No more programming but training …
© Sogeti 2017 26
The tester as “trainer” of the intelligent machine

27. |
Different angles of quality for robotics
© Sogeti 2017 27
Mechanical
Electrical
Information
Processing
Machine
Intelligence
Business
Impact
Social
Impact
Safety
Functionality (ISO25010)
Engagement

28. |
Testing still has the well-know foundation
© Sogeti 2017 28
Quality
Risks
Confidence
Quality characteristics
Analysis and mitigation

29. |
New risks having impact on business process
© Sogeti 2017 29
Product Risk Analysis
New risks…
Well-known approach…
with other outcomes for
• Test strategy
• Test intensity
• Test coverage
• Test techniques

30. |
Image remains the highest impact-area…
© Sogeti 2017 30

31. |
When will self-driving cars be on the road?
In how many years do you expect a self-driving vehicle will be
allowed on the road in the Netherlands
Think of a number…
Who thinks in … years?
© Sogeti 2017 31
10
5
2
1

32. |
Sogeti has already tested a self-driving vehicle
The Wepod, an
autonomous bus
in Wageningen.
On the public
road since
October 2016 !!
(and tested by my
colleagues of Sogeti)
© Sogeti 2017 32

33. |
Non-functionals become more important
TMap NEXT has 17 quality attributes
ISO9126 has 27 quality attributes
ISO25010 has 37 quality attributes
For “the new world” we have now made a list of
over 70 possible quaility attributes.
(work in progress)
© Sogeti 2017 33
ethics
embodiment
empathy

34. |
Augmented people
© Sogeti 2017 34

35. |
Will robots take over your job?
Source: www.pewinternet.org/2016/03/10/public -predictions-for-the-future-of-workforce-automation/ © Sogeti 2017 35
The tester as philosopher for applying new technology…

36. |
Different angles of quality for robotics
© Sogeti 2017 36
Mechanical
Electrical
Information
Processing
Machine
Intelligence
Business
Impact
Social
Impact
Ethics, Empathy and Embodiment

37. |
Different angles of quality for robotics
This is R&D work in progress ...
© Sogeti 2017 37
Mechanical
Electrical
Information
Processing
Machine
Intelligence
Business
Impact
Social
Impact
Safety
Ethics, Empathy and Embodiment
Maintainability (ISO25010)
Compatibility (ISO25010)
Portability (ISO25010)
Security (ISO25010)
Operability (ISO25010)
Performance (ISO25010)
Reliability (ISO25010)
Functionality (ISO25010)
Engagement
Personality
Human friendliness

38. Robots testing…
Existing foundations;
New possibilities and
additions!
Make sure you’re ready for it!
7th World Congress
for Software Quality