Weitere ähnliche Inhalte Ähnlich wie Quality engineering & testing in DevOps IT delivery with TMAP (20) Mehr von Rik Marselis (20) Kürzlich hochgeladen (20) Quality engineering & testing in DevOps IT delivery with TMAP2. 3
Quality Engineering & Testing in DevOps with the TMAP body of knowledge © 2021 Sogeti. All rights reserved.
Challenges of today’s high-performance IT delivery
The business demands:
• Deliver business value
• Deliver quality at speed
The team challenges are:
• Quality engineering is everyone's responsibility
• QA & testing is integrated in process and people
The focus is:
• Organize high-performing cross-functional teams (you build it, you run it!)
• Automate everything (as long as it is useful)
3. 4
Quality Engineering & Testing in DevOps with the TMAP body of knowledge © 2021 Sogeti. All rights reserved.
The starting point of organizations is to deliver business value
4. 5
Quality Engineering & Testing in DevOps with the TMAP body of knowledge © 2021 Sogeti. All rights reserved.
Supply information
based on
measuring indicators
Our “VOICE model” for delivering business value
Measuring indicators
=
(mainly) testing
Figure 3.1 from the book Quality for DevOps teams
5. 6
Quality Engineering & Testing in DevOps with the TMAP body of knowledge © 2021 Sogeti. All rights reserved.
Examples of indicators in the VOICE model
TMAP defines four groups of indicators, per group you find a few examples below (more in book)
You must find indicators that show whether you are moving towards the pursued business value.
Business value related indicators
- Customer satisfaction
- Number of service calls to helpdesk
IT delivery related indicators
- Business features done
- Percentage of availability (up-time)
Team related indicators
- Availability of necessary skills
- Satisfaction and happiness of team members
Problem related indicators
- Number of anomalies registered compared to expected
- Mean time to fix operational failures
And keep in mind:
a few well-measured and properly
followed-up indicators are much
better than a long list of unpractical
indicators.
Figure 5.4 from the book Quality for DevOps teams
6. 7
Quality Engineering & Testing in DevOps with the TMAP body of knowledge © 2021 Sogeti. All rights reserved.
Waterfall V-model Scrum DevOps Demand-Supply SAFe®
Figure 7.1 from the book Quality for DevOps teams
7. 8
Quality Engineering & Testing in DevOps with the TMAP body of knowledge © 2021 Sogeti. All rights reserved.
DevOps – highlights
The six DevOps principles:
1. Customer-centric action
2. Create with the end in mind
3. End-to-end responsibility
4. Cross-functional autonomous teams
5. Continuous improvement
6. Automate everything you can
DevOps is a cross-functional systems engineering culture that aims at unifying
systems development (Dev) and systems operations (Ops) with the ability to create and
deliver fast, cheap, flexible and with adequate quality, whereby the team as a whole is
responsible for the quality. Other areas of expertise, such as business analysis and quality
assurance (including testing) are usually integrated in the team. A DevOps culture has an
Agile mindset that can be supported/implemented by e.g. the Scrum framework.
The six fundamental DevOps activities
8. TEAM:
Together
Everyone
Achieves
More
Example of a cross-functional
high-performance IT delivery team
The starting point for DevOps:
A cross-functional team
Aligned with the DevOps culture of IT delivery – where roles are important and functions
are not – keep in mind that if in TMAP you see a term such as developer or tester, we
refer to the role of a team member at a specific moment in time, not to a function.
9. Quality Engineering & Testing in DevOps with the TMAP body of knowledge © 2021 Sogeti. All rights reserved.
Quality Engineering is about team members and their stakeholders
taking joint responsibility to continuously deliver IT systems
with the right quality at the right moment
to the businesspeople and their customers.
It is a principle of software engineering concerned with
applying quality measures to assure built-in quality.
10. 11
Quality Engineering & Testing in DevOps with the TMAP body of knowledge © 2021 Sogeti. All rights reserved.
Quality Engineering & Testing support delivering business value
Quality Engineering
Testing
Focus:
“Detective”
quality measures
Focus:
“Preventive”
quality measures
Build quality in,
right first time
Provide information
about quality & risks
11. 12
Quality Engineering & Testing in DevOps with the TMAP body of knowledge © 2021 Sogeti. All rights reserved.
Definition of testing within quality engineering & business delivery
Testing consists of verification, validation and exploration activities
that provide information about the quality and the related risks,
to establish the level of confidence
that a test object will be able to deliver the pursued business value.
Figure 5.6 from the book Quality for DevOps teams
12. 13
Quality Engineering & Testing in DevOps with the TMAP body of knowledge © 2021 Sogeti. All rights reserved.
Approaches for test design
Figure 43.1 from the book Quality for DevOps teams
Focus on exploration
Focus on
verification & validation
13. 14
Quality Engineering & Testing in DevOps with the TMAP body of knowledge © 2021 Sogeti. All rights reserved.
Poll for the audience: which subjects would you like us to discuss?
Photo of the book launch event on 17 March 2020
From left to right: Rik Marselis, Dennis Geurts, Berend van Veenendaal and Wouter Ruigrok
This photo was edited to include Dennis who couldn’t be present at the actual launch
Go to: Menti.com
14. 15
Quality Engineering & Testing in DevOps with the TMAP body of knowledge © 2021 Sogeti. All rights reserved.
Select from the following subjects
QA & Testing topics and Test Design
Continuous everything: Monitoring & Control and Reporting & Alerting
CI/CD pipeline and Test automation
Unit testing and mutation testing
Closing
16. 17
Quality Engineering & Testing in DevOps with the TMAP body of knowledge © 2021 Sogeti. All rights reserved.
QA & testing topics
Figures 11.1 and 11.2 from the book Quality for DevOps teams
17. 18
Quality Engineering & Testing in DevOps with the TMAP body of knowledge © 2021 Sogeti. All rights reserved.
The QA & testing topics related to the DevOps activities
Organizing
topics
Performing
topics
DevOps
activities
Figure 1.4 from the book Quality for DevOps teams
18. 19
Quality Engineering & Testing in DevOps with the TMAP body of knowledge © 2021 Sogeti. All rights reserved.
Coverage-based testing with test design techniques
A test design technique is a standardized method of deriving test cases from a specific test basis
that will achieve a certain coverage.
Applying a test design technique results in test situations, logical test cases & physical test cases.
Our investigation has shown that there are about 25 to 30 test design techniques in literature.
All test design techniques can be categorized in the four coverage groups mentioned below.
Keep in mind that you don’t have to
know all existing test design
techniques.
Knowing a few techniques for each
coverage group makes that with just
about 8 techniques you can handle
any testing challenge.
Figure 45.1 from the book Quality for DevOps teams
19. 20
Quality Engineering & Testing in DevOps with the TMAP body of knowledge © 2021 Sogeti. All rights reserved.
16 test design techniques defined in TMAP
https://tmap.net/page/test-design and https://tmap.net/building-blocks/coverage
Figures 45.2, 45.3, 45.4 and 45.5 from the book Quality for DevOps teams
21. 22
Quality Engineering & Testing in DevOps with the TMAP body of knowledge © 2021 Sogeti. All rights reserved.
Continuous everything to achieve built-in quality
Figure 6.2 from the book Quality for DevOps teams
22. 23
Quality Engineering & Testing in DevOps with the TMAP body of knowledge © 2021 Sogeti. All rights reserved.
Continuous improvement is about Product, Process and People
Product
People
Process
Figure 25.1 from the book Quality for DevOps teams
23. 24
Quality Engineering & Testing in DevOps with the TMAP body of knowledge © 2021 Sogeti. All rights reserved.
Monitoring, control, reporting & alerting
Figures 19.9 and 19.10 from the book Quality for DevOps teams
25. 26
Quality Engineering & Testing in DevOps with the TMAP body of knowledge © 2021 Sogeti. All rights reserved.
Automate everything (as long as it is useful)
An automated pipeline enables: elimination of errors due to manual task execution, provisioning
of standardized development feedback loops and enabling of fast product iterations.
Figure 6.1 from the book Quality for DevOps teams
26. 27
Quality Engineering & Testing in DevOps with the TMAP body of knowledge © 2021 Sogeti. All rights reserved.
Capabilities & tools
When talking about a CI/CD pipeline people often show all
the tools in the pipeline.
But tools are exchangeable and often change.
To set up a good pipeline it’s very important to define the
capabilities your organization needs from the pipeline.
Over time the capabilities needed will evolve, at the start
the pipeline will only cover a few capabilities, when maturity
grows the capabilities that are automated also grow.
Examples of capabilities are (more examples are described in section 6.3 of the book):
• Continuous Integration code review
• Continuous Delivery configuration management
• Continuous Deployment deployment of the build application (container, package)
• Continuous Monitoring user telemetry to measure user sentiment and behavior
• Continuous Quality & Testing automated anomaly detection
Examples of tools in a pipeline
Figure 6.3 from the book Quality for DevOps teams
27. 28
Quality Engineering & Testing in DevOps with the TMAP body of knowledge © 2021 Sogeti. All rights reserved.
Organize your automated test varieties:
Testing pyramid and testing quadrants
Figures 32.2 and 32.3 from the book Quality for DevOps teams
29. 30
Quality Engineering & Testing in DevOps with the TMAP body of knowledge © 2021 Sogeti. All rights reserved.
Unit testing
Keep in mind that the statement “we have reached
100% code coverage” in itself doesn’t give useful
information.
The type of code coverage achieved is what matters
Code coverage demonstrates the percentage of program code that is
covered by tests. Different test design techniques guarantee less or
more coverage.
The spheres of testing
Figures 37.2 and 46.14 from the book Quality for DevOps teams
30. 31
Quality Engineering & Testing in DevOps with the TMAP body of knowledge © 2021 Sogeti. All rights reserved.
Mutation testing tests the test
Mutation testing is a type of testing where certain statements in the source code are changed
(mutated) to check if test cases will identify the fault that was introduced this way.
This is a manner to verify the quality of the test set (instead of the object under test).
Mutation testing usually is supported by tools.
Mutation testing focuses on conditional statements.
Suppose we have the following code and test cases:
IF A > 10 PRINT “YES” ENDIF
Testcase1: A := 10, expected result no print
Testcase2: A := 11, expected result “YES”
What mutation in this code would not make any of these tests fail?
IF A <> 10 PRINT “YES” ENDIF Testcase3: A := 9, expected result no print
IF A = 11 PRINT “YES” ENDIF Testcase4: A := 12, expected result “YES”
32. 33
Quality Engineering & Testing in DevOps with the TMAP body of knowledge © 2021 Sogeti. All rights reserved.
We proudly present: Quality for DevOps teams
and the renewed body of knowledge www.tmap.net
33. 34
Quality Engineering & Testing in DevOps with the TMAP body of knowledge © 2021 Sogeti. All rights reserved.
TMAP training courses and certification scheme of
in partnership with
TMAP: Quality
for cross-
functional teams
TMAP: High-
performance
quality
engineering
TMAP:
Organizing built-
in quality
at scale
Performing
people
3-day training course
exam of 30 questions
3-day training courses
exams of 40 questions
All people
involved in
cross-
functional
teams
Organizing
people
Exams provided by
Info about TMAP certification at:
www.TMAPcert.com
34. 35
Quality Engineering & Testing in DevOps with the TMAP body of knowledge © 2021 Sogeti. All rights reserved.
TMAP:
body of knowledge for
quality engineering in
all IT delivery models
www.ICT-books.com
35. 37
Quality Engineering & Testing in DevOps with the TMAP body of knowledge © 2021 Sogeti. All rights reserved.
Stay up-to-date with the latest TMAP developments
Follow TMAP on LinkedIn:
www.linkedin.com/company/2020-tmap-quality-engineering-for-high-performance-it-delivery/
(use the link or just search for TMAP and the
world of quality engineering on LinkedIn)
36. 38
Quality Engineering & Testing in DevOps with the TMAP body of knowledge © 2021 Sogeti. All rights reserved.
About Sogeti
Learn more about us at
www.sogeti.com This message contains information that may be privileged or
confidential and is the property of the Capgemini Group.
Copyright© 2020 Sogeti. All rights reserved.
Sogeti is a leading provider of technology and engineering services. Sogeti delivers
solutions that enable digital transformation and offers cutting-edge expertise in Cloud,
Cybersecurity, Digital Manufacturing, Digital Assurance & Testing, and emerging
technologies. Sogeti combines agility and speed of implementation with strong technology
supplier partnerships, world class methodologies and its global delivery model,
Rightshore®. Sogeti brings together more than 25,000 professionals in 15 countries,
based in over 100 locations in Europe, USA and India. Sogeti is a wholly-owned subsidiary
of Capgemini SE, listed on the Paris Stock Exchange.
TMAP website:
www.tmap.net
Sogeti academy:
academy.sogeti.nl
TMAP:
body of knowledge for
quality engineering
Feel free to contact me on:
Rik.Marselis@Sogeti.com
37. 39
Quality Engineering & Testing in DevOps with the TMAP body of knowledge © 2021 Sogeti. All rights reserved.
TMAP: body of knowledge for quality engineering