The document discusses continuous quality in DevOps. It notes that development is quick but testing lags, creating a conundrum. Continuous quality is described as integrating all testing activities continuously with development and delivery. This allows for immediate bug fixing, provisioning of test environments, and automated unit, functional and non-functional testing orchestrated by continuous integration and delivery tools. The key is to assess business risk coverage and establish a safety net to protect users. Continuous quality should be seamlessly integrated into the software delivery pipeline and provide actionable feedback at each stage.

1. Continuous Quality
DevOps with Quality
Friday, March 1, 2019

2. TAPFIN | Proprietary & Confidential 4
4Experis Solutions | Friday, March 1, 2019
About the Speaker
Joshua Eastman
Competency Owner, Testing &
Continuous Quality
joshua.eastman@experis.com
www.experis.com

3. TAPFIN | Proprietary & Confidential 5
5Experis Solutions | Friday, March 1, 2019
The Conundrum
DEV IS QUICK
TESTING LAGS
Is my QUALITY PROCESS -
Coordinated, end-to-end?

4. TAPFIN | Proprietary & Confidential 6
6Experis Solutions | Friday, March 1, 2019
DevOps Requires Continuous Quality
Plan Code Build Test Release Deploy Operate Monitor
Agile Development
Continuous Integration
Continuous Delivery
Continuous Deployment
DevOps

5. TAPFIN | Proprietary & Confidential 7
7Experis Solutions | Friday, March 1, 2019
What is Continuous Quality
A core capability in Agile and DevOps where all testing activities run continuously in
an integrated fashion with development and delivery. Immediate bug fixing is
enabled, test environments are provisioned instantly, and unit, functional, and
nonfunctional tests are run in an automated way, orchestrated by continuous
integration and continuous delivery tools.
We have to integrate “Continuous Quality” into your existing
DevOps delivery pipelines to achieve:
to refine the development process so that business expectations
are continuously met.

6. TAPFIN | Proprietary & Confidential 8
8Experis Solutions | Friday, March 1, 2019
Key Points to Continuous Quality
Primary goal is
assessing
business risk
coverage
Establishes a
safety net that
helps the team
protect the
user
Always on,
stable test
environment
Seamlessly
integrated into
the software
delivery
pipeline and
DevOps
toolchain
Delivers
actionable
feedback
appropriate for
each stage of
the delivery
pipeline

7. What it Looks Like

8. TAPFIN | Proprietary & Confidential 10
10Experis Solutions | Friday, March 1, 2019
Testing @ Every Stage in the Delivery Pipeline
DEV TEST BETA PROD
Design
Manual
• Design Review
• Performance
• Security
Semi-Automated
• Code Review
(pre and post commit)
Automated
• Static Code Analysis
• Unit Testing
• Performance
• Security
Implementation
Automated
• Sanity/Smoke Tests
• Integration/API Tests
• UI Tests
• Regression Tests
• Performance
• Security
• E2E Tests
Manual
• Exploratory Testing
• UI/Ad-hoc Testing
Testing
Automated
• Sanity/Smoke Tests
• E2E Tests
• Subset of:
• Integration/API
Tests
• UI Tests
• Regression Tests
• Performance
• Security
Manual
• Exploratory Testing
• UI/Ad-hoc Testing
• UAT
Acceptance Testing
Automated
• Sanity/Smoke Tests
Manual
• Accessibility check
Production Testing

9. TAPFIN | Proprietary & Confidential 11
11Experis Solutions | Friday, March 1, 2019
Plan

10. TAPFIN | Proprietary & Confidential 12
12Experis Solutions | Friday, March 1, 2019

11. TAPFIN | Proprietary & Confidential 13
13Experis Solutions | Friday, March 1, 2019

12. TAPFIN | Proprietary & Confidential 14
14Experis Solutions | Friday, March 1, 2019

13. TAPFIN | Proprietary & Confidential 15
15Experis Solutions | Friday, March 1, 2019
Monitor

14. How Do We Get There

15. TAPFIN | Proprietary & Confidential 17
17Experis Solutions | Friday, March 1, 2019
DevOps Laggards vs. DevOps Leaders
DEVOPS
LEADERS
DEVOPS
LAGGARDS
Follow 4 core
Continuous Quality
practices
Are twice as likely
to consider
automating the
software quality
process a “critical
business
differentiator”
Focus on
contextual metrics
(i.e. requirements
coverage)
Measure the user
experience across
an end-to-end
transaction
Rely on manual
testing
Are twice as likely
to report that
testing is a
bottleneck
Focus on counting
metrics (i.e.
number of tests)
Measure
application-specific
or team-specific
metrics

16. TAPFIN | Proprietary & Confidential 18
18Experis Solutions | Friday, March 1, 2019
4 Core Continuous Quality Practices
1. Make a Cultural Commitment to Collaboration and Quality1
1. Apply a Broad Range of Tests, Anchored by Automation2
1. Leverage Cloud for Breadth and Scalability3
1. Develop a Strategic Approach to Analytics4

17. TAPFIN | Proprietary & Confidential 19
19Experis Solutions | Friday, March 1, 2019
Shift Left - Test Automation Strategy

18. TAPFIN | Proprietary & Confidential 20
20Experis Solutions | Friday, March 1, 2019
4 Core Continuous Quality Practices
1. Make a Cultural Commitment to Collaboration and Quality1
1. Apply a Broad Range of Tests, Anchored by Automation2
1. Leverage Cloud for Breadth and Scalability3
1. Develop a Strategic Approach to Analytics4

19. TAPFIN | Proprietary & Confidential 21
21Experis Solutions | Friday, March 1, 2019
4 Core Continuous Quality Practices
1. Make a Cultural Commitment to Collaboration and Quality1
1. Apply a Broad Range of Tests, Anchored by Automation2
1. Leverage Cloud for Breadth and Scalability3
1. Develop a Strategic Approach to Analytics4

20. TAPFIN | Proprietary & Confidential 22
22Experis Solutions | Friday, March 1, 2019
4 Core Continuous Quality Practices
1. Make a Cultural Commitment to Collaboration and Quality1
1. Apply a Broad Range of Tests, Anchored by Automation2
1. Leverage Cloud for Breadth and Scalability3
1. Develop a Strategic Approach to Analytics4

21. Questions?