This document discusses continuous testing in the context of DevOps. It defines continuous testing as how software is tested and outlines a three step process for increasing quality while also increasing speed of development and delivery. The first step is to understand what causes quality to decrease with increased speed. The second step is to find or develop solutions to identified root problems like lack of test coverage or quality holes. The third step is to integrate the solutions into continuous integration/continuous delivery workflows. It provides an example case study of one company's technical stack and CI/CD workflow that incorporates continuous testing.
2. Eran Sher
Co-founder & CEO of SeaLights
Nolio (Acquired by CA Technologies)
Mercury Interactive (Acquired by HP)
Conduct (Acquired by Mercury)
3. The State of QA & DevOps
“managing and implementing quality and test measures represent a challenge for
teams trying to increase productivity (or release velocity) with acceptable quality”
WORLD QUALITY REPORT 2016-2017
Increasing
Release speed
Decreasing
Release quality
Continuous Delivery is becoming the de-facto standard
6. What QA Means for DevOps
Continuous Integration
How we build software
Continuous Deployment
How we deploy software
Continuous Testing
How we test software
Continuous Delivery
17. Case Study
Our Technical Stack:
1. Node.js, Java, and Python
2. Microservices architecture, Mongo DB
3. CI: Jenkins
4. SCM: Github
5. Test automation frameworks: Mocha, TestNG, JUnit, and Py.test
6. Infrastructure: AWS
7. CD: homegrown
18. Case Study
Our CI/CD Workflow
1. Pull request
2. Build + unit test
3. Verify that all tests passed, no decrease in coverage, and no quality holes
4. Integration build
5. Setup and deploy build to integration lab (AWS)
6. Run integration tests
7. Verify that all tests passed, no decrease in coverage, and no quality holes
8. Ready for production » Deploy