Continuous Testing is the crucial ingredient to rapidly deliver quality apps from your GitLab CI/CD pipelines. Rather than weekly or monthly release cycles, you need scalable quality patterns that deliver rapid daily or hourly releases. You need to thoroughly test every commit. We’ll discuss building powerful continuous testing workflows with GitLab CI/CD pipelines and GitLab’s mabl integration to deliver testing insights on every push, directly into the relevant developer commit or merge request. We’ll review common UI testing patterns you can use from simple page tests to complex end to end application test suites, all running and scaling automatically within your GitLab CI/CD pipelines.

1. #GitLabCommit
Ubiquitous Quality Through Continuous Testing

2. #GitLabCommit
Ubiquitous Quality Through Continuous Testing
Joe Lust @ Mabl Engineering

3. #GitLabCommit#GitLabCommit
Outline
● Why do you need Continuous Testing
● How you can deliver Continuous Testing
on GitLab CI / CD
● Real world examples
@lustcoder
● Building the web for twenty years
● Cloud Infra Lead @ mabl
● Blogger, meetup organizer, speaker
About me
Joe Lust

4. #GitLabCommit
Continuous Testing

5. #GitLabCommit
What is Continuous Testing?
Testing at all levels of your code lifecycle
● At the commit
● In the merge request
● In the CI / CD pipeline
● In the blue / green environments
● In production

6. #GitLabCommit#GitLabCommit
Testing
Pre-DevOps
● Infrequent testing
○ Weekly / Monthly / Quarterly
● Manual testing
● Limited coverage
● Sparse / static environments
● Reams of log output
Testing
Post-DevOps
● Continuous testing
○ Test every commit
● Highly automated
● End to end coverage
● Unlimited environments
● Targeted insights

7. #GitLabCommit
Shifting quality left: Go to the source
The further left you shift application testing, the greater the time / money saved
● Catch the bugs before they merge, for greatest savings
● Reduce developer firefighting
● Let QA focus on the meaty testing problems
● Shift as far left as possible
Push Bug
Commit
Author
Bug
Merge
Bug
Bug to QA Bug to
Prod
Business
Impact

8. #GitLabCommit
Implementing
Continuous Testing in GitLab

9. #GitLabCommit
For Continuous Testing, We’ll Need to Scale
● Scalable Testing Environment Solution
○ e.g. GitLab Review Apps
● Scalable Test Running Solution
○ SaaS: e.g. mabl
○ Self Hosted: e.g. Puppeteer on Kubernetes

10. #GitLabCommit
What’s mabl?
Mabl is an intelligent test automation platform
● Easily create reliable automated UI tests that are just as easy to maintain.
● Unified platform integrations with bug tracking, source control, and CI /CD
● Comprehensive test results help you gain actionable insights across the
entire development lifecycle
● Create customer-centric end-to-end tests from your end users perspective

11. #GitLabCommit
What are Preview Environments?
● On demand, short live environments
● Used to validate changes and deployments
● No / minimal impact if broken
○ Single use
● Also known as:
○ Review App / Ephemeral Environments

12. #GitLabCommit
Slide option 1 of 2
This seems like a
completely innocuous
MR, especially if you
don’t speak CSS.

13. #GitLabCommit
Slide option 1 of 2
But what if we could
SEE the changes in a
Preview Environment?

14. #GitLabCommit
Slide option 1 of 2
Oh, no! This certainly
isn’t what I meant to
approve!

15. #GitLabCommit#GitLabCommit
Challenges to pervasive
Preview Environments
● Server cost
● Networking complexity
● Domain routing
● Cost $$$

16. #GitLabCommit#GitLabCommit
Challenges to pervasive
Preview Environments
● Server cost
● Networking complexity
● DNS setup
● Cost $$$
Serverless
Solution
● No servers
● Networking outsourced
● Wildcard domains
● “Scale to zero” low cost

17. #GitLabCommit
Making a Serverless Preview Environment
Let’s look at the Single Page Application case (SPA)
● Host the static contents in a bucket
● Host the dynamic parts on a function
● High scalability for low cost

18. #GitLabCommit
e.g. Preview Environments on Google Cloud
Real world example at mabl:
● Push HTML/JS to a bucket with commit hash in path
○ Place bucket behind vanity domain / CDN
● Place dynamic bits in container on Cloud Run
● Host 1,000 preview environments for $1
preview.acme.com/petstore/a32df28
app name commit hash
Read more detailed
examples with
source code here.

19. #GitLabCommit
Harness Your Preview Environments
Make the preview environment work for you
● Run your end to end automation against the preview
● Link your merge requests to the preview
○ Show, don’t tell reviewers what you changed
● Test merges to main before you merge
○ Keep main green!

20. #GitLabCommit
Real World Examples

21. #GitLabCommit
Testing on GitLab Pipelines
Make the preview environment work for you
● Use Review Apps
○ Just add hosting
● Use Dynamic Environments
○ Pass environment URL to your testing tools

22. #GitLabCommit
Testing on GitLab Pipelines: In YAML - Deploy

23. #GitLabCommit
Mabl GitLab Pipelines Integration
Easily run intelligent tests in the mabl cloud in just a few lines of YAML
● Unlimited parallel test execution for quick commit feedback
● Test status updates and results in your merge request
● Rich test results for rapid root cause analysis

24. #GitLabCommit
Mabl GitLab Pipelines Integration
Easily run intelligent tests in the mabl cloud in just a few lines of YAML
● Unlimited parallel test execution for quick commit feedback
● Test status updates and results in your merge request
● Rich test results for rapid root cause analysis
Easily track the status of
your hundreds of tests
and thousands of test
runs.

25. #GitLabCommit
Mabl GitLab Pipelines Integration
Easily run intelligent tests in the mabl cloud in just a few lines of YAML
● Unlimited parallel test execution for quick commit feedback
● Test status updates and results in your merge request
● Rich test results for rapid root cause analysis
Dive deep into fine
grained test run details
for easy diagnostics

26. #GitLabCommit
Testing on GitLab Pipelines: In YAML - Test

27. #GitLabCommit
Mabl CLI: For the DevOps Experts
DevOps experts can use the mabl CLI
● Script complex test suites with bash scripts
● Don’t leave your developer workflow
● View real time test completion and results

28. #GitLabCommit
Conclusions

29. #GitLabCommit
Key Takeaways
● DevOps ia an enabling group for Continuous Testing
● Serverless enables cost effective, scalable
○ Testing environments
○ Testing tools
● Shift your testing as far left as possible

30. #GitLabCommit
References
Serverless
● Serverless Preview Environments on AWS and GCP
● Presentation Slide Deck - bit.ly/2DyiEfU
Testing w/ mabl
● Mabl GitLab Integration - www.mabl.com/integrations/gitlab
● Try mabl for free - mabl.com/trial-registration
@lustcoder@mablhq

31. #GitLabCommit
Thank You