Hear from Verizon Media’s Venu Narayanabhatla (Director, Software Development Engineering) and Sauce Labs’ Alissa Lydon (Continuous Testing SME/Product Marketing Manager) as they discuss: - Why Verizon Media decided to standardize their testing practices across a number of distributed teams, and how they implemented that change - The development of Screwdriver, now an open source CI tool, and how it solved internal challenges and enabled teams to embrace continuous integration - Their decision to integrate Screwdriver with Sauce Labs, and how it allowed for scaling to 2.7 million automated tests run per month against all of their applications

1. How Verizon Media Embraced
Open Source to Accelerate
Release Velocity
Venu Narayanabhatla
Director of Software Development Engineering, Verizon Media

2. 2
Verizon Media is home to media, tech, and
communication products that more than a billion
people love and trust.
With leading brands in its portfolio including AOL,
HuffPost, Yahoo Finance, Yahoo Sports, Yahoo
Mail, TechCrunch, and others.

3. Verizon confidential and proprietary. Unauthorized disclosure, reproduction or other use prohibited.
The Challenge?
CICD & Testing @ Scale
3

4. 4
Our engineers develop and manage thousands of web, mobile,
and server apps that comprise tens of thousands of projects.
In 2012, teams were creating code for each of these projects and
needed continuous integration and testing to ensure the code
performed up to standard and reached users/customers at a
rapid pace.

5. Most of the teams were
relying on adhoc release
processes and a
combination of manual &
automated testing.
5

6. We needed a continuous
integration & testing solution that
would scale to our volume.
6

7. Verizon confidential and proprietary. Unauthorized disclosure, reproduction or other use prohibited.
The Solution?
7

8. 8
● Committed to Continuous Integration &
Continuous Delivery
● Integrated CICD into product development
● Invested in automation
● Explored a need for centralized build
infrastructure & tooling

9. 9
● Built and operated Screwdriver at scale within
Verizon Media
● Open sourced Screwdriver and encouraged
external contributions
● Moved release cycles from months to at-will
How?

10. 10
Open source build platform for ContinuousDelivery

11. 11
Secure Continuous Delivery Integrated with Daily Habits
Pipeline as Code Runs Anywhere

12. 12
Screwdriver Stats
Usage
45k+
Projects
50k+
Daily builds
Platforms
67
Platforms
10+
Programming
Languages
Footprint
470+
Hosts
5
Clusters
History
4
Versions
2012
Product was introduced

13. 13
“Making Screwdriver open source has allowed us to augment
our internal team with talented developers all over the world.
Screwdriver is open source under a permissive Apache
license, so any organization that operates at scale and needs
to test its code can also benefit from a free, open source build
platform.”
- Gil Yehuda
Sr. Director of Open Source, Verizon Media

14. 14
Learn more @
screwdriver.cd

15. 15
Deep Integration of Continuous Testing
into…Continuous Integration

16. 16
Continuous Delivery Pipeline

17. Verizon confidential and proprietary. Unauthorized disclosure, reproduction or other use prohibited.
Test Philosophy
17

18. 18

19. 19

20. 20
+

21. 21
● Invested in automated test suites at scale
● Integrated Sauce Labs’ Continuous Testing
Cloud with Screwdriver, our open source
CICD platform
How?

22. 22
○ Configured tunnels from build infrastructure
○ User can configure and launch automated
tests to the Sauce Labs Cloud from CICD
pipelines
○ Automated quality gates in the CICD pipeline
How we’re using Sauce Labs...

23. 23
Screwdriver + Sauce Labs Integration

24. 24
Continuous Testing @ Scale

25. 25
Where we’re seeing value from Sauce Labs...
○ Debug and analyze test results
○ Continuous testing at various stages of pipeline such
as UI and end-to-end
○ Sauce Labs Cloud scales to the continuous testing
load from the multitenant CICD system

26. 26
● Sharing Best Practices
○ Communities within Verizon Media:
■ Frontend Automation Community
■ Sauce Labs Community
● Review results with our Sauce Labs support team
● Training
We enjoy collaborating with Sauce Labs...

27. Verizon confidential and proprietary. Unauthorized disclosure, reproduction or other use prohibited.
Results
27

28. 28
After implementing Screwdriver and Sauce
Labs, we ran more than 2.7 million tests in one
month—proving that the solution could scale.

29. 29
By implementing Screwdriver and Sauce Labs
for testing, we’ve saved a total of 4,500 hours
per year in staffing and maintenance.

30. Q&A
30
venugopal@verizonmedia.com