Diese Präsentation wurde erfolgreich gemeldet.
Wir verwenden Ihre LinkedIn Profilangaben und Informationen zu Ihren Aktivitäten, um Anzeigen zu personalisieren und Ihnen relevantere Inhalte anzuzeigen. Sie können Ihre Anzeigeneinstellungen jederzeit ändern.
Efficient Parallel Testing with Docker
Laura
Frank
Senior Engineer, Codeship
1. Parallel Testing: Goals and Benefits
2. DIY with LXC
3. Using Docker and the Docker Ecosystem
Agenda
Parallel Testing
Create a customizable, flexible
test environment that enables us
to run tests in parallel
GOAL
Spend less time waiting around for your automated
testing steps and deployments to finish.
• Ship newest code to productio...
Developers should have full autonomy over testing
environments, and the way tests are executed.
• Move testing commands to...
Let’s eliminate friction in the development cycle
• For local testing, e.g. unit and integration tests run
by a development team
• On internal CI/CD systems
• As part of a ...
• Split up testing tasks
• Give developer full control of dependencies
per task
• Use containers to run multiple tests at ...
Run tasks across multiple
processors in parallel
computing environments
TASK PARALLELISM
Distributed Task Parallelism
A distributed system of containerized computing
environments takes the place of a single
mult...
Why not VMs?
• Isolation of running builds on infrastructure
• Challenges with dependency management
• No clean interface ...
Containers, duh!
• Impose resource limits and increase
virtualization density
• Run customer code in isolation
• Provide c...
DIY with LXC
Codeship has been powered by
containers since the very beginning
Codeship was founded
Flowing salty water on Mars
International Year of Forests
Preparations ahead of 12.04 Precise release...
I should use LXC…
Checkbot (Codeship Classic)
• Still running in production as our classic infrastructure
• Well-suited for users who want 1...
Checkbot
39K builds per day
7.8M builds per year
Architecture
• Universal container with provided dependencies
• Fixed amount of available containers per VM
• Implement pa...
User Commands
Universal Container
Pipeline
User Commands
Universal Container
Pipeline
Heroku
Deployment Provider
Capistran...
Limitations
• Parity between dev and test
• Can’t really debug locally
• No useable interface between user and container
•...
We weren’t able to provide the
best, most efficient product to
our customers (or ourselves)
Using Docker and the
Docker Ecosystem
Create a customizable, flexible
test environment that enables us
to run tests in parallel
GOAL
Big Wins with Docker
Even before 1.0, Docker was a clear choice
• Support and tooling
• Standardization
• Community of mot...
Using Docker allowed us
to build a much better testing
platform than with LXC alone
Codeship Jet
A Docker-based Testing Platform
• Development started in 2014
• First beta in 2015
• Official launch February 2016
A Docker-based Testing Platform
Built with Docker
in order to support Docker workflows
Codeship Jet
2.3K builds per day
~250K total builds
Docker Workflow Tools
• Docker Compose: service and step definition syntax
• Docker Registry: storage for images; remote c...
The workflow tools provided by
Docker are indispensable
Parallel Testing
with Docker
Managing containers with
Docker allowed us to improve our
parallel testing workflow
A New Parallel Workflow
• Introducing services adds additional layer of flexibility
• Loosen coupling between steps and se...
Services
• Pull image from any registry or build from Dockerfile
• Optimize service for testing tasks
• Fully customizable...
Steps
• Each step is executed in an independent environment
• Can be nested in serial and parallel groups
• Two functions
...
User Commands
Universal Container
Pipeline
User Commands
Universal Container
Pipeline
User Commands
Universal Container
Pi...
Step
postgres
redis
command
web
Step
postgres
redis
command
web
Step
command
ruby
Step
postgres
redis
command
web
T1 T2 T3
codeship-services.yml
db:
image: postgres:9.5
app:
encrypted_dockercfg_path: dockercfg.encrypted
build:
image: user/some-i...
codeship-steps.yml
- type: serial
steps:
- type: parallel
steps:
- name: rspec
service: app
command: bin/ci spec
- name: r...
ProTip: Your push or deploy step should
never be part of a parallel step group
Demo!
Docker for Mac and Windows
• All users can test locally
• Jet CLI is available at bit.ly/codeship-jet-tool
• Free, and you...
Engineering
Challenges
Infrastructure
Build allocation
• Customers can choose specs for their build machines
• Machine provisioning used to be pa...
Performance
Image Caching
• Old way: rely on the registry for caching
• A pull gave us access to each parent layer;
rebuil...
Performance
Image Caching
• Great news: 1.11 restores parent/child relationship
when you save the images via docker save
•...
What’s Next?
Swarm!
• Jet was born pre-Swarm
• We manage build machines on AWS via our
own service
• Previous concerns about security —...
libcompose
• Currently use APIs directly for container-level
operations (Jet was also born before Fig was popular)
• Minim...
You can create a highly efficient
parallel testing platform with LXC
alone, but using Docker tools
makes it better
TL;DR
Thank you!
Efficient Parallel Testing with Docker
Efficient Parallel Testing with Docker
Efficient Parallel Testing with Docker
Efficient Parallel Testing with Docker
Nächste SlideShare
Wird geladen in …5
×

Efficient Parallel Testing with Docker

3.623 Aufrufe

Veröffentlicht am

Fast and efficient software testing is easy with Docker. We often
use containers to maintain parity across development, testing, and production environments, but we can also use containerization to significantly reduce time needed for testing by spinning up multiple instances of fully isolated testing environments and executing tests in parallel. This strategy also helps you maximize the utilization of infrastructure resources. The enhanced toolset provided by Docker makes this process simple and unobtrusive, and you’ll see how Docker Engine, Registry, and Compose can work together to make your tests fast.

Veröffentlicht in: Technologie
  • D0WNL0AD FULL ▶ ▶ ▶ ▶ http://1url.pw/VC74i ◀ ◀ ◀ ◀
       Antworten 
    Sind Sie sicher, dass Sie …  Ja  Nein
    Ihre Nachricht erscheint hier

Efficient Parallel Testing with Docker

  1. 1. Efficient Parallel Testing with Docker Laura Frank Senior Engineer, Codeship
  2. 2. 1. Parallel Testing: Goals and Benefits 2. DIY with LXC 3. Using Docker and the Docker Ecosystem Agenda
  3. 3. Parallel Testing
  4. 4. Create a customizable, flexible test environment that enables us to run tests in parallel GOAL
  5. 5. Spend less time waiting around for your automated testing steps and deployments to finish. • Ship newest code to production faster • Be alerted quickly when tests fail Why? If you’re still not sure why testing is important, please talk to me at the Codeship booth.
  6. 6. Developers should have full autonomy over testing environments, and the way tests are executed. • Move testing commands to separate pipelines • Designate commands to be run serially or in parallel • Declare specific dependencies for each service Why?
  7. 7. Let’s eliminate friction in the development cycle
  8. 8. • For local testing, e.g. unit and integration tests run by a development team • On internal CI/CD systems • As part of a hosted CI/CD solution (like Codeship) Where?
  9. 9. • Split up testing tasks • Give developer full control of dependencies per task • Use containers to run multiple tests at once How?
  10. 10. Run tasks across multiple processors in parallel computing environments TASK PARALLELISM
  11. 11. Distributed Task Parallelism A distributed system of containerized computing environments takes the place of a single multiprocessor machine A container is a process, not a small VM
  12. 12. Why not VMs? • Isolation of running builds on infrastructure • Challenges with dependency management • No clean interface for imposing resource limits • Infrastructure is underutilized which makes it expensive
  13. 13. Containers, duh! • Impose resource limits and increase virtualization density • Run customer code in isolation • Provide consistent build environment across many build runs • Run testing tasks in parallel 👍
  14. 14. DIY with LXC
  15. 15. Codeship has been powered by containers since the very beginning
  16. 16. Codeship was founded Flowing salty water on Mars International Year of Forests Preparations ahead of 12.04 Precise release shipped to support LXC improvements Green Bay Packers won Super Bowl XLV 2011: A Brief History Lesson
  17. 17. I should use LXC…
  18. 18. Checkbot (Codeship Classic) • Still running in production as our classic infrastructure • Well-suited for users who want 1-click test environments without much customization • Compromise flexibility for ease of use
  19. 19. Checkbot 39K builds per day 7.8M builds per year
  20. 20. Architecture • Universal container with provided dependencies • Fixed amount of available containers per VM • Implement parallel testing pattern using pipelines with ParallelCI • Users can have N pipelines running in isolation during a build
  21. 21. User Commands Universal Container Pipeline User Commands Universal Container Pipeline Heroku Deployment Provider Capistrano AppEngine Elastic Beanstalk etc…
  22. 22. Limitations • Parity between dev and test • Can’t really debug locally • No useable interface between user and container • Resource consumption is too high
  23. 23. We weren’t able to provide the best, most efficient product to our customers (or ourselves)
  24. 24. Using Docker and the Docker Ecosystem
  25. 25. Create a customizable, flexible test environment that enables us to run tests in parallel GOAL
  26. 26. Big Wins with Docker Even before 1.0, Docker was a clear choice • Support and tooling • Standardization • Community of motivated developers
  27. 27. Using Docker allowed us to build a much better testing platform than with LXC alone
  28. 28. Codeship Jet
  29. 29. A Docker-based Testing Platform • Development started in 2014 • First beta in 2015 • Official launch February 2016
  30. 30. A Docker-based Testing Platform Built with Docker in order to support Docker workflows
  31. 31. Codeship Jet 2.3K builds per day ~250K total builds
  32. 32. Docker Workflow Tools • Docker Compose: service and step definition syntax • Docker Registry: storage for images; remote caching* • Docker for Mac and Windows: give users ability to reproduce CI environments locally *not for long!
  33. 33. The workflow tools provided by Docker are indispensable
  34. 34. Parallel Testing with Docker
  35. 35. Managing containers with Docker allowed us to improve our parallel testing workflow
  36. 36. A New Parallel Workflow • Introducing services adds additional layer of flexibility • Loosen coupling between steps and services — execute N steps against M services • Parallel and serial steps can be grouped and ordered in any way
  37. 37. Services • Pull image from any registry or build from Dockerfile • Optimize service for testing tasks • Fully customizable by the user
  38. 38. Steps • Each step is executed in an independent environment • Can be nested in serial and parallel groups • Two functions • Run: execute a command against a service • Push: push image to registry • Tag regex matching to run steps on certain branches or tagged releases
  39. 39. User Commands Universal Container Pipeline User Commands Universal Container Pipeline User Commands Universal Container Pipeline T1 T1 T1
  40. 40. Step postgres redis command web Step postgres redis command web Step command ruby Step postgres redis command web T1 T2 T3
  41. 41. codeship-services.yml db: image: postgres:9.5 app: encrypted_dockercfg_path: dockercfg.encrypted build: image: user/some-image dockerfile: Dockerfile.test cached: true links: - db deploy: encrypted_dockercfg_path: dockercfg.encrypted build: dockerfile: Dockerfile.deploy
  42. 42. codeship-steps.yml - type: serial steps: - type: parallel steps: - name: rspec service: app command: bin/ci spec - name: rubocop service: app command: rubocop - name: haml-lint service: app command: haml-lint app/views - name: rails_best_practices service: app command: bin/railsbp - service: deploy type: push image_name: rheinwein/notes-app tag: ^master$ registry: https://index.docker.io/v1/ encrypted_dockercfg_path: dockercfg.encrypted 1 2
  43. 43. ProTip: Your push or deploy step should never be part of a parallel step group
  44. 44. Demo!
  45. 45. Docker for Mac and Windows • All users can test locally • Jet CLI is available at bit.ly/codeship-jet-tool • Free, and you don’t need a Codeship account • HUGE advantage over our previous LXC implementation
  46. 46. Engineering Challenges
  47. 47. Infrastructure Build allocation • Customers can choose specs for their build machines • Machine provisioning used to be part of the build process • Now we pool build machines • Allocation time is ~1 second!
  48. 48. Performance Image Caching • Old way: rely on the registry for caching • A pull gave us access to each parent layer; rebuilding the image used the local cache • 1.10 content addressable breaking change
  49. 49. Performance Image Caching • Great news: 1.11 restores parent/child relationship when you save the images via docker save • ETA: 1 month • Double-edged sword of relying on external tools ¯_( )_/¯
  50. 50. What’s Next?
  51. 51. Swarm! • Jet was born pre-Swarm • We manage build machines on AWS via our own service • Previous concerns about security — single tenancy • Swarm (and services like Carina) are promising for the future
  52. 52. libcompose • Currently use APIs directly for container-level operations (Jet was also born before Fig was popular) • Minimal change for our users and builds, but much easier for our engineers • Preliminary work has been completed 🐳
  53. 53. You can create a highly efficient parallel testing platform with LXC alone, but using Docker tools makes it better TL;DR
  54. 54. Thank you!

×