Today developers want to change the code, build and deploy often, even several times per day. New versions of software may need to be tested on different distributions, and with different configurations. Achieving this with Virtual Machines it’s possible, but it’s very resource and time consuming. Docker provides an incredibly good solution for this, in particular if combined with Continuous Integration tools like Jenkins and Configuration Management tools like Puppet. This presentation focuses on the opportunities to configure automatically Docker images, use Docker containers as disposable workers during your tests, and even running your Continuous Integration system inside Docker.

1. Docker and Puppet for
Continuous Integration
Giacomo Vacca, Founder at RTCSoft, gv@rtcsoft.net

2. About me
• 15 years “in the trenches cubicles”
• Developer of RTC (VoIP, IM, WebRTC) solutions
• Founder of RTCSoft in 2015
@giavac
https://github.com/giavac
gv@rtcsoft.net

3. Docker usage scenarios
• Run services
• Packaging/Deployment
mechanism
• Fast prototyping
• Continuous Integration/
Delivery

4. Continuous Integration 1/2
• A team of developers work on the same project
• Each developer commits several times per day
• Each commit is potentially verified (Build, Deploy, Test)
• The project is always kept in a stable/releasable state

5. Continuous Integration 2/2
• Developers work on personal/bug/feature (short-lived) branches
• These branches are merged into the “release” branch for the release
Image source: www.atlassian.com

6. CI components
Image source: myself (This is why it sucks)

7. Server-side apps for Linux
• Apps are built, packaged, deployed and tested
• Unit testing during the builds
• Component/Integration/Load testing after test deployments
• Artefacts are uploaded into file servers or deb/yum repos

8. Old school: “Build and run”
Build the app on the runtime machine
• PROBLEMS:
• Repeat the build on each host
• Unnecessary pollution of the runtime host
• Maintenance nightmare: the easiest upgrade is problematic
• Potential exposure of source code to external attacks

9. Using a “Build machine”
Dedicate a machine to the various builds
• PROs:
• Keep the runtime environment reasonably clean
• Source code is protected inside the private network
• CONs:
• You’re reducing a multi-dimensional problem into a bi-dimensional one
• You’ll pay the price one day!

10. • Widely adopted, Open Source CI tool
• Core + many plugins
• Integrates well with git and Docker
• Can manage multiple “slaves”
https://jenkins-ci.org/

11. Jenkins and Master worker
• “Cron jobs manager with a GUI”
• What’s a Jenkins job?
• The simplest case: Jenkins runs the jobs inside its own host
• “Master” worker, no slaves involved
• Still you need to manage Jenkins’ configuration…

12. Building machineS
Assign a VM per building scenario (app/OS)
• A step in the right direction
• CONs:
• Hard to maintain (see Configuration Management later)
• VM resources allocated even when the builds are not running
• Explosion of number of VM types to use (OS/versions matrix)
• VMs are “heavy”

13. Jenkins and Slaves
• Configure a number of “slave workers”
• Assign a slave to a job
• But you need to maintain all those slaves…

14. Configuration Management
You need something better than that.

15. Configuration Management intro
• Define programmatically the configuration of a machine
• Many tools available: Puppet, Ansible, Chef, Salt, etc.
• Make configuration predictable AND fix divergence automatically
• Track changes during time (it’s all text)
• Define a formal language across developers, sysadmin, Ops

16. • Puppet defines the final state of a host (What, not How)
• Has its own declarative syntax
• Written in Ruby
• Designed for Master-Slave interaction, can be used Standalone
• Idempotent
https://puppetlabs.com/

17. Configuration Management
You really, really need it. Even if you like 3-ways diffs and hate the term “idempotence”.

18. Example of Puppet codepackage { 'nginx':
ensure => present,
} ->
file { '/etc/nginx/nginx.conf':
ensure => file,
owner => 'root',
group => 'root',
mode => '0644',
content => template('mymodule/nginx.conf.erb'),
notify => Service['nginx'],
} ->
file { '/etc/nginx/conf.d/global.conf':
ensure => file,
owner => 'root',
group => 'root',
mode => '0644',
content => template('mymodule/nginx_global.conf.erb'),
notify => Service['nginx'],
}
service { 'nginx':
ensure => running,
enable => true,
}

19. Back to Jenkins and multiple Slaves
• We saw the approach with multiple slaves, with VMs as slave
• Functionally valid
• Cumbersome to maintain
• What if we could have “on demand”, lightweight workers? e.g.
Docker containers?

20. • A new form of virtualisation (w/ real HW performance)
• A delivery mechanism
• Many “images” freely available
• “Here’s the ingredients and recipe” VS “Here’s the cake”
https://www.docker.com/

21. Virtual Machines vs Docker
Source: https://www.docker.com/what-docker

22. Docker workflow
• Build an image (Dockerfile + base images)
• docker build -t gvacca/nginx .
• Store image in a registry (Public, e.g. Dockerhub or private)
• docker push gvacca/nginx
• Pull image on a target host
• docker pull gvacca/nginx
• Run container on a target host
• docker run -it gvacca/nginx

23. Dockerfile - example
FROM ubuntu:latest
MAINTAINER Giacomo Vacca <gv@rtcsoft.net>
RUN apt-get update
# Install nginx
RUN apt-get -y -q install nginx
# Prepare target dir for website (Must match global.conf)
RUN mkdir -p /var/www/html
# Configure nginx
COPY nginx/nginx.conf /etc/nginx/nginx.conf
COPY nginx/global.conf /etc/nginx/conf.d/global.conf
EXPOSE 8080
CMD [ "/usr/sbin/nginx" ]
https://github.com/giavac/docker-experiments/tree/master/simple_nginx

24. Build a Docker image
gvacca@dockerubuntu:~simple_nginx$ sudo docker build -t gvacca/simple-nginx .
Sending build context to Docker daemon 7.168 kB
Step 1 : FROM ubuntu:14.04
---> 1d073211c498
…
Step 2 : MAINTAINER Giacomo Vacca <gv@rtcsoft.net>
---> Running in 10a8334becfd
---> 559310abf4fb
Removing intermediate container 10a8334becfd
Step 3 : RUN apt-get update
---> Running in 873b09debab0
…
Step 8 : EXPOSE 8080
---> Running in a27f73413c02
---> f5ef8527f2a3
Removing intermediate container a27f73413c02
Step 9 : CMD /usr/sbin/nginx
---> Running in d99de19cc782
---> df76d20cd00f
Removing intermediate container d99de19cc782
Successfully built df76d20cd00f

25. Run the container!
gvacca@dockerubuntu:simple_nginx$ sudo docker run -d --name nginx -p 8080:8080 -v
$PWD/website:/var/www/html/website gvacca/simple-nginx
add8d371f82f615ebbd121cea804511dcdafac893b14325366744797424fa44c
gvacca@dockerubuntu:simple_nginx$ sudo docker ps
CONTAINER ID IMAGE COMMAND CREATED
STATUS PORTS NAMES
add8d371f82f gvacca/simple-nginx "/usr/sbin/nginx" 11 seconds ago
Up 9 seconds 0.0.0.0:8080->8080/tcp nginx
gvacca@dockerubuntu:simple_nginx$ ps aux |grep docker
root 792 0.0 4.7 693136 23740 ? Ssl Nov04 11:20 /usr/bin/docker
daemon
root 1977 0.0 2.6 152692 13172 ? Sl 11:26 0:00 docker-proxy -
proto tcp -host-ip 0.0.0.0 -host-port 8080 -container-ip 172.17.0.2 -container-port
8080

26. Intermediate topics
• Volumes
• Network (was ‘link’ before 1.9)
• Debugging (‘inspect’)
• ARG (after 1.9)
• Docker on OSX/Windows (Docker Machine)
• Interacting with Kernel modules

27. Docker orchestration
• Docker Composer
• Docker Swarm
• Google’s Kubernetes
• Apache Mesos
• Puppet & others

28. Let’s start exploring interaction
opportunities between the tools
seen so far…

29. Puppet to manage Jenkins
• https://forge.puppetlabs.com/rtyler/jenkins
• Manage Jenkins as a service
• Install plugins automatically, etc
• Or build your own module!
• Install .war
• Configure Jenkins (config.xml)
• Configure jobs ($JENKINS_PATH/jobs/JOB/config.xml)

30. Docker as Slave for Jenkins
• Associate specific Docker images to specific Jenkins build jobs
• Keep builds clean & reproducible (“Non-event releases”)
• Run slave containers on demand - stop them when not needed.
• Jenkins Plugin: https://wiki.jenkins-ci.org/display/JENKINS/Docker
+Plugin
• Other Cloud methods, e.g. Mesos

31. Jenkins to build Docker images
• A Jenkins job:
• Checks out a Dockerfile
• Builds the image
• Upload the new image in a repo
• May re-use images for new builds (careful about integrity)

32. Puppet to manage Docker containers
• Module for docker: https://forge.puppetlabs.com/garethr/docker
• Installs Docker and required dependencies
• Launches the Docker daemon
• Sets DNS, users and other configuration items
• Pull images from selected repos
• Run containers (image + command)

33. Run Jenkins inside Docker
• Manage Jenkins with a dedicated image + volume with data
• From the official Docker registry: https://hub.docker.com/_/jenkins/
docker run -d -p 8080:8080 -p 50000:50000 -v ~:/var/jenkins_home jenkins
• TCP 8080: Web UI
• TCP 50000: interface to connect slaves

34. Docker inside Docker
But you need the privileged mode

35. Conclusion
• There are many opportunities to automate Continuous Integration
• CI for server-side apps poses specific challenges
• You can’t ignore Configuration Management
• Docker represents a huge opportunity
• You need to find your own use case and solution

36. APPENDIX - Puppet vs Docker
• Some stop using Puppet for Configuration Management once they
move their apps inside containers
• A Dockerfile defines “how” to build an image
• A Dockerfile must be distribution-specific (yum or apt?)
• Puppet is OS-agnostic: the differences are hidden inside modules

37. Questions and Answers
Thanks

38. Recommended Books
• “The Docker book”, J. Turnbull, http://www.amazon.co.uk/Docker-
Book-Containerization-new-virtualization-ebook/dp/B00LRROTI4
• “Continuous Delivery”, J. Humble, http://www.amazon.com/
Continuous-Delivery-Deployment-Automation-Addison-Wesley/dp/
0321601912
• “Pro Puppet”, J. Turnbull, http://www.amazon.co.uk/Pro-Puppet-
Second-Professional-Apress/dp/1430260408

39. See also…
http://www.slideshare.net/GiacomoVacca/when-voip-meets-web-the-
revolution-of-webrtc