This document discusses using Jenkins and Amazon ECS for continuous delivery of Docker containers. It describes setting up a Jenkins master and agent in AWS, using plugins to manage Docker containers. The Jenkinsfile and Dockerfile are kept in source control. The pipeline builds a Docker image, pushes it to Docker Hub, then updates an ECS service to deploy the new image. Finally, it provides the URLs to demo the continuous delivery pipeline deploying an app to staging and production environments on ECS.

1. Continuous Delivery with
AWS ECS and Jenkins
Tracy Kennedy - 5/12/2016

2. February 23, 2016
Email: tkennedy@cloudbees.com
Git|Docker Hub: lavaliere
Twitter: @Tracy_Kennedy
Who am I?

3. What is continuous delivery?
Development Production
Commit Build Test Stage Deploy
Feedback Loop

4. What is Jenkins?

5. What is Jenkins?
Over
1000 Jenkins
Plugins
Integration
with over 100
DevOps Tools
Orchestration
of the DevOps
Toolchain
End-to-End
CD Pipeline
Management
Code & Commit Build & Config Scan & Test Release Deploy

6. What does CloudBees do?
What does CloudBees do?

7. What is Amazon Elastic Container Service
(ECS)?

8. Using Docker with Jenkins
+
Jenkins Pipeline
TEST
STAGE
PRODUCTION
App
<code>
(git, etc.)
Gold
Docker
Image
(~per app)
<OS config>
Certified
Docker
Images
(Ubuntu, etc.)
<OS config>

9. Jenkins and container management

10. Jenkins and container management

11. Target deployment environment in
ECS
ECS Cluster Management
Engine
Task Task
Agent
Communication
Service
ECS
Agent

12. Target development pipeline

13. Setting up Jenkins in AWS
Master spec:
T2.medium - 2 vCPU, 4 GB memory
Amazon Linux
Jenkins 2.1 RPM
Tools installed:
Git - 2.7.4-1.47.amzn1
Credentials set up:
Github - Username with password

14. Plugins Used
CloudBees Docker Pipeline Plugin
CloudBees Docker Build and Publish Plugin
CloudBees AWS CLI Plugin*
Pipeline Plugin
Docker Plugin
Pipeline Stage View Plugin
Git Plugin
Docker Hub Plugin

15. Setting up Jenkins in AWS
Agent spec:
AMI ID: ami-07b3b36d
T2.medium - 2 vCPU, 4 GB memory
Ubuntu
Installed tools:
Docker - 1.10.3 Ubuntu package
Maven - 3.0.5 Ubuntu package
Git - 2.7.3 Ubuntu package
Java 8 - OpenJDK 1.8.0_72-internal

16. The Jenkinsfile
● Kept in GitHub project’s root directory, Jenkins executes
● Scripts out the target deployment pipeline
● Triggered by pipeline job

17. The Dockerfile
● Kept in GitHub project’s root directory - copied into the Jenkins workspace
● Copies built artifact into a Docker container running Tomcat 8
● Triggered by pipeline step

18. Target Docker registry - Docker Hub
● Docker pipeline step “withRegistry” specifies target registry for push
● “push” step sends built image to target registry, tagged “docker-demo”

19. Target flow in ECS
● ECS steps run in an agent container (cloudbees/java-build-tools:0.0.7.1) with
AWS CLI
○ AWS credentials in Jenkins
○ Wraps AWS-related services
○ Calls ECS agent to trigger task that pulls the newly built app image
○ Checks ECS service status
○ “update-service” = “docker stop”
○ Manually gated deployment

20. Target environments in ECS
● 2 clusters: “production” and “staging” for the webapp
○ Service per cluster
■ Task definitions for pulling the app image
○ EC2 instance per cluster
■ “ECS-optimized”
■ IAM roles

21. Demo time!
ECS: https://console.aws.amazon.com/ecs/home?region=us-east-1#/clusters
Jenkins: http://52.72.46.249:8080/job/game-pipeline/
Github: https://github.com/lavaliere/game-of-life
Docker Hub: https://hub.docker.com/r/lavaliere/game-of-life/
Staging: http://52.200.92.100/

22. Challenges and blockers
Usability and troubleshooting
Learning curve
Autoscaling
“Point of no return” configs
Silent failures

23. Additional reading
Jenkins Pipeline
How to point Jenkins to a custom registry (e.g. local)
Setting up Jenkins slaves on AWS
Game of Life pipeline deployment to ECS
AWS’s alternative approach to building a pipeline with Jenkins and ECS
Creating an ECS cluster with ELB and autoscaling
Building multiple containers with Docker Compose and pipeline
Blue-green ECS deployments

24. Jenkins goodies
Join our meetup group(s)!
Jenkins http://www.meetup.com/San-Francisco-Jenkins-Area-Meetup/
Product management http://www.meetup.com/San-Jose-Product-
Management-Meetup/
Upcoming area events
Jenkins World - Sept 13-15 in Santa Clara
Jenkins 2.0 Webcast - May 17 online
JFrog swampUp - May 23 in Napa