Learn how to leverage Amazon EC2 Spot Instances to create CI/CD pipelines, Jenkins build nodes, and test environments at a fraction of the cost of On-Demand Instances. We also implement mechanisms to ensure that our CI/CD pipelines remain available by decoupling the application state from our compute resources. Finally, we migrate everything to a container environment for maximum performance and cost efficiency. In addition to covering the details of Spot Instances, we share how some customers use Spot Instances to reduce the cost of their test and production solutions. Bring a laptop, and have an AWS account with IAM administrator permissions. All attendees must bring their own laptop (Windows, macOS, and Linux all supported). Tablets are not appropriate. We also recommend having the current version of Chrome or Firefox installed.

2. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Run Your CI/CD and Test Workloads
for 90% Less with Amazon EC2 Spot
Instances
Jarrod Spiga
Solutions Architect
Amazon Web Services
C M P 4 0 1
Chad Schmutzer
Sr. EC2 Spot Specialist SA
Amazon Web Services

3. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Agenda
Workshop preparation and an introduction to Amazon Elastic Compute Cloud
(Amazon EC2) Spot
Lab one: Reduce the cost of builds using Amazon EC2 Spot Fleet
Lab two: Deploy testing environments using Spot & launch templates
Lab three: Externalize state data to add resiliency to Jenkins
Lab four: Using containers backed by Spot Instances in Auto Scaling groups
Workshop clean up

4. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.

5. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Workshop prerequisites
Before you get started, please check that you have the following
A laptop or tablet device with an SSH client.
Please ensure that this device is connected to the re:Invent Wi-Fi network.
An AWS account where you have “administrator access” IAM policy level rights.
The cost for running the resources required for this workshop should only amount to a couple of dollars.
That said, we will be providing you with AWS credits to cover this spend at the completion of the
workshop.
Additionally, we will be providing detailed clean up instructions to ensure that resources are not left
running in the account that you use.
If you don’t meet these prerequisites, reach out to a session facilitator.

6. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Workshop preparation
Ensure that you’re using the EU (Ireland) region
An interactive workshop guide is available at the following address
https://amzn.to/reInvent2018-spotcicdworkshop
Please commence following the instructions contained in the workshop guide up
to and including the workshop preparation page
If you have trouble deploying the AWS CloudFormation template, please reach out
to a session facilitator

7. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.

8. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Amazon EC2 purchasing options

9. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
With Amazon EC2 Spot, the rules are simple

10. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Jenkins master
What has the AWS CloudFormation template deployed?
AWS CloudFormation
stack
EC2 (On-Demand)
eu-west-1beu-west-1ceu-west-1a
Application load
balancer
S3 Bucket
eu-west-1 (Ireland) region

11. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.

12. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Spot use case: Batch processing
Leverage per-
second billing
Optimize for
lowest cost
Determine job
completion and
retry failed jobs
Be instance
flexible
Also consider AWS Batch for batch workloads or AWS CodeBuild as a build agent service

13. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Jenkins master
What are we doing in lab one?
eu-west-1beu-west-1ceu-west-1a
Application load
balancer
Jenkins agent
Jenkins agent
Spot Fleet
You
EC2 (Spot)
GitHub
eu-west-1 (Ireland) region

14. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Lab one: Important notes
When provisioning your Spot Fleet, use a lowest price allocation strategy in order
to minimize cost
You’ll need to configure the AWS Spot Fleet plug-in with an access key and secret
key from the SpotCICDWorkshopJenkins AWS Identity and Access Management
(IAM) user that has already been created by AWS CloudFormation

15. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.

16. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Spot use case: Stateless applications
Cloud best
practices also
apply when
using Spot
Diversify to
improve
resilience
Don’t rule out
older instance
generations
Embrace the
chaos

17. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Jenkins master
What are we doing in lab two?
eu-west-1beu-west-1ceu-west-1a
Jenkins
application load
balancer
Jenkins agent
App host
Spot Fleet
You
Workshop AWS
CloudFormation
stack
Testing
application load
balancer
Spot Fleet
Artifact S3
bucket
eu-west-1 (Ireland) region
GitHub
DynamoDB table
Deployment
Lambda function
EC2 (Spot)
EC2 (Spot)App host
Testing AWS
CloudFormation
stack

18. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Lab two: Important notes
You should increase the number of executors on your build agents for this lab
(three should suffice).
The syntax for the Jenkins pipeline is contained in the lab guide.
There is an issue with the community-developed AWS Lambda plug-in where
output values from the Lambda function are not stored in Jenkins environment
variables when configured to do so. To work around this issue, the Lambda
function is persisting state data in an Amazon DynamoDB table. Similar
mechanism could be used for other elements of your deployment workflows.

19. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.

20. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Spot use case: Stateful applications
Many apps
persist data to
local file systems
Amazon Elastic File
System (Amazon
EFS) can be used to
store data
independent of
compute instances
AWS has a wide
variety of other
services to
persist data
Consider your
RPOs and RTOs

21. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Spot Fleet
Jenkins master
What are we doing in lab three?
eu-west-1 (Ireland) region
eu-west-1beu-west-1ceu-west-1a
Application load
balancer
Jenkins master
Jenkins master
You
EFS volume
AWS CloudFormation
stack
EC2 (Spot)
EC2 (Spot)

22. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Lab three: Important notes
This lab also uses launch templates in order to simplify the deployment of Amazon
EC2 instances
The EFS file system ID (deployed by the AWS CloudFormation template) can be
found in the outputs tabs for the AWS CloudFormation stack
Jenkins stores all of the data that it needs to persist on the file system under
/var/lib/jenkins

23. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.

24. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Spot use case: Containers
Use the two-
minute warning
to drain
containers
Specify target
capacity by
weighted
resources
Use Amazon
CloudWatch
Alarms to trigger
scaling
Diversify across
capacity pools,
bid based on
defined weight

25. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Connection draining during the two-minute warning
#!/bin/bash
while sleep 5; do
if [ -z $(curl -Isf http://169.254.169.254/latest/meta-data/spot/termination-time) ]; then
/bin/false
else
ECS_CLUSTER=$(curl -s http://localhost:51678/v1/metadata | jq .Cluster | tr -d ")
CONTAINER_INSTANCE=$(curl -s http://localhost:51678/v1/metadata
| jq .ContainerInstanceArn | tr -d ")
aws ecs update-container-instances-state --cluster $ECS_CLUSTER
--container-instances $CONTAINER_INSTANCE --status DRAINING
fi
done

26. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Amazon EC2 Fleet

27. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Before: Multiple ASGs to use Spot, On-Demand, and RIs together
m4.large Spot ASG min: 1 max: 10
m5.large Spot ASG min: 1 max: 10
c4.xlarge O-D ASG min: 1 max: 10
Availability
Zone 1
Availability
Zone 2
Availability
Zone 3
The old way
with three ASGs
—one for each
instance type/
purchase option

28. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
After: Include Spot, On-Demand and RIs in a single ASG
m4.large Spot Instances
m5.large Spot Instances
c4.xlarge On-Demand instances
Availability
Zone 1
Availability
Zone 2
Availability
Zone 3
The new way
combines purchase
options, instance
types, and AZs in a
single ASG
Single ASG

29. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
What are we doing in lab four?
eu-west-1 (Ireland) region
eu-west-1beu-west-1ceu-west-1a
Application load
balancer
Jenkins master
Auto
Scaling
group
You
EFS volume
AWS CloudFormation
stack
Spot Fleet
EC2 (Spot)
EC2 (On-Dem.)
jenkins.local
Amazon Route 53
hosted zone

30. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Lab four: Important notes
Auto-scaling of EC2 resources within the Amazon Elastic Container Service
(Amazon ECS) cluster has not been configured, though it should not be required
for this workshop. While the deployed Auto Scaling group provides self-healing
capabilities, you may want to implement scaling triggers in your environments.
When deployed via YUM, Jenkins runs with a different UID than what the
containerized version runs with—hence why you must change the ownership of
data on the EFS volume.
As accessing containers via SSH is trivial, build agents will report status back to the
master using the Java Native Launch Protocol (JNLP), with assistance by Amazon
ECS Service Discovery to allow agents to locate the master.

31. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.

32. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Cleaning up your account
Please execute these steps in order
• Delete all objects within the S3 bucket created by AWS CloudFormation
• Modify the Jenkins Master ECS Service so that 0 tasks are desired
• Terminate all Auto Scaling groups
• Cancel all remaining Spot requests
• Ensure that all EC2 instances have been terminated
• Delete all AWS CloudFormation stacks
• Delete all CloudWatch Log groups related to the SpotCICDWorkshop
• Remove the Amazon EC2 keypair created during the workshop preparation
phase
More prescriptive instructions are found in the lab guide

33. Thank you!
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Jarrod Spiga
spigaj@amazon.com
Chad Schmutzer
schmutze@amazon.com

34. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.