AWS Technical Day - Optimizing costs with Spot Instances

1. Run large scale workloads
cost effectively in the cloud
EC2 Spot Instances
#ec2spot
Jafar Shameem

2. Agenda
• What is Spot?
• Use Cases
• Best Practices
• Spot Fleet

3. What is Spot?
Name your own price for EC2
Compute
– A market where price of compute
changes based upon Supply and
Demand
– When Bid Price exceeds Spot Market
Price, instance is launched
– Instance is terminated (with 2 minute
warning) if market price exceeds bid
price
– Unused On-Demand Instances

4. On average, AWS adds enough new
server capacity every day to support
Amazon’s global infrastructure when it
was a $7B business.

5. Pools of capacity
General-purpose: M1, M3 , T2
Compute-optimized: C1, CC2, C3, C4
Memory-optimized: M2, CR1, R3, M4
Dense-storage: HS1, D2
I/O-optimized: HI1, I2
GPU: CG1, G2
Micro: T1, T2
.micro
.medium
.large
.xlarge
.2xlarge
.4xlarge
.8xlarge
Windows
Linux
-1a
-1b
-1c
….
Type Size OS AZ

6. c3.2xlarge
On-Demand
Price:
$0.420/hr

7. cc2.8xlarge
32 cores, 60.5 GB
memory
On-Demand
Price:
$2.00/hr

9. Some Spot Use Cases
• Stateless Web/App Server Fleets
• Hadoop Workloads
• Continuous Integration (CI)
• High Performance Computing (HPC)
• Grid Computing
• Media Rendering / Transcoding

10. Web-based Architecture with Spot
Elastic Load
Balancing
Stateless
Web Servers
Stateless
Web Servers
On Demand
Autoscaling group
Session
State Data
Stateless Web
Servers (spot)
Stateless Web
Servers (spot)
Spot Autoscaling
group
Availability Zone A
Availability Zone B
Stateless Web
Servers (spot)
Stateless Web
Servers (spot)
Spot Autoscaling
group

11. Scaling Hadoop Jobs with Spot
http://engineering.bloomreach.com/strategies-for-reducing-your-amazon-emr-costs/
Bloomreach
launches 1,500 to
2,000 Amazon EMR
clusters and run
6,000 Hadoop jobs
every day.

12. Continuous Integration
& Testing with Spot
• Tapjoy - Premier Mobile Ad Network Across iOS & Android
• Global Network (435 Million Monthly Reach)
• Jenkins + Spot Instances
• https://github.com/bwall/ec2-plugin (thanks to an RIT senior project)
• Go wide during business hours, scale back in the evenings.
Automatically kicks online at 06:00ET
• Workers scale horizontally to support dozens of simultaneous regression
tests spread out over dozens of workers
• Jenkins automatically guards against spot termination

13. Queue-based media transcoding Ooyala
- Video technology
platform that serves
ESPN, Bloomberg, ...
- Uses combo of
OD/RI/Spot to ensure it
can cover predicted
volumes while keeping
costs low
- http://aws.amazon.com/solutions/case-
studies/ooyala/
Vevo
- Library of over 75,000
HD videos
- Must be able to rapidly
transcode library to a
new screen format
- Can spin up 100s of Spot
instances to transcode
entire library in a matter
of days (instead of the
weeks)

14. Some example frameworks with built-in Spot
support
• Hadoop
– Elastic MapReduce (EMR)
• High Performance Computing
– CFNCluster (https://github.com/awslabs/cfncluster)
– Cycle Computing
• Web Applications / Microservices
– AutoScaling
• Continuous Integration
– Jenkins
– Bamboo

15. Using Spot Effectively – Normalize
- CPU Generation
- Memory/core
- Networking
- VPC or Classic EC2

16. Using Spot Effectively – Diversify
- Regions (US-East1, EU-West2, US-West2,…..)
- Availability Zones (US-East1a, US-East1b, US-East1c,…)
- Instance Families (c3, r3, ….)
- Instance Types (c3.xlarge, r3.xlarge, m3.xlarge,….)

17. Using Spot Effectively – Bidding Strategies
• You only pay what the Market price is
• But, bid what you are willing to pay
• You pay for the price as you enter the hour
• And pay for it at the end of the hour
• If you get interrupted, you don’t pay for that hour
Bid only what you are willing to pay.
(by default, bid limited to 10 * On Demand Price)

18. Frontend Applications
on On-Demand/Reserved Instances
+
Backend Applications*
on Spot Instances
* e.g., batch video transcoding
• Example:
Using Spot Effectively: Mix and Match

19. Using Spot Effectively – let AWS do the heavy lifting
Auto Scale
as-create-launch-config spotlc-5cents
--image-id ami-e565ba8c
--instance-type m1.small
--spot-price “0.05”
. . .
as-create-auto-scaling-group spotasg
--launch-configuration spotlc-5cents
--availability-zones “us-east-1a,us-east-1b”
--max-size 16
--min-size 1
--desiredcapacity 3

21. Introducing Spot Fleet
• Instead of writing all that code to manage Spot
Instances, simply specify:
– Target Capacity – The number of EC2 instances that you want
in your fleet.
– Maximum Bid Price – The maximum bid price that you are
willing to pay.
– Launch Specifications – # of and types of instances, AMI id,
VPC, subnets or AZs, etc.
– IAM Fleet Role – The name of an IAM role. It must allow EC2 to
terminate instances on your behalf.

22. Introducing Spot Fleet
• Optionally also specify:
– Client Token – A unique, case-sensitive identifier for the
request.
– Valid From – The start date and time of the request
– Valid Until – The end date and time of the request
– Terminate on Expiration – If set to TRUE, all Spot instances in
the fleet will be terminated when the Valid Until time is reached.

23. Spot Fleet
• Will attempt to reach the desired target capacity
given the choices that were given
• Manage the capacity even as Spot prices
change
• Launch using launch specifications provided

24. Using Spot Fleet
• Create EC2 Spot Fleet IAM Role
• Requesting a fleet:
– aws ec2 request-spot-fleet --spot-fleet-request-config file://mySmallFleet.json
• Describe fleet:
– aws ec2 describe-spot-fleet-requests
– aws ec2 describe-spot-fleet-requests --spot-fleet-request-ids <sfr-………..>
• Describe instances within the fleet
– aws ec2 describe-spot-fleet-instances --spot-fleet-request-id <sfr-…………>
• Cancel Spot Fleet (with termination):
– aws ec2 cancel-spot-fleet-requests --spot-fleet-request-ids <sfr-…………..> -
terminate-instances

25. mySpotFleet.json
{
"TargetCapacity": 5,
"SpotPrice": "1.00",
"IamFleetRole": "arn:aws:iam::962872214910:role/fleetRole",
"LaunchSpecifications": [
{
"ImageId": "ami-ff527ecf",
"InstanceType": "m1.small"
},
{
"ImageId": "ami-ff527ecf",
"InstanceType": "m1.medium"
},
{
"ImageId": "ami-ff527ecf",
"InstanceType":"m1.large"
}
]
}

26. Further reading
• EC2 Spot Instances:
– http://aws.amazon.com/ec2/purchasing-options/spot-instances/
• EC2 Spot Fleet API:
– https://aws.amazon.com/blogs/aws/amazon-ec2-spot-fleet-api-
manage-thousands-of-instances-with-one-request/
• Spot Best Practices:
– https://aws.amazon.com/blogs/aws/focusing-on-spot-instances-
lets-talk-about-best-practices/

27. EC2 Spot Instances
#ec2spot
Jafar Shameem
shameemj@amazon.com