AWS Atlanta Meetup -AWS Spot Blocks and Spot Fleet
1.
2.
3.
4. Significant discount compared to On-
Demand instance pricing.
Pay for compute capacity by the
second or hour with no long-term
commitments
On-Demand
Spare EC2 capacity for up to
90% off the On-Demand price
Reserved Spot
Steady state applications or predictable
usage, databases.
For fault tolerant, instance
flexible or time-insensitive
workloads.
For spiky workloads or to define
needs initially.
5. You can group your accounts into organizational units (OUs) and
attach different policies to each OU.
On-Demand
(2006)
Reserved Instance
(2009)
Spot Block
(2015)
Spot
(2009)
Spot Fleet
(2015)
No Upfront RI
(2014)
All Upfront RI
(2014)
Partial Upfront RI
(2014)
6. The prices changes infrequently
based on supply and demand of
spare capacity
Just request capacity and pay the
current rate.When AWS needs the
capacity back they give a 2 minute
warning.
7. Each Instance Family
Each Instance Size
Each Availability Zone
C3
8XL
4XL
2XL
XL
L
1A 1B 1C
On
Demand
$1.76
$0.88
$0.44
$0.22
$0.11
In Every Region
Is a separate Spot Market
$.50
$.21
$.08
$.27 $.29
$.04
$.01
$.30 $.16
$.07
$.05
$.01
$.08
$.04
$.04
* Pricing shown is for example purposes
only
8. Example CLI command
Spot instances with a specified duration are known as Spot Blocks.
They are designed not to be interrupted and will run continuously for the duration you select
You can specify a duration of 1, 2, 3, 4, 5, or 6 hours.
The price you pay depends on the specified duration.
aws ec2 request-spot-instances --instance-count 5 --block-duration-minutes 120 --type "one-time"
--launch-specification file://specification.json
9. Get Diversified Resources
Diversify your fleet. Grow your availability
AND
Get the Best Price
Find the lowest priced instance power that works for you
or
LaunchThousands of Spot Instances
With one RequestSpotFleet call
Apply Custom Weighting
Create your own capacity unit based on your application needs
10. • Multiple EC2 Spot Instances are Selected
• Multiple Availability Zones selected
• Pick the instances with similar performance
characteristics:
eg c3.large, m3 large, m4.large, r3.large, c4.large
Spot Fleet
11. capacityOptimized
The Spot Instances come from the pool with optimal capacity for
the number of instances that are launching.
diversified
The Spot Instances are distributed across all pools.
lowestPrice
The Spot Instances come from the pool with the lowest price.
This is the default strategy .
InstancePoolsToUseCount
The Spot Instances are distributed across the number of Spot pools
that you specify.This parameter is valid only when used in
combination with lowestPrice
12. https://aws.amazon.com/ec2/spot/instance-advisor/
The Spot Instance advisor helps you
determine pools with the least chance of
interruption and provides the savings you get
over on-demand rates.You should weigh your
application’s tolerance for interruption and
your cost saving goals when selecting a Spot
instance.The lower your interruption rate, the
longer your Spot instances are likely to run.
13. • When the Spot price exceeds your bid
price, the instance will receive a two-
minute warning
• Check for the 2 minute spot instance
termination notification every 5 seconds
leveraging a script invoked at instance
launch.
15. Cannot be stateless and fault tolerant? Running a Monolith or in-mem cache on the instance?
Running Dev/Test on Spot could still be successful as an interim step, risk vs cost calculation,
choose low interruption rate instance types
Running periodic benchmarks or load tests - Use Spot Blocks
To ensure that load does not decrease against app under test when
Spot is interrupted
Jenkins Build Slaves - use Spot Fleet Plugin
Jenkins will automatically increase Spot Fleet target capacity upon
pending jobs and scale in when no queue
16. Your instance type requirements, budget requirements, and application design will determine how to apply the
following best practices:
Be Flexible about InstanceTypes
Test your application on different instance types when possible.
Because prices fluctuate independently for each instances type in an Availability Zone, you can often get more
compute capacity for the same price when you have instance type flexibility. Request all instance types that meet
your requirements to further reduce costs and improve application performance.
Spot Fleets enable you to request multiple instance types simultaneously.
Minimize the impact of interruptions .
Amazon’s EC2 Spot’s Hibernate feature allows you to pause and then resume Amazon EBS backed instances when
capacity is available. Hibernate is just like closing and opening your laptop lid, with your application starting up right
where it left off.
17. CONTINUED
Choose pools where prices haven’t changed much
Because prices adjust based on long-term demand, popular instance types (such as recently launched instance
families), tend to have more price adjustments. Therefore, picking older-generation instance types that are less
popular tends to result in lower costs and fewer interruption. Similarly, the same instance type can have different
prices in different Availability Zones.
Start with LaunchTemplates sooner then later
These will be required for new features.
19. 1. Open up the ElasticContainer Service Console using the link https://console.aws.amazon.com/ecs/ .
2. On the left hand menu under Amazon ECS click on Clusters
3. Now click on the blue Create Cluster Button.
4. Select the EC2 Linux + Networking template then click on the Next Step button at the bottom.
5. Enter the cluster name as SpotECScluster
6. Under the provisioning model - Select Spot
7. For the Spot Instance allocation strategy - choose Diversified
8. Choose an instance type(s) from the dropdown (suggested r3.large) and put in number of instances as 3
9. Under maximum price, check table - try to go for $0.02 per hour if you set (optional)
10. Leave the other values as default and then choose a keypair to launch the instances with
11. Choose create a new role for the ECS instances if you don’t already have a specified ECS role
12. Now click on the blue Create Button.
13. When the cluster completion creates you will be able to see the EcsSpotFleet logical ID
14. Copy this ID to your clipboard
15. Go to the EC2 services part of the Console https://console.aws.amazon.com/ec2/
Testing Spot Fleet
20. 15. Under Instances choose Spot Requests on the Left Hand menu
16. Paste the resource id from your clipboard into the search bar at the top
17. If you expand the request you can see the EC2 instances which are part of the request
18. Go back to the ECS console and view your cluster
We will schedule a task
19. Click on the Services tab
20. Click the Blue Create tab under the Services tab
21. On the Configure Service screen - choose EC2 as the Launch type
22. Choose yourTask Definition and revision
23. Choose the SpotECScluster from the Cluster name Dropdown list
24. Change Minimum Healthy Percent to 50 (leave everything else defaults)
25. Click Next Step
26. Choose None for load balancer
27. Click Create Service
28. Go back to the EC2 Console -> Spot Requests and select the current Spot Fleet
29. At the top of the menu SelectActions -> Modify target Capacity
30. Set the new target capacity to 2 (from previous value of 3) , Click Submit
Testing Spot Fleet
Hinweis der Redaktion
You simply submit a Spot instance request and use the new BlockDuration parameter to specify the number of hours your want your instance(s) to run, along with the maximum price that you are willing to pay. When Spot instance capacity is available for the the requested duration, your instances will launch and run continuously for a flat hourly price. They will be terminated automatically at the end of the time block (you can also terminate them manually). This model is a good for situations where you have jobs that need to run continuously for up to 6 hours.
Either do Whitelisting or Blacklisting - but don’t do both
Either do Whitelisting or Blacklisting - but don’t do both
Even though you access the hub console from Oregon, you can move into any region as long as the migration tool supports it.
Even though you access the hub console from Oregon, you can move into any region as long as the migration tool supports it.