Technical deep dive in to 10 AWS Cloud best practices with in-depth look at the tips and tricks of architecting on the AWS platform.

1. AWS Government, Education, &
Nonprofits Symposium
Canberra, Australia | May 20, 2014
Managing Seasonal Workloads on AWS
Clayton Brown
Ecosystem Solution Architect

2. Managing Seasonal Workloads on AWS

3. Why are customers adopting cloud computing?
Variable expense Replace capital expenditure with variable expense
Source
IDC
Whitepaper,
sponsored
by
Amazon,
“The
Business
Value
of
Amazon
Web
Services
Accelerates
Over
Time.”
July
2012
Average
of
400
servers
replaced
per
customer

4. Economies of scale Lower variable expense than companies can achieve
themselves
Why are customers adopting cloud computing?
Saved
$34m
on
SmartHub
applica;on
10’s
of
millions
of
$
saved
with
first
12
apps
migrated
to
AWS
50%
reduc;on
in
analy;cs
costs

5. Mul;ple
global
regions
helps
build
highly
available
applica;ons
Web
Server
Availability
Zone
1
Web
Server
Availability
Zone
2
Web
Server
Regional AWS design provides Highly Availability as a Baseline

6. Corporate Data
Center
Which can be fully integrated with existing assets

7. Demand
Time
Week
1
Week
2
Week
3
Week
4
Week
5
Wasted
Capacity
Lost
Customers,
Rush HardwareWasted
Capacity
Lost
Customers,
Rush
Hardware
Lost Customers,
Rush Hardware
1m
1.5m
2.0m
Scaling on-premise infrastructure can be a challenge

8. Sizing capacity for peak is harder even still
Demand
Q1
Q2
Q3
Q4
Q1
Wasted
Capacity
Lost
Customers,
Order
Hardware
Wasted
Capacity
Wasted
Capacity
Wasted
Capacity
200k
300k
600k
Time
Capacity
of
Resources
Actual
Demand

9. 3000 Cores for risk management processesNumberofCores
300 Cores on
weekends
Thu Fri Sun Mon TueSatWed
3000 -
300 -
Different workloads have different usage patterns

10. Sunday Monday Tuesday Wednesday Thursday Friday Saturday
Typical
weekly
traffic
to
Amazon.com
Provisioned capacity

11. November
traffic
to
Amazon.com
November

12. November
traffic
to
Amazon.com
Provisioned capacity
November

13. November
traffic
to
Amazon.com
76%
24%
Provisioned capacity
November

14. Actual
demand
Predicted
demand
Customer
dissa;sfac;on
Waste
Demand
Time
Elastic capacity No need to guess capacity requirements and over-provision
AWS enables companies to match resources to demand

15. Elastic capacity No need to guess capacity requirements and over-provision
Elas;c
capacity
Demand
Time
AWS enables companies to match costs to demand

16. November 10th 2010
Turned off last physical web server of
Amazon.com
October 31st 2011
Turned off last web servers supporting
European business

17. November
traffic
to
Amazon.com
November

18. NumberofEC2Instances
4/12/2008 4/14/2008 4/15/2008 4/16/2008 4/18/2008 4/19/2008 4/20/20084/17/20084/13/2008
40
servers
to
5000
in
3
days
EC2 scaled to peak of 5000
instances
“Techcrunched”
Launch of Facebook
modification
Steady state of ~40
instances

19. Automation is a key enabler to elastic usage
Bootstrapping or DEV-OPS
The process of automatically
configuring the software and settings
on your machines as they boot, each time
they boot.
Your infrastructure as code.

20. Amazon Route 53 Elastic Load Balancer
The
image
cannot
be
displaye
S3 BucketCloudFront Distribution
The image
cannot be
displayed. Your
computer may
not have enough
memory to open
the image, or
the image may
have been
Web Servers
The image
cannot be
displayed. Your
computer may
not have enough
memory to open
the image, or
the image may
have been
Web Servers
Web ASG Elastic Beanstalk
App
App
Master
Standby
RR 1
RR 2
RR 3
RR 4
ElastiCache Cluster
This is a
stack
In AWS everything can be Automated , everything is an API
Resources are not longer finite, they are elastic in AWS

21. Cloud
Forma=on
is
a
great
Cookie
Cu@er
Your infrastructure as code.

22. This is a STACK.
JavaScript Object Notation ( JSON )
A template of your datacenter / workload.
Your infrastructure as code.
Headers
Parameters
Mappings
Resources
Outputs
Git
Subversion
Mercurial
Dev
Test
Prod

23. Cloud
Forma=on
is
context
aware
Your infrastructure as code.
Create: PROD
dev.mysite.com test.mysite.com prod.mysite.com
Create: TESTCreate: DEV

24. Elastic resources requires Utility Pricing

25. Enabling customers to Optimize Costs based on Utilization

26. Meeting base workload, variable and peak with different pricing models

27. Architecting Tips for scaling to meet
Seasonal Patterns

28. Auto Scaling groups are useful for more than just fault tolerance
• Vertical Scaling
• Horizontal Scaling
• Auto Scaling
• Scheduled Scaling
• Programmatic Scaling
• Datasbse Tier Scaling
• Asynchronous Process Scaling
• Event Scaling
ASG == Minimum unit of deployment

29. myAutoScalingGroup
- myLaunchConfig
- Min 1
- max 1
- desired 1
Launch
Configuration
ami-0535d66c
ap-southeast2-a ap-southeast2-b
myElasticLoadBlancer
myLaunchConfig
- ami-0535d66g
- m3.large
Minimum instance of 1 creates Auto Healing Groups

30. Vertical Scaling (Scale UP)

31. Vertical Scaling using different instance types
0
0.5
1
1.5
2
2.5
3
3.5
1 3 5 7 9 11 13 15 17 19 21 23 25 27 29
DBInstanceType
Days of the Month
End of the Month Scaling
75% Savings

32. Small 1.7 GB,
1 ECU
1 virtual core
Large 7.5 GB
4 ECUs
2 virtual cores
Extra Large 15 GB
8 ECUs
4 virtual cores
Hi-Mem XL 17.1 GB
6.5 ECUs
2 virtual cores
Hi-Mem 2XL 34.2 GB
13 ECUs
4 virtual cores
Hi-Mem 4XL 68.4 GB
26 ECUs
8 virtual cores
High-CPU Med 1.7 GB
5 ECUs
2 virtual cores
High-CPU XL 7 GB
20 ECUs
8 virtual cores
Micro 613 MB
Up to 2 ECUs (for
short bursts)
Cluster GPU 4XL 22 GB
33.5 ECUs
8 Nehalem virtual cores
2 x NVIDIA Tesla “Fermi”
M2050 GPUs
Cluster Compute 4XL 23 GB
33.5 ECUs
8 Nehalem virtual cores
Cluster Compute 8XL 60.5 GB
88 ECUs
8 core 2 x Intel Xeon
Medium 3.75 GB
2 ECUs
1 virtual cores
Memory intensive
Cluster Compute
Processor Intensive
Average Applications
Minimal resources
Multiple Family Types, optimized for different uses

33. Multiple sizes of instance within a family type

34. Vertical Scaling using Launch Configurations
myAutoScalingGroup
- smallConfig
- Min 1
- Max 2
- desired 1
- TP: Oldest Instance
ami-0535d66c
ElasticIP (EIP) / Elastic NIC (ENI)
Launch Config A
smallConfig
- ami-0535d66g
- small
ap-southeast2-a
Launch Config B
bigConfig
- ami-0535d66g
- large
UPDATE
myAutoScalingGroup
- largeConfig
- Min 1
- Max 2
- Desired 2
- TP: Oldest Instance
Ver;cal
Scaling
UPDATE Desired = 1

35. Database Tier scaling is automated when using RDS
Push Button Scaling
UP - DOWN
Read Only Replica
IN- OUT
Snapshot & Restore
ON – OFF

36. Database Tier management is heavily automated using RDS
High Availability
Host Replacement
High Scalability
Asynchronous Replication

37. Horizontal Scaling (Scale OUT)

38. ap-southeast2-a ap-southeast2-b
Launch
Configuration
ami-0535d66c
myLaunchConfig
- ami-0535d66g
- m3.large
myAutoScalingGroup
- myLaunchConfig
- Min 2
- max 100
- Desired 2
elb-cname.amazonaws.com
ASG UPDATE
Desired = 4
Elastic Load Balancing (ELB) over multiple Availability Zones (AZs)
ASG UPDATE
Desired = 2

39. HOST
LEVEL
METRICS
AGGREGATE
LEVEL
METRICS
LOG
ANALYSIS
EXTERNAL
SITE
PERFORMANCE

40. Auto Scaling (Elastic Usage)

41. ap-southeast2-a ap-southeast2-b
Launch
Configuration
ami-0535d66c
myLaunchConfig
- ami-0535d66g
- m3.large
myAutoScalingGroup
- myLaunchConfig
- Min 2
- max 100
- Desired 2
Desired = 4
Auto Scaling using Policies to Scale Out
Scale UP +1
Scale DOWN -1

42. ap-southeast2-a ap-southeast2-b
Launch
Configuration
ami-0535d66c
myLaunchConfig
- ami-0535d66g
- m3.large
myAutoScalingGroup
- myLaunchConfig
- Min 2
- max 100
- Desired 2
API Update Desired = 4
Auto Scaling using API to Scale In / Out
Scale UP +1
Scale DOWN -1

43. AutoSclaingGroups*
- myLaunchConfig
- Min 0
- max 100
- Desired 0
Launch
Configuration
ami-0535d66c
ap-southeast2-a ap-southeast2-b
launchWhenCheap
- ami-0535d66g
- m3.large
- Spot-price : 0.05
Automate Workload Patterns using Scheduled Scaling
as-put-scheduled-update-group-action ScaleUp
--auto-scal`ing-group my-test-asg
--recurrence “30 0 1 1,6,12 0”
--desired-capacity 20
as-put-scheduled-update-group-action ScaleOff
--auto-scaling-group my-test-asg
--start-time "2013-05-13T08:00:00Z"
--desired-capacity 0

44. Auto Scaling with Alarms & Policies

45. Achieve High Utilization with this style of architecture, eliminating waste
Trigger
auto-­‐scaling
policy
Reserved Instances
On Demand
Spot Pricing
Scheduled
Adaptive
Predictive

46. Optimize delivery using S3 static hosting and CloudFront
London
Paris
NY
Served
from
S3
/images/*
3
Served
from
EC2
*.php
2
Single
CNAME
www.mysite.com
1
Lower Cost
Lower Latency
Higher Scale
Fault Tolerance
High Availability
High Utilization

47. Scaling Asynchronous Processing

48. Asynchronous Process Scaling with SQS Messaging
• Amazon managed queue service
• Decouple your components
• Think parallel
• Implement elasticity
• Drive Auto Scaling fleets using Queue Depth
Controller A Controller B Controller C
Controller A Controller B Controller C
Q Q Q
Tight Coupling
Loose Coupling using Queues
Amazon
SQS
Processing
task/
processing
trigger
Processing
results
Min 5 Min 10 Min 2

49. S3 Bucket
For Ingest
User
SNS
Topic
RRS S3
Bucket to
Serve
content to
CloudFron
t
S3 Bucket
For
originals
CloudFront
Download
Distribution
SQS Queue
Size for Thumbnail
SQS Queue
Size Image for
Mobile
SQS Queue
Size Image for Web
Auto scaling
Group
Instances
Auto scaling
Group
Instances
Auto scaling
Group
Instances
Asynchronous Process Scaling with SQS Messaging (SQS)

50. S3 Bucket
For Ingest
User
RRS S3
Bucket to
Serve
content to
CloudFront
S3 Bucket
For
originals
CloudFront
Download
Distribution
Auto scaling
Group
Instances
Auto scaling
Group
Instances
Auto scaling
Group
Instances
SWF
Instance running
decider
Asynchronous Process Scaling with Simple Workflow (SWF)

51. AutoSclaingGroups*
- myLaunchConfig
- Min 0
- max 100
- Desired 0
Launch
Configuration
ami-0535d66c
ap-southeast2-a ap-southeast2-b
launchWhenCheap
- ami-0535d66g
- m3.large
- Spot-price : 0.05
Optimize costs using Auto Bidding groups and spot pricing
aws autoscaling create-launch-configuration
--launch-configuration-name launchWhenCheap
--spot-price 0.05
SQS queue
Consumers
Producer
Consumers

52. Scale 1000s of nodes when needed a back to zero using EMR
Optionally using a Spot Pricing strategy on task nodes

53. Event Based Scaling

54. Parameterized Scaling via CloudFormation
myAutoScalingGroup
- myLaunchConfig
- Min 2
- max 100
- Desired inputParameter

55. Are you confident your N+1?
February, 2012

56. Automated failover using pilot light configurations
Web
Server
Application
Server
Database
Server
Data
Volume
Data Mirroring/
Replication
Not Running
Smaller Instance
Amazon Route 53
User or system
Web
Server
Application
Server
Database
Server
Data
Volume
UPDATE
Desired = 0 à 1
Desired = 0 à 1
Desired = 1 à 1
Web
Server
Application
Server

57. Just in Time systems which can be during an event

58. • ~30th biggest E-commerce operation, globally
• ~200 distinct applications, many mobile
• Hundreds of new, untested analytical approaches
• Processing hundreds of TB of data on thousands of servers
• Spikes of hundreds of thousands of concurrent users
• Critically compressed budget
• Less than a year to execute
• Core systems will be used for a single critical day
• Constitutionally-mandated completion date
Support Systems which can be retired immediately after an event

59. THANK YOU
Please give us your feedback by filling out the Feedback Forms
AWS Government, Education, &
Nonprofits Symposium
Canberra, Australia | May 20, 2014