Elastic Load Balancing provides a scalable and highly-available load balancer that automatically distributes incoming application traffic across multiple Amazon EC2 instances. It enables you to achieve even greater fault tolerance in your applications, seamlessly providing the amount of load balancing capacity needed in response to incoming application traffic. In this session, we take a deeper look at some of the existing and newer features that enable application developers to architect highly-available architectures that are resilient to load spikes and application failures. We also explore some of the features that allow seamless integration with services such as Auto Scaling and Amazon Route 53 to further improve the scalability and resilience of your applications.

1. Architecting for Availability & Scalability with
Elastic Load Balancing and Amazon Route 53
David Brown (Elastic Load Balancing)
Sean Meckley (Amazon Route 53)
Paul Kearney (InfoSpace)
November 15, 2013
© 2013 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified, or distributed in whole or in part without the express consent of Amazon.com, Inc.
Thursday, November 21, 13

2. welcome
!2
Thursday, November 21, 13

3. “Everything fails all the time”
Werner Vogels, CTO, Amazon.com
!2
Thursday, November 21, 13

4. Avoid single points of failure.
!4
Thursday, November 21, 13

5. Elastic Load Balancing and Amazon Route 53 are
critical components when building scalable and
highly-available applications.
!5
Thursday, November 21, 13

6. Load Balancer
Elastic
Secure
Integrated
Cost-Effective
[ What is Elastic Load Balancing? ]
!6
Thursday, November 21, 13

7. Availability Zone 1a
EC2 Instances
EC2 Instances
Elastic
Load Balancing
(Internal)
Elastic
Load Balancing
Client
EC2 Instances
EC2 Instances
Availability Zone 1b
[ What is Elastic Load Balancing? ]
!7
Thursday, November 21, 13

8. 3
Levels
of
Availability
!7
Thursday, November 21, 13

9. 1
Instance
Availability
!8
Thursday, November 21, 13

10. 1
2
Instance
Availability
Zonal
Availability
!8
Thursday, November 21, 13

11. 1
2
3
Instance
Availability
Zonal
Availability
Regional
Availability
!8
Thursday, November 21, 13

12. 1
Instance
Availability
!9
Thursday, November 21, 13

13. First step in increasing the availability
of a system or application.
!10
Thursday, November 21, 13

14. Client
EC2 Instance
Load balancer used to
route incoming
requests to multiple
EC2 instances
[ Instance Redundancy ]
!12
Thursday, November 21, 13

15. EC2 Instance
Elastic
Load Balancing
Client
EC2 Instance
Load balancer used to
route incoming
requests to multiple
EC2 instances
EC2 Instance
[ Instance Redundancy ]
!13
Thursday, November 21, 13

16. Incoming request load shared
by all instances behind the load balancer.
!13
Thursday, November 21, 13

17. EC2 Instance
Elastic
Load Balancing
Client
EC2 Instance
Leastconns used to spread
request across healthy
instances
EC2 Instance
[ Request Routing ]
!15
Thursday, November 21, 13

18. EQUAL UTILIZATION
ON EACH INSTANCE
EC2 Instance
Elastic
Load Balancing
Client
EC2 Instance
Leastconns used to spread
request across healthy
instances
EC2 Instance
[ Request Routing ]
!15
Thursday, November 21, 13

19. TARGETS INSTANCES WITH
EQUAL UTILIZATION
ON EACH INSTANCE
FEWEST OUTSTANDING REQUESTS
EC2 Instance
Elastic
Load Balancing
Client
EC2 Instance
Leastconns used to spread
request across healthy
instances
ADJUSTS TO REQUEST
SMOOTHS REQUEST LOAD
RESPONSE TIMES
ACROSS ALL INSTANCES
EC2 Instance
[ Request Routing ]
!15
Thursday, November 21, 13

20. Instances that fail can be replaced seamlessly
while other instances continue to operate.
!15
Thursday, November 21, 13

21. EC2 Instance
Elastic
Load Balancing
Client
EC2 Instance
Application level
health checks ensure that
request traffic is shifted
away from a failed instance
EC2 Instance
[ Health Checks ]
!17
Thursday, November 21, 13

22. FAILURE DETECTED
X
EC2 Instance
Elastic
Load Balancing
Client
EC2 Instance
Application level
health checks ensure that
request traffic is shifted
away from a failed instance
EC2 Instance
[ Health Checks ]
!17
Thursday, November 21, 13

23. TRAFFIC SHIFTED
FAILURE DETECTED
X
X
EC2 Instance
Elastic
Load Balancing
Client
EC2 Instance
Application level
health checks ensure that
request traffic is shifted
away from a failed instance
EC2 Instance
[ Health Checks ]
!17
Thursday, November 21, 13

24. TRAFFIC SHIFTED
FAILURE DETECTED
X
X
EC2 Instance
Elastic
Load Balancing
Client
EC2 Instance
Application level
health checks ensure that
request traffic is shifted
away from a failed instance
EC2 Instance
HEALTHY INSTANCES CARRY
ADDITIONAL REQUEST LOAD
[ Health Checks ]
!17
Thursday, November 21, 13

25. TRAFFIC SHIFTED
FAILURE DETECTED
USED TO DETERMINE THE HEALTH OF
THE INSTANCE X
AND APPLICATION
X
EC2 Instance
Elastic
Load Balancing
Client
EC2 Instance
TCP AND HTTP
Application level
health checks ensure that
request traffic is shifted
away from a failed instance
CONSIDER THE DEPTH AND
ACCURACY OF YOUR
EC2 Instance
HEALTH CHECKS
[ Health Checks ]
CUSTOMIZE FREQUENCY
AND FAILURE THRESHOLDS
HEALTHY INSTANCES CARRY
ADDITIONAL REQUEST LOAD
503 ERRORS RETURNED IF
NO HEALTHY INSTANCES
!17
Thursday, November 21, 13

26. Auto Scaling can be used to automatically adjust
instance capacity up or down depending on
conditions you define.
!18
Thursday, November 21, 13

27. Elastic
Load Balancing
EC2 Instance
EC2 Instance
EC2 Instance
[ ELB & Auto Scaling ]
!19
Thursday, November 21, 13

28. Elastic
Load Balancing
EC2 Instance
EC2 Instance
EC2 Instance
[ ELB & Auto Scaling ]
LOAD INCREASES
!19
Thursday, November 21, 13

29. Elastic
Load Balancing
INSTANCES ADDED FOR
INCREASED LOAD
EC2 Instance
EC2 Instance
EC2 Instance
EC2 Instance
EC2 Instance
[ ELB & Auto Scaling ]
!19
Thursday, November 21, 13

30. Elastic
Load Balancing
EC2 Instance
LOAD DECREASES
EC2 Instance
EC2 Instance
EC2 Instance
EC2 Instance
[ ELB & Auto Scaling ]
!19
Thursday, November 21, 13

31. Elastic
Load Balancing
EC2 Instance
EC2 Instance
INSTANCES REMOVED
AS LOAD DECREASES
EC2 Instance
[ ELB & Auto Scaling ]
!19
Thursday, November 21, 13

32. Elastic
Load Balancing
INSTANCES REMOVED
AS LOAD DECREASES
AUTOMATICALLY SCALES
INSTANCES UP OR DOWN
EC2 Instance
AUTOMATICALLY REPLACES
EC2 Instance
CUSTOM SCALING METRICS
EC2 Instance
REDUCES COSTS
FAILED INSTANCES
[ ELB & Auto Scaling ]
!19
Thursday, November 21, 13

33. 2
Zonal
Availability
!19
Thursday, November 21, 13

34. Availability Zones are distinct geographical
locations that are engineered to be insulated from
failures in other zones.
!20
Thursday, November 21, 13

35. Region
Availability Zone
!21
Thursday, November 21, 13

36. It is important to run application
stacks in more than one zone.
!22
Thursday, November 21, 13

37. Avoid unnecessary dependencies
between zones.
!23
Thursday, November 21, 13

38. Zone 1a
EC2 Instances
Load balancer used to
balance across instances in
multiple Availability Zones.
Elastic
Load Balancing
Client
EC2 Instances
Zone 1b
[ Availability Zone Redundancy ]
!25
Thursday, November 21, 13

39. Each load balancer will contain one or more
DNS records, one for each load balancer node.
!25
Thursday, November 21, 13

40. Client
Elastic
Load Balancing
192.0.2.1
EC2 Instance
EC2 Instance
EC2 Instance
192.0.2.2
EC2 Instance
EC2 Instance
EC2 Instance
[ Understanding DNS ]
!27
Thursday, November 21, 13

41. Client
Elastic
Load Balancing
192.0.2.1
192.0.2.2
DNS ROUND ROBIN USED TO
EXPECT DNS RECORDS
BALANCE TRAFFIC BETWEEN
AVAILABILITY ZONES
EC2 Instance
EC2 Instance
EC2 Instance
TO CHANGE OVER TIME
EC2 Instance
EC2 Instance
EC2 Instance
[ Understanding DNS ]
EACH LOAD BALANCER DOMAIN NAME
MAY CONTAIN MULTIPLE A RECORDS
!27
Thursday, November 21, 13

42. Using multiple Availability Zones does
bring a few challenges.
!27
Thursday, November 21, 13

43. requests / minute
Availability Zones may
see traffic imbalances
due to clients caching
DNS records.
time
[ Multiple Zone Challenges ]
!28
Thursday, November 21, 13

44. 2
Zone 1a
An unequal number of
instances per zone can
lead to over utilization of
instances in a zone.
EC2 Instances
Elastic
Load Balancer
Client
3
EC2 Instances
Zone 1b
[ Multiple Zone Challenges ]
!30
Thursday, November 21, 13

45. Problem solved.
!30
Thursday, November 21, 13

46. Cross-Zone Load Balancing distributes
traffic across all healthy instances,
regardless of Availability Zone.
!31
Thursday, November 21, 13

47. Zone 1a
2
Effectively balances the
request load across all
instances behind the load
balancer.
EC2 Instances
Elastic
Load Balancing
Client
3
EC2 Instances
Zone 1b
[ Cross-Zone Load Balancing ]
!33
Thursday, November 21, 13

48. requests / minute
Traffic is spread evenly
across each of the active
Availability Zones.
time
[ Cross-Zone Load Balancing ]
!33
Thursday, November 21, 13

49. requests / minute
Availability Zones may
ELIMINATES IMBALANCES IN
NO BANDWIDTH CHARGE
FOR CROSS-ZONE TRAFFIC
REQUESTS DISTRIBUTED EQUALLY TO
ALL INSTANCES REGARDLESS OF ZONE
see UTILIZATION
INSTANCE traffic imbalances
due to clients caching
DNS records.
REDUCES IMPACT OF CLIENTS
CACHING DNS RECORDS
time
[ Cross-Zone Load Balancing ]
!33
Thursday, November 21, 13

50. 3
Regional
Redundancy
!35
Thursday, November 21, 13

51. Elastic Load Balancing and Amazon Route 53 have
been integrated to support a single application
across multiple regions.
!36
Thursday, November 21, 13

52. Region
Availability Zone
!37
Thursday, November 21, 13

53. ROUTE
•
53
AWS’s authoritative Domain Name Service (DNS)
•
Health checking service
•
Highly available and scalable
•
Offers tools that provide flexible, high-performance, and
highly available architectures on AWS
[ What is Amazon Route 53? ]
!39
Thursday, November 21, 13

54. Improves availability by …
•
health checking load balancer nodes and rerouting
traffic to avoid failures
•
supporting multi-region and backup architectures for
high-availability
ROUTE
53
[ What is Amazon Route 53? ]
!40
Thursday, November 21, 13

55. Health Checks
Automated requests sent over the
Internet to your application to verify
that your application is reachable,
available, and functional.
+
Failover
Only returns answers for resources
that are healthy and reachable from
the outside world, so end users are
routed away a failed application.
[ What is DNS failover? ]
!40
Thursday, November 21, 13

56. Work on Failure
System activity
Time to react
Constant Work
System activity
Time to react
time
time
When nothing is failing, volume of API
Health checkers and edge locations
calls is zero. When failure occurs,
perform the same volume of activity
volume of API calls spikes.
whether endpoints are healthy or
unhealthy.
[ How does it work? ]
!41
Thursday, November 21, 13

57. Amazon Route 53
conducts health checks
from within each AWS
region
[ Global Health Check Network ]
!43
Thursday, November 21, 13

58. NETWORK PARTITION
!43
Thursday, November 21, 13

59. 150
SECONDS
MANUAL FAILOVER
vs.
• operator receives an alarm
• operator manually
configures DNS update
• wait for DNS changes to
propagate
[ How does it work? ]
!44
Thursday, November 21, 13

60. 150
SECONDS
NO CONTROL PLANE INVOLVEMENT
REQUIRED FOR FAILOVER TO OCCUR
MANUAL FAILOVER
• operator receives an alarm
• operator manually
DIRECTLY FROM GLOBALLY DISTRIBUTED
configures DNS
HEALTH CHECKER FLEET update
• wait for DNS changes to
propagate
EDGE LOCATIONS PULL HEALTH RESULTS
vs.
DON’T HAVE TO WAIT FOR API REQUESTS
TO SUCCEED AND THEN PROPAGATE
[ How does it work? ]
FAILOVER HAPPENS ENTIRELY WITHIN
THE AMAZON ROUTE 53 DATA PLANE
!44
Thursday, November 21, 13

61. •
•
Region
E-commerce site: example.com
Elastic
Load Balancing
Running application stack in multiple Availability
Zones in a single AWS region
•
Wants a backup in case:
-
Own application goes down across multiple
Availability Zones
-
Some parts of the world experience
degraded connectivity to this AWS region
EC2 Instances
EC2 Instances
[ Simple Failover Scenario ]
!46
Thursday, November 21, 13

62. Region
Primary
Elastic
Load Balancing
EC2 Instances
Health
Check
ROUTE
Secondary
53
S3
EC2 Instances
[ Simple Failover Scenario ]
!47
Thursday, November 21, 13

63. Region
X
Elastic
Load Balancing
X
Primary
Health
Check
ROUTE
Secondary
53
S3
FAILOVER
HEALTH CHECK FAILS
EC2 Instances
EC2 Instances
[ Simple Failover Scenario ]
!48
Thursday, November 21, 13

64. Static Site
Static vs. dynamic content
[ Static Backup Site Options ]
!48
Thursday, November 21, 13

65. •
Provides your globally-distributed end users
with faster performance
•
Tag each destination end-point to the
Amazon EC2 region that it’s located in
•
Amazon Route 53 will route end users to the
end-point that provides the lowest latency
[ Latency Based Routing ]
!50
Thursday, November 21, 13

66. •
Better performance than running in a single region
•
Improved reliability relative to running in one region
•
Easier implementation than traditional DNS solutions
•
Much lower prices than traditional DNS solutions
[ LBR Benefits ]
“Our customers bid on video ad
inventory in real time and our system
must evaluate the content they're
sponsoring and respond with a
decision in less than 50ms, or they'll
lose the auction. Route 53’s Latency
Based Routing lets us easily run
multiple stacks of our whole targeting
platform in each AWS region so we can
meet our customers latency needs.”
Jonathan Dodson,
Vice President of Engineering at Affine
!50
Thursday, November 21, 13

67. •
Region 1
example.com wants faster
page load for customers
•
Region 2
Launches application stack in
Elastic
Load Balancing
Elastic
Load Balancing
additional AWS regions
•
Uses Amazon Route 53
Latency Based Routing
•
Amazon Route 53 DNS
Failover ensures that end
users are only routed to a
region where the application is
EC2 Instances
EC2 Instances
EC2 Instances
EC2 Instances
healthy
[ Multi-Region Failover ]
!52
Thursday, November 21, 13

68. Region 1
Region 2
Primary
Elastic
Load Balancing
EC2 Instances
Health
Check
ROUTE
53
Primary
Health
Check
EC2 Instances
Elastic
Load Balancing
EC2 Instances
EC2 Instances
[ Multi-Region Failover ]
!53
Thursday, November 21, 13

69. Region 1
Region 2
Primary
Elastic
Load Balancing
Health
Check
ROUTE
53
Primary
Health
Check
X
X
Elastic
Load Balancing
HEALTH CHECK FAILS AND
TRAFFIC SHIFTS AWAY
EC2 Instances
EC2 Instances
EC2 Instances
EC2 Instances
[ Multi-Region Failover ]
!54
Thursday, November 21, 13

70. Region 1
Elastic
Load Balancing
EC2 Instances
Region 2
Elastic
Load Balancing
EC2 Instances
EC2 Instances
S3
EC2 Instances
[ Multi-Region & S3 Failover ]
!55
Thursday, November 21, 13

71. [ Configuring DNS Failover ]
!56
Thursday, November 21, 13

72. AWS & InfoSpace
Elastic Load Balancing & Amazon Route 53 for High-Availability
!57
Thursday, November 21, 13

73. InfoSpace Search
Since 1996, our mission has been to make
it fast and easy for users to find what they
need online.
!57
Thursday, November 21, 13

74. InfoSpace Search
!58
Thursday, November 21, 13

75. InfoSpace Search
Search Sites
!58
Thursday, November 21, 13

76. InfoSpace Search
Search Sites
Search API
!58
Thursday, November 21, 13

77. Types of Users
!59
Thursday, November 21, 13

78. Types of Users
Search Site
Users
• 400 million queries per
month
• Broad geographical
distribution
!59
Thursday, November 21, 13

79. Types of Users
Search Site
Users
• 400 million queries per
month
• Broad geographical
distribution
Search API
Partners
• 150+ partners
worldwide
• Located primarily in
US and EU
• 2 billion queries/month
!59
Thursday, November 21, 13

80. Types of Users
Search Site
Users
• 400 million queries per
month
• Broad geographical
distribution
Search API
Partners
• 150+ partners
worldwide
• Located primarily in
US and EU
• 2 billion queries/month
Click Users
• 6.5 billion clicks/month
• Broad geographical
distribution
!59
Thursday, November 21, 13

81. Global Distribution of Traffic
!60
Thursday, November 21, 13

82. Global Distribution of Traffic
!60
Thursday, November 21, 13

83. Global Distribution of Traffic
AZ#
AZ#
AZ#
AZ#
AZ#
AZ#
AZ#
AZ#
AZ#
!60
Thursday, November 21, 13

84. Global Distribution of Traffic
AZ#
AZ#
AZ#
AZ#
AZ#
AZ#
AZ#
AZ#
AZ#
!60
Thursday, November 21, 13

85. Global Distribution of Traffic
AZ#
AZ#
AZ#
AZ#
AZ#
AZ#
AZ#
AZ#
AZ#
!60
Thursday, November 21, 13

86. Global Distribution of Traffic
AZ#
AZ#
AZ#
AZ#
AZ#
AZ#
AZ#
AZ#
AZ#
!60
Thursday, November 21, 13

87. Global Distribution of Traffic
AZ#
AZ#
AZ#
AZ#
AZ#
AZ#
AZ#
AZ#
AZ#
!60
Thursday, November 21, 13

88. Global Distribution of Traffic
AZ#
AZ#
AZ#
AZ#
AZ#
AZ#
AZ#
AZ#
AZ#
!60
Thursday, November 21, 13

89. Key Statistics
• 4.5 billion requests/month
• Migrated from 2 data centers to AWS in 5 months
• Deployed in 4 regions
• Approximately 500 EC2 instances
• Approximately 50 load balancers
• Approximately 70 Amazon Route 53 zones
!62
Thursday, November 21, 13

90. AWS Infrastructure
Route$53$
Private$Subnet$
Public$Subnet$
NAT$
TSG$
Suppor+ng$
Services$
Search$
API$
Search$
Sites$
Outbound$via$NAT$
Suppor+ng$Services$
!62
Thursday, November 21, 13

91. Fire and Forget
!63
Thursday, November 21, 13

92. Fire and Forget
Production
System under test
!63
Thursday, November 21, 13

93. Fire and Forget
Production
System under test
!63
Thursday, November 21, 13

94. Fire and Forget
Asynchronous
Production
System under test
!63
Thursday, November 21, 13

95. Fire and Forget
Production
System under test
!63
Thursday, November 21, 13

96. Fire and Forget
Production
System under test
!63
Thursday, November 21, 13

97. Fire and Forget
!64
Thursday, November 21, 13

98. Fire and Forget
!64
Thursday, November 21, 13

99. Fire and Forget
!64
Thursday, November 21, 13

100. Fire and Forget
LBR
LBR
!64
Thursday, November 21, 13

101. Fire and Forget
LBR
LBR
!64
Thursday, November 21, 13

102. Fire and Forget
LBR
!64
Thursday, November 21, 13

103. Results
• Regional failover in 150 seconds consistently
• Decreased latency – 25% less latent worldwide
• Can easily reroute individual partners to different region to avoid routing
problems
• Replaced expensive network gear from datacenter
!65
Thursday, November 21, 13

104. What next?
• Expanding to additional regions
• Integration of monitoring data with traffic routing
!66
Thursday, November 21, 13

105. 3
Levels
of
Availability
!67
Thursday, November 21, 13

106. 1
Instance
Availability
!68
Thursday, November 21, 13

107. 1
2
Instance
Availability
Zonal
Availability
!68
Thursday, November 21, 13

108. 1
2
3
Instance
Availability
Zonal
Availability
Regional
Availability
!68
Thursday, November 21, 13

109. Please give us your feedback on this
presentation
CPN104
As a thank you, we will select prize
winners daily for completed surveys!
Thursday, November 21, 13
Thank You