This document discusses concepts related to scalability, availability, and autoscaling in cloud computing. It defines key terms like scalability, availability, and high availability. It describes techniques for vertical and horizontal scaling as well as load balancing. The document discusses capacity planning and choosing the required scale based on factors like throughput and cost of downtime. It also covers autoscaling and how systems can dynamically scale based on load. Finally, it compares different architecture patterns for availability from single node to multi-region deployments.

1. Last Updated: May. 2014
Committer and PMC member of Apache Stratos
Senior Software Engineer, WSO2
Lahiru Sandaruwan
Cloud Concepts

2. *
Agenda
• Some core concepts
• Vertical/horizontal scaling techniques
• Capacity planning
• SLA Awareness
• Autoscaling
• Fault tolerance
• Q&A, Discussion

3. *
Scalability
"The ability of the of a system to
continue to operate correctly even
when it is scaled to a larger size”

4. *
Availability

5. *
Availability

6. *
High Availability
A system that is designed for continuous operation
in the event of a failure of one or more
components. However, the system may display
some degradation of service, but will continue to
perform correctly.
High Availability: The proportion of time during
which the service is accessible with reasonable
response times should be close to 100%.

7. *
How to decide required scale
(capacity) & availability?
• Average throughput (TPS)
• Max throughput (TPS)
• Monetary value of a transaction
• Average loss & max loss per second of
downtime
• Decide on how much to invest based on cost vs.
benefit tradeoff

8. *
Vertical Scaling
• Get the maximum out of each allocated JVM or
resource
• Increase CPU size
• Increase memory

9. *
Horizontal Scaling

10. *
Load Balancing
• Load balancing algorithms
• Round robin
• Weighted
• Response based
• Health check
• Failover-only

11. *
Clustering for scalability

12. *
Clustering for availability
Group Communication Channel/State replication

13. *
Capacity Planning
Source: http://srinathsview.blogspot.com/2012/05/how-to-measure-performance-of-
server.html

14. *
Capacity Planning
Throughput = number of completed requests / time to complete the requests
No. of servers = (projected max load * 1.3) / max throughput of one server
max throughput of one server = max throughput of that server for the slowest
scenario in the set of use cases

15. *
Static membership
• Predefined members
• Other (non-predefined) nodes cannot join
M2
M1
M3
M4
Static group
N

16. *
Dynamic membership
• No predefined members
• Nodes can join & leave
M2
M1
M3
M4
Dynamic group
N
Join

17. *
Hybrid membership
• Some predefined (well-known) members, and some
dynamic members
• Nodes can join & leave
• Membership revolves around the static members
Hybrid group
N
M2
M1
M3 M4
Static members
M6M5
M7
Dynamic members
Join (IP,
Port)

18. *
Multicast based membership
management
N
M2
M1
M3
M4
Join (IP,
Port)

19. *
Well-known Address (WKA) based
membership management
Hybrid group
N
WK
2
WK
1
WK
3 WK
4
Static members
M6
M5
M
7
Dynamic members
Join (IP,
Port)
Notify

20. *
Multicast vs. WKA
Multicast WKA
All nodes should be in the same subnet Nodes can be in different networks
All nodes should be in the same multicast domain
No multicasting requirement
Multicasting should not be blocked
No fixed IP addresses or hosts required At least one well-known IP address or host
required
Failure of any member does not affect
membership discovery
New members can join with some WKA nodes
down, but not if all WKA nodes are down
Does not work on IaaSs such as Amazon EC2 IaaS-friendly
Requires keepalived, elastic IPs or some other
mechanism for remapping IP addresses of WK
members in cases of failure

21. *
● Solves performance, availability, and
economic costs issues
● Currently the customer needs a considerable
capacity planning
● Reduce cost of cloud: reduce economic cost
and energy footprint
SLA Awareness

22. *
● Monitoring the cloud parameters
● Trigger an event if a SLA violation happened
● Acting upon the events
SLA Awareness: Solution

23. *
Auto-scaling
• Scale-out when load increases
• Scale-in when load decreases
• Always use the optimum amount of resources
• Try out
• AWS ELB
• Apache Stratos Load Balancer
• WSO2 ELB

24. *
Auto-scaling – steady load

25. *
Auto-scaling – load increasing

26. *
Auto-scaling – load increasing

27. *
Auto-scaling – steady load

28. *
Auto-scaling – decreased load

29. *
Auto-scaling – decreased load

30. *
Auto-scaling – steady load

31. *
Autoscaling - Analysis & Results

32. *
Autoscaling - Analysis & Results

33. *
Single nodeAvailability
Cost
LOWEST
HIGHEST

34. *
Primary-secondary
Primary Secondary
Availability
Cost
LOWEST
HIGHEST

35. *
Primary-secondary, multiple LB
Primary Secondary
keepalived
Availability
Cost
LOWEST
HIGHEST

36. *
Active cluster, multiple LB
Active Active
keepalived
Active
Availability
Cost
LOWEST
HIGHEST

37. *
Multi-zone or multi-datacenter
Deployment
Region X
Zone 1
Zone 2
Cloud
Controller
Availability
Cost
LOWEST
HIGHEST

38. *
Multi-region deployment
Region X
Zone 1
Zone 2
Region Y
Zone 1
Zone 2
Availability
Cost
LOWEST
HIGHEST

39. *
Multiple IaaS (hybrid) DeploymentAvailability
Cost
LOWEST
HIGHEST
Private cloud (data center)
Zone 1
Zone 2
Amazon EC2
Zone 1
Zone 2
Rackspace Cloud
Zone 1
Zone 2

40. *
Single
Node
Primary-Secondary,single
LB
Multi-nodeactivecluster
-Singlezone
Multi-region
Primary-
Secondary,
withmultipleLBs
Multi-zone
Multi-IaaS
Cost of Availability

41. *
Stratos Architecture

42. *
References
๏ http://www.cari-info.org/cari-2012/session%201/1B3.pdf
๏ http://domino.research.ibm.com/library/cyberdig.
nsf/papers/16D9BBF221540A0785257784004FC33D
๏ http://ieeexplore.ieee.org/xpl/login.jsp?
tp=&arnumber=5452504&url=http%3A%2F%2Fieeexplore.ieee.
org%2Fxpls%2Fabs_all.jsp%3Farnumber%3D5452504
๏ http://ieeexplore.ieee.org/xpl/login.jsp?
tp=&arnumber=5990687&url=http%3A%2F%2Fieeexplore.ieee.
org%2Fxpls%2Fabs_all.jsp%3Farnumber%3D5990687
๏ http://www.slideshare.net/afkham_azeez/java-colombo-developing-
highly-scalable-apps
- Azeez’s talk at Java Colombo

43. *
DISCUSSION

44. *
Thanks!