Successfully reported this slideshow.
Scalable Web Design – Principles and
Patterns
Speaker : Sachin Prakash Sancheti
Principal Architect – Cloud (Windows Azure...
Context
2
3
Server
Busy!!
Your Request can
not be processed,
please try after
some time
I am trying
to book a
ticket for 1
hour now ...
Any Real life Examples?
4
Survey
What is Scalability?
• It is NOT
– Only Performance
– High Availability
– Business Continuity Planning
• It Is
– Traffic, ...
What is Scalability?
• Scalability
– “The Scalability is measure of number of users it can effectively
support at the same...
What is the Concern?
• Scalability is a business concern
– Google observed 500-milisecond delay to page response caused 20...
Handling Scalability – Degraded Application
• Degraded Application
– Doing nothing  and loosing business
8
Handling Scalability - Throttling
• Throttling
– Throttling the requests to temporarily stop accepting new requests
and se...
Handling Scalability – Adding Resources
• Adding Resources
– Scaling up – Vertical Scaling
• Get Bigger
• Widening the roa...
Typical Web Application Resources
• Web Server, Application Server (Middle Tier) and Database
Tier
11
Web
Server
Database
...
Scaling Solutions
• Vertical Scaling OR Scaling Up
– Increasing resource power
– Remember widening the roads!!
• Horizonta...
Vertical Vs. Horizontal Scaling
13
Vertical Scaling Horizontal Scaling
Higher Capital Investment On Demand Investment
Util...
Web/Application Server Scalability
14
Scaling Out Web Server – Load Balancing
15
Web
Server
Web
Server
Web
Server
• Design for Fault Tolerance
– Intent : Enable...
Pattern - Bi-directional Scaling
• Design for Scaling Out (Bidirectional)
– Intent: Deployment built using commodity of ha...
Scaling Out / Horizontally: Adding Removing Boxes
Design Principle - Stateless Design
• Stateless designs increases scalability
– Don’t store anything locally on Web Server...
Design Principle – Loosely Coupled
• Components and layers should be loosely coupled to be able to scale each
layer separa...
Caching in Scalability
• Caching helps in avoiding scale
• In-memory distributed cache offers an excellent solution to
dat...
Design Pattern - Cache Aside Pattern
• Prefer Cache to Database for
Reading
– Intent : Increase read throughput and
reduce...
Design Pattern - Cache Read-through/Write-through (RT/WT)
• Prefer Cache to Database
– Intent: Increase read throughput an...
23
Design Pattern - Cache Read-through/Write-through (RT/WT)
Database Scalability
24
CAP Theorem
• CAP theorem, also known as Brewer's theorem, states that
it is impossible for a distributed computer system ...
CAP Theorem – Database Placements
26
Database Scaling – Replication - Read Mostly Pattern
• Intent: Increase database scalability by separating write and
read ...
Database Scaling – Read Write Separation
28
Reads
and
Writes
Reads
Design Pattern – Partitioning / Sharding
• Design for Database Sharding
– Intent: Increasing data size might rise throttli...
Shard Resolver
Shard = User ID % 4
Database Sharding Example
30
Shard 0
25%
Shard 1
25%
Shard 2
25%
User ID=3
Shard 3
25%
Design Principles – Eventually Consistent
• BASE Opposite to ACID
– Intent: Real internet scale model. Postpone the consis...
Design Principles – Asynchronous Processing
• Blocking is bane for Scalability
– Intent:
• Avoid blocking calls, reduce co...
Design Principles – Parallel Design
• Design for Parallel and Reliable Work
– Intent: Increasing resources should results ...
Queue Based Pattern
34
Queue - Load Leveling, Load Balancing, Loose Coupling
35
Design Principles – Queue Based Pattern
• Idempotent
– Design the operation to be idempotent; that is, if it's carried out...
Design Principles – Capacity Planning
• Everything has a limit: Compose a Scale
– Intent: Design Around Provider SLAs and ...
Design Pattern – Multi Site Deployment Pattern
38
Database
Server
Web Servers
Application
Servers
Database
Server
Web Serv...
Summary
39
Scalability Principles
40
Scalability
Stateless
Parallelization
Asynchronous
Partitioning
Idempotent
Fault Tolerance
Vertical Vs. Horizontal Scaling
41
Vertical Scaling Horizontal Scaling
ACID BASE
Availability First Focus on Commit
Pessim...
Thank You !
42
43
44
Some of the images are taken by utilizing Google
search and due credit to the source.
Author do not claim any creation ...
Nächste SlideShare
Wird geladen in …5
×

Scalability Design Principles - Internal Session

19.809 Aufrufe

Veröffentlicht am

Took a session around scalability design principles.
Important to understand the scalability is, what it is not, example of traffic.

Veröffentlicht in: Design, Technologie

Scalability Design Principles - Internal Session

  1. 1. Scalable Web Design – Principles and Patterns Speaker : Sachin Prakash Sancheti Principal Architect – Cloud (Windows Azure) 1
  2. 2. Context 2
  3. 3. 3 Server Busy!! Your Request can not be processed, please try after some time I am trying to book a ticket for 1 hour now  Please wait!
  4. 4. Any Real life Examples? 4 Survey
  5. 5. What is Scalability? • It is NOT – Only Performance – High Availability – Business Continuity Planning • It Is – Traffic, User Growth – Dataset, Database Size Growth 5
  6. 6. What is Scalability? • Scalability – “The Scalability is measure of number of users it can effectively support at the same time without degrading the defined performance” – Has limits – E.g. “With two load balanced capacity it should support 1000 concurrent users with average response time of 3 seconds” • “Performance is what an individual user experiences; Scalability is how many users get to experience it TOGETHER” 6
  7. 7. What is the Concern? • Scalability is a business concern – Google observed 500-milisecond delay to page response caused 20% decrease in traffic – Amazon.com observed 100-milisecond delay caused a 1% decrease in retail revenue – Remember “Performance is what an individual user experiences; Scalability is how many users get to experience it TOGETHER” 7
  8. 8. Handling Scalability – Degraded Application • Degraded Application – Doing nothing  and loosing business 8
  9. 9. Handling Scalability - Throttling • Throttling – Throttling the requests to temporarily stop accepting new requests and serve better to existing or important users 9
  10. 10. Handling Scalability – Adding Resources • Adding Resources – Scaling up – Vertical Scaling • Get Bigger • Widening the roads – Scaling out – Horizontal Scaling • Get More • Routing the traffic (Partitioning) 10
  11. 11. Typical Web Application Resources • Web Server, Application Server (Middle Tier) and Database Tier 11 Web Server Database Server Application Server
  12. 12. Scaling Solutions • Vertical Scaling OR Scaling Up – Increasing resource power – Remember widening the roads!! • Horizontal Scaling OR Scaling Out – Adding additional machines/nodes – Remember routing the traffic 12
  13. 13. Vertical Vs. Horizontal Scaling 13 Vertical Scaling Horizontal Scaling Higher Capital Investment On Demand Investment Utilization concerns Utilization can be optimized Relatively Quicker and works with the current design Relatively more time consuming and needs redesigning Limiting Scale Internet Scale Not Cloud Native Design Cloud Native Design
  14. 14. Web/Application Server Scalability 14
  15. 15. Scaling Out Web Server – Load Balancing 15 Web Server Web Server Web Server • Design for Fault Tolerance – Intent : Enables system to continue its intended operation, possibly at a reduced level, rather than failing completely, when some part of the system fails – Drivers: Degraded services are better than no service at all. Compare cost effectiveness – Solution: • Load Balancing • Monitoring, Self Healing, Restart
  16. 16. Pattern - Bi-directional Scaling • Design for Scaling Out (Bidirectional) – Intent: Deployment built using commodity of hardware working together for economies of scale. Optimization is easier with scaling out and in, rather than scaling up and down. Driven for Elasticity – Driver: Optimized utilization, cost saving – Solution: • Stateless Application Design • Nothing is shared except Database • Scaling every tier is possible – Web/Service/Database etc. 16
  17. 17. Scaling Out / Horizontally: Adding Removing Boxes
  18. 18. Design Principle - Stateless Design • Stateless designs increases scalability – Don’t store anything locally on Web Server • Session State – Local Sessions – Avoid – Not Scalable • Load Balancer Sticky sessions can create hot spot load – Central Session – Good – Distributed Cache, Database – Client Session – Better – Client Cookie – No Session – Awesome 18
  19. 19. Design Principle – Loosely Coupled • Components and layers should be loosely coupled to be able to scale each layer separately 19 Database Server Web Servers Application Servers
  20. 20. Caching in Scalability • Caching helps in avoiding scale • In-memory distributed cache offers an excellent solution to data storage bottlenecks • Distributed caching clusters can keep growing horizontally, just like the application servers. This reduces pressure on data storage so that it is no longer a scalability bottleneck. 20
  21. 21. Design Pattern - Cache Aside Pattern • Prefer Cache to Database for Reading – Intent : Increase read throughput and reduce database bottleneck – Drivers: Distributed cache are faster and shared across web/application servers – Solution: • Update cache and database both for synchronization • Read from Cache • Decorator Design Pattern 21 Distributed Cache Write Read
  22. 22. Design Pattern - Cache Read-through/Write-through (RT/WT) • Prefer Cache to Database – Intent: Increase read throughput and reduce database bottleneck. Use Cache for read write both – Drivers: Distributed cache are faster and shared across web/application servers – Solution: • Application treats cache as the main data store and reads data from it and writes data to it. • The cache is responsible for reading and writing this data to the database, thereby relieving the application of this responsibility, asynchronously 22
  23. 23. 23 Design Pattern - Cache Read-through/Write-through (RT/WT)
  24. 24. Database Scalability 24
  25. 25. CAP Theorem • CAP theorem, also known as Brewer's theorem, states that it is impossible for a distributed computer system to simultaneously provide all three of the following guarantees: Consistency, Availability and Partition tolerance. • Consistency: All clients always have the same view of the data • Availability: Each client can always read and write • Partition Tolerance: The system works well despite physical network partition 25
  26. 26. CAP Theorem – Database Placements 26
  27. 27. Database Scaling – Replication - Read Mostly Pattern • Intent: Increase database scalability by separating write and read operations – Generally most of the applications have around 80% read and 20% write • Drivers: Separate read write responsibilities, High availability benefits • Solution: – Read Write Separation – Master Slave Pattern 27
  28. 28. Database Scaling – Read Write Separation 28 Reads and Writes Reads
  29. 29. Design Pattern – Partitioning / Sharding • Design for Database Sharding – Intent: Increasing data size might rise throttling. Database scale and performance is more important than reliability. CAP Theorem – Drivers: Scaling database layer, increasing database throughput – Solution: • Database Sharding / Horizontal Partitioning • Database Federation 29
  30. 30. Shard Resolver Shard = User ID % 4 Database Sharding Example 30 Shard 0 25% Shard 1 25% Shard 2 25% User ID=3 Shard 3 25%
  31. 31. Design Principles – Eventually Consistent • BASE Opposite to ACID – Intent: Real internet scale model. Postpone the consistency. • Basically Available, Soft state, Eventual consistency – Solution: • Queue Based processing Model • Change in behavior – Order Placed successfully TO Order Received Successfully 31
  32. 32. Design Principles – Asynchronous Processing • Blocking is bane for Scalability – Intent: • Avoid blocking calls, reduce contention – Solution: • Queue Based processing Model • Fire and Forget Calls • 1000 users blocked for 5 seconds = 5000 users per second 32
  33. 33. Design Principles – Parallel Design • Design for Parallel and Reliable Work – Intent: Increasing resources should results in a proportional increase in performance. Dependent services might not be available. Blocking is bane of scalability – Drivers: Higher reliability, Proportional distribution – Solution: • Concern Independent Scaling • Reliability through Queue • Queue driven worker tasks - more messages more workers faster work 33
  34. 34. Queue Based Pattern 34
  35. 35. Queue - Load Leveling, Load Balancing, Loose Coupling 35
  36. 36. Design Principles – Queue Based Pattern • Idempotent – Design the operation to be idempotent; that is, if it's carried out more than once, it's as if it was carried out just once – Implement the receiver in such a way that it can receive a message multiple times safely, either through a filter that removes already received messages or by adjustment of message semantics 36
  37. 37. Design Principles – Capacity Planning • Everything has a limit: Compose a Scale – Intent: Design Around Provider SLAs and Capacity – Solution: • Know the limits, measure the scalability and increase the scale • E.g. Storage supports up to 10000 transactions/sec – Add storage for higher scale • E.g. Queue supports 5000 messages per seconds – Add additional Queues (Partitioning) for additional scale 37
  38. 38. Design Pattern – Multi Site Deployment Pattern 38 Database Server Web Servers Application Servers Database Server Web Servers Application Servers Sync Routing • Performance Based • Round Robin • Failover Asia United States
  39. 39. Summary 39
  40. 40. Scalability Principles 40 Scalability Stateless Parallelization Asynchronous Partitioning Idempotent Fault Tolerance
  41. 41. Vertical Vs. Horizontal Scaling 41 Vertical Scaling Horizontal Scaling ACID BASE Availability First Focus on Commit Pessimistic Locking Optimistic Locking Transactional Shared nothing Favor Consistency Maximum Scalability Most Distributed Systems Realize Both
  42. 42. Thank You ! 42
  43. 43. 43
  44. 44. 44 Some of the images are taken by utilizing Google search and due credit to the source. Author do not claim any creation or originality of the contents. It is used only for learning purposes

×