- Scalability & Availability for the Global Markets
- Global scaled Scalability, Availability and Security
- Architecture for 100, 1K, 100K, 500K, 1M and 10M global users
- Auto-Scaling
- Understand Cloud Services
- Cloud Demo(AWS, GCP, Azure and Cloudflare)
- Wrap-Up
Scalability strategies for cloud based system architecture
1. Service Platform Architect
Brandon Kang
sangjinn@gmail.com
https://tech.brandonkang.net
May 2020
Scalability Strategies for
Cloud based System Architecture
2. Agenda
• Scalability & Availability for the Global Markets
• Global scaled Scalability, Availability and Security
• Architecture for 100, 1K, 100K, 500K, 1M and 10M global users
• Auto-Scaling
• Understand Cloud Services
• Cloud Demo(AWS, GCP, Azure and Cloudflare)
• Wrap-Up
4. Scalability
• Scalability = capability of a system to handle a growing work
• Vertical : Scale Up or Down
ü Add or Remove Resources
ü CPU
ü Memory
ü Storage
• Horizontal: Scale Out or In
ü Add or Remove Systems
ü Instance Scale OutScale In
VM VM VM VMVM VMVM VM
Scale Down
Scale Up
VMVM
6. Scalability vs. Availability
• Need 4 * VMs to provide services
Scalability: (2* VMs in a region) + (2* VMs in another region)
Availability: (4* VMs in a region) + (4* VMs in another region) for HA
13. Users > 500,000
• MSA(Microservices Architecture)
ü Every functions move to Microservices
ü Independent and loosely coupled
• API Gateway
ü API Routing
ü API Security
ü Authentication
ü Authorization
ü API Caching
ü Hits Rate Limit
ü Hits Throttling
ü Traffic Monitoring
15. Actions for 1M Users
• What data should be move to NoSQL from RDBMS?
ü Temporary but massive data e.g.) click information, log data, session data
ü Hot tables (tables who have very frequent updates)
ü Meta data store and search e.g.) object information from storage
ü Dynamic schema
ü Time-series table e.g.) monitoring logs
17. Actions for 10M Users
• Multi Zone architecture in every region
• Consider expansion for all components e.g.) Multi-Master DB
• Maximize caching utilization from architecture and network
• Make autonomous systems
• Build auto-scaled systems using Cloud or Kubernetes
ü Virtual machine
ü Load balancer
ü Object storage
ü Notification system
ü Queuing system,
ü Workflows like Emailing, Alarming, etc.
18. Actions for 10M Users
• Entire Architecture tuning
• Application level tuning
• Globally distributed systems
• Global HA architecture (Multi Zone -> Multi Region)
• DevOps based application deployment
• CI(Continuous Integration)& CD(Continuous Development)
21. Actions for 10M Users
• Autoscaling in On-Premise
desiredReplicas = ceil[currentReplicas * ( currentMetricValue / desiredMetricValue )]
* RC
Replication Controller
* Pod
A Group of containers
25. Cloud Pros
• Fully Managed Services
• Flexible Scale Up/Down and Scale Out/In
• Auto Scaling
• HA using Multi Availability Zone and Multi Regions
• Performance
• Unlimited Capacity
• Serverless Architecture
• KMS(Key Management System)
• Computing, Network Big Data, IoT, AI, Machine Learning, etc.
• Low CAPEX/OPEX
• Compliance and Regulation e.g.) GDPR, CCPA
26. Cloud Cons
• Not Cheap and Unexpected Cost
• Not easy to migrate from On-Premise
• Sensitive Security
• No regions in Vietnam yet
• Vendor Stickiness?
• 99.999999999% Availability e.g.)AWS S3
• Support from outside vendors
• Need Experience and Training
35. - Cloud Demo
Demo for 20minutes with these implementation
1. Video Conference(WebRTC) platform using GCP
2. Testing PostgreSQL Multi-Master using Azure
3. AWS Chatbot usages for CloudWatch and Serverless Lambda
4. CDN, DNS and WAF integration using Cloudflare
37. Best Practices for Global Markets
• Multi Regional Data Center (or Cloud)
• CDN(Content Delivery Network)
• Prepare Attacks -> WAF, DDoS Scrubbing, Bot Managing
• Server -> Instance in Cloud -> Serverless/Kubernetes
• Maximize Queuing
• Authentication out of Server
• DB -> Master/Slave, Primary/Secondary
• N/W Caching, DB Caching, Client Caching for Performance!
• Manual and routine jobs -> Automatic Workflows
• Microservices Architecture, DevOps and CI/CD
• Global Regulations Awareness
• Smart small, Grow Bigger!
38. Best Practices for Global Markets
Auto-Scale Everything,
Cache Everything,
And Protect Everything!
39. - Thank You. -
Service Platform Architect
Brandon Kang
sangjinn@gmail.com
https://tech.brandonkang.net