Design a scalable site: Problem and solutions

1. Design a scalable site:
Problems and Solutions
Châu Nguyễn Nhật Thanh
Head of Game Backend and Cloud
VNG Corp.

2. Content
Client Internet
1
2
3
4
Server
Kernel
App
Server
6
7
8
5
9

3. How to scale
● Optimize at node
– Maximize the performances at node
● Scale to multiples nodes
– How to scale ?
● Scale UP
● Scale OUT

4. 1.Client Problems
Client Internet
1
2
3
4
Server
Kernel
App
Server
6
7
8
5
9

5. 1. Client Problems
● Web browser (HTTP)
– Static contents: js, css , image, video
– JS, CSS, image:
● minify, merge
● Compress header
– Caching:
● Expires header
● Local storage
● JS caching

6. 1. Client Problems
● Mobile client
– Local storage
– In app resources bundle

7. 2. Client to ISP
Client Internet
1
2
3
4
Server
Kernel
App
Server
6
7
8
5
9

8. 2. Client to ISP
● CDN (Content Delivery Network)
– Redirect (302)
– DNS server
● Content in ISP DC
● Return IP of ISP near your devices

9. 2. Client to ISP

10. 2. Client to ISP Problems

11. 3.ISP Problem
Client Internet
1
2
3
4
Server
Kernel
App
Server
6
7
8
5
9

12. 3.ISP Problem
● Routing
– Internal ISP
– Cross ISP
● Capacity of lease line between 2 ISPs
● Traffic monitoring
– 3G/4G users
● Telco routing

13. 4.ISP to DC Problem
Client Internet
1
2
3
4
Server
Kernel
App
Server
6
7
8
5
9

14. 4.ISP to DC Problem
● Routing
– DC to ISP
– Internal DC
● FW
● Network hardware capability
● Cross product routing
– Bandwidth
– FW
– SDN ????

15. 5.Server Problems
Client Internet
1
2
3
4
Server
Kernel
App
Server
6
7
8
5
9

16. 5.Server Problems
● Network card
– Gigabits
– Fiber
– Virtualization support ???
● CPU
● RAM
● GPU

17. 5.Server Problems
● Scale UP vs scale OUT

18. 5.Server Problems
● How to load balancing between nodes
– HA Proxy
– Nginx
– F5

19. 5.Server Problems
● HA Proxy config sample
backend web-backend
balance roundrobin
server web1 web1.yourdomain.com:80 check
server web2 web2.yourdomain.com:80 check
backend blog-backend
balance roundrobin
mode http
server blog1 blog1.yourdomain.com:80 check

20. ● HA Proxy config sample
frontend http
bind *:80
mode http
acl url_blog path_beg /blog
use_backend blog-backend if url_blog
default_backend web-backend

21. 6. Kernel Problems
Client Internet
1
2
3
4
Server
Kernel
App
Server
6
7
8
5
9

22. 6. Kernel Problems
● Drivers for HW optimization
● Max fd
● IP forwarder
● Local FW
– Iptables
● Network IO multiplexing support ???
– EPOLL
● https://code.facebook.com/posts/938078729581886/impro
ving-the-linux-kernel-with-upstream-contributions/

23. 7. Application Problems
Client Internet
1
2
3
4
Server
Kernel
App
Server
6
7
8
5
9

25. 7. Application Problems
● Data structures
– Array
– Hash map
– Linked list
==> how to use ???

26. 7. Application Problems
● Caching
– Local Cache
● Use Data structure
– External Cache
● Memcache
● Redis
– Caching algorithm
● LRU
● FIFO
● ….

27. 7. Application Problems
● Data serialize
– Text Protocol
● Json, XML
– Binary
● Thrift, Protobuff …
– Compression
● Snapy, ZIP, GZIP

28. 8. Application to kernel problems
Client Internet
1
2
3
4
Server
Kernel
App
Server
6
7
8
5
9

29. 8. Application to kernel problems
● Memory copy overhead
– Share mem
– Zero copy buffer
● IO multiplexing
– Blocking
● Process base
● Thread base
– Nonblocking
● EPOLL
● KQUEUE
● Use libevent

30. 9.Cross Server Problems
Client Internet
1
2
3
4
Server
Kernel
App
Server
6
7
8
5
9

31. 9.Cross Server Problems
● Cross platform RPC
– Thrift
– GRPC

32. Thank you
● Q&A