Slides from the talk I gave a

1. How Flipkart
Scales PHP
Tips & Tricks

2. About Flipkart
• Flipkart.com
• India’s largest eCommerce website*
• Millions of requests/day

3. About Me
• Siddhartha Reddy
• Tech Lead at Flipkart
• Taking care of Search & Browse experience

4. Today
• Flipkart.com’s Architecture
• How each component addresses scale

5. Flipkart.com’s
Architecture
Session
Database
Store
CMS
Load Web
Balancer Server(s)
Search
Caches OMS

6. What is Scaling?

7. What is Scaling?
• Handle large amount of traffic
• High availability
• Good response times

8. How Flipkart Scales

9. How Flipkart Scales
• Logging and measuring
• Horizontal Scaling
• Load-balancing
• High Availability setup
• Web-server optimizations
• Caching

10. How Flipkart Scales

11. How Flipkart Scales
• Logging and measuring
• Horizontal Scaling
• Load-balancing
• High Availability setup
• Web-server optimizations
• Caching

12. Logging & Measuring

13. Logging & Measuring

14. How Flipkart Scales
• Horizontal Scaling
• Load-balancing
• High Availability setup
• Web-server optimizations
• Caching

15. Traffic

16. Servers: Chuck Norris Style

17. Servers: Spartan Style

18. Horizontal Scaling

19. Horizontal Scaling
Traffic
web-1 web-2 ... web-n

20. Horizontal Scaling
Load-
Traffic balancer
web-1 web-2 ... web-n

21. Horizontal Scaling
Load-
Traffic balancer
web-1 web-2 ... web-n

22. Horizontal Scaling
Load-
Traffic balancer
web-1 web-2 ... web-n

23. Horizontal Scaling
Load-
Traffic balancer
web-1 web-2 ... web-n

24. Horizontal Scaling
Load-
Traffic balancer
web-1 web-2 ... web-n

25. How Flipkart Scales
• Horizontal Scaling
• Load-balancing
• High Availability setup
• Web-server optimizations
• Caching

26. Load Balancer

27. Load Balancer
• Internal binary services (Apache Thrift)
• Internal web services (HTTP)
• External web traffic (HTTP/HTTPS)

28. Load Balancer
• Internal binary services (Apache Thrift)
• Internal web services (HTTP)
• External web traffic (HTTP/HTTPS)

29. Load Balancer
• Internal binary services (Apache Thrift)
• Software: HAProxy
• Transport layer (TCP) load balancer
• TCP connect for availability check
• Internal web services (HTTP)
• External web traffic (HTTP/HTTPS)

30. Load Balancer
• Internal binary services (Apache Thrift)
• Internal web services (HTTP)
• External web traffic (HTTP/HTTPS)

31. Load Balancer
• Internal binary services (Apache Thrift)
• Internal web services (HTTP)
• Software: Varnish
• Application Layer (HTTP) load balancer
• Configurable/scriptable caching
• HTTP GET for availability check
• External web traffic (HTTP/HTTPS)

32. Load Balancer
• Internal binary services (Apache Thrift)
• Internal web services (HTTP)
• External web traffic (HTTP/HTTPS)

33. Load Balancer
• Internal binary services (Apache Thrift)
• Internal web services (HTTP)
• External web traffic (HTTP/HTTPS)
• Software: Nginx
• Application Layer (HTTP/HTTPS) load balancer
• Caches and serves static content (JS/images)
• HTTP GET for availability check

34. Load Balancer:
Session Affinity?
Load-
balancer
... web-n
web-1 web-2

35. Load Balancer:
Session Affinity?
SID: 123 (1)
Load-
balancer
... web-n
web-1 web-2

36. Load Balancer:
Session Affinity?
SID: 123 (1)
Load-
balancer
... web-n
web-1 web-2

37. Load Balancer:
Session Affinity?
SID: 123 (1)
Load-
balancer
SID: 123 (1)
... web-n
web-1 web-2

38. Load Balancer:
Session Affinity?
SID: 123 (1)
Load-
SID: 123 (2) balancer
SID: 123 (1)
... web-n
web-1 web-2

39. Load Balancer:
Session Affinity?
SID: 123 (1)
Load-
SID: 123 (2) balancer
SID: 123 (1)
... web-n
web-1 web-2

40. Load Balancer:
Session Affinity?
SID: 123 (1)
Load-
SID: 123 (2) balancer
SID: 123 (1)
SID: 123 (2)
... web-n
web-1 web-2

41. HTTP Load Balancing:
Session Affinity

42. HTTP Load Balancing:
Session Affinity
• (+) Sessions can be stored on the web server
• (+) Different code on different web servers
• Useful for testing new code on a subset of users
• (-) If any web server goes down, need to migrate sessions
• (-) Not easy to take a web server out-of-rotation

43. HTTP Load Balancer:
Session Non-affinity

44. HTTP Load Balancer:
Session Non-affinity
• Sessions stored in a common session-store
• (+) Easy to manage -- all web servers have identical state
• (+) If a web server goes down -- no problem
• (+) Can take any web server out-of-rotation
• (-) Can’t deploy to a subset of servers for testing
• (-) Central session-store could be slower

45. HTTP Load Balancer:
Session Some-affinity

46. HTTP Load Balancer:
Session Some-affinity
• Sessions sticky to a subset (group) of machines
• Use a common session-store

47. How Flipkart Scales
• Horizontal Scaling
• Load-balancing
• High Availability setup
• Web-server optimizations
• Caching

48. Load Balancer:
Hardware?

49. Load Balancer:
Hardware?
• Expensive
• Inflexible to manage
• They run Linux anyway!

50. Flipkart’s Load-
Balancers

51. Flipkart’s Load-
Balancers

52. Flipkart’s Load-
Balancers

53. Flipkart’s Load-
Balancers

54. Flipkart’s Load-
Balancers

55. Flipkart’s Load-
Balancers

56. Flipkart’s Load-
Balancers

57. Flipkart’s Load-
Balancers

58. Flipkart’s Load-
balancers
bond:1 bond:1
(10.3.1.1) (10.3.1.2)
bond:0 bond:0
(10.3.0.1) (10.3.0.2)

59. Flipkart’s Load-
balancers
bond:1 bond:1
(10.3.1.1) (10.3.1.2)
bond:0 bond:0
(10.3.0.1) (10.3.0.2)
Active

60. Flipkart’s Load-
balancers
bond:1 bond:1
(10.3.1.1) (10.3.1.2)
bond:0 bond:0
(10.3.0.1) (10.3.0.2)
Active Hot Stand-by

61. Flipkart’s Load-
balancers
bond:1 bond:1
(10.3.1.1) (10.3.1.2)
bond:0 bond:0
(10.3.0.1) (10.3.0.2)
bond:0:0
(10.3.0.0)
Active Hot Stand-by

62. Flipkart’s Load-
balancers
bond:1 bond:1
(10.3.1.1) (10.3.1.2)
bond:0 bond:0
Heartbeat
(10.3.0.1) (10.3.0.2)
bond:0:0
(10.3.0.0)
Active Hot Stand-by

63. How Flipkart Scales
• Horizontal Scaling
• Load-balancing
• High Availability setup
• Web-server optimizations
• Caching

64. On the Web Servers

65. On the Web Servers
• PHP5

66. On the Web Servers
• PHP5
• Apache/mod_php

67. On the Web Servers
• PHP5
• Apache/mod_php

68. On the Web Servers
• PHP5
• Apache/mod_php
• Apache/mod_fcgid (FastCGI)

69. On the Web Servers
• PHP5
• Apache/mod_php
• Apache/mod_fcgid (FastCGI)

70. On the Web Servers
• PHP5
• Apache/mod_php
• Apache/mod_fcgid (FastCGI)
• PHP-FPM

71. On the Web Servers
• PHP5
• Apache/mod_php
• Apache/mod_fcgid (FastCGI)
• PHP-FPM
• PHP FastCGI Process Manager
• Great process management
• Security
• Configurability & control
• Adaptive process spawning (5.3.3RC1+)

72. On the Web Server:
fk-w3-agent

73. On the Web Server:
fk-w3-agent
• Simple Java “middleware” daemon

74. On the Web Server:
fk-w3-agent
• Simple Java “middleware” daemon
• Deployed on each web server

75. On the Web Server:
fk-w3-agent
• Simple Java “middleware” daemon
• Deployed on each web server
• Communicates with PHP through local socket

76. On the Web Server:
fk-w3-agent
• Simple Java “middleware” daemon
• Deployed on each web server
• Communicates with PHP through local socket
• Hosts pluggable “handlers”

77. On the Web Server:
fk-w3-agent
• Simple Java “middleware” daemon
• Deployed on each web server
• Communicates with PHP through local socket
• Hosts pluggable “handlers”

78. fk-w3-agent: Why?

79. fk-w3-agent: Why?
• “I wish PHP could ...”

80. fk-w3-agent: Why?
• “I wish PHP could ...”
• Logging

81. fk-w3-agent: Why?
• “I wish PHP could ...”
• Logging
• Connection pools

82. fk-w3-agent: Why?
• “I wish PHP could ...”
• Logging
• Connection pools
• Parallelization

83. fk-w3-agent: Why?
• “I wish PHP could ...”
• Logging
• Connection pools
• Parallelization
• Config & discovery

84. fk-w3-agent: Why?
• “I wish PHP could ...”
• Logging
• Connection pools
• Parallelization
• Config & discovery
• Running statistics

85. fk-w3-agent: Logging

86. fk-w3-agent: Logging
• We like to log, a lot

87. fk-w3-agent: Logging
• We like to log, a lot
• Logs need to be flushed to disk

88. fk-w3-agent: Logging
• We like to log, a lot
• Logs need to be flushed to disk
• PHP resources wasted
• waiting for disk I/O
• waiting for lock on log files
• or logs go missing

89. fk-w3-agent: Logging

90. fk-w3-agent: Logging
• PHP processes send logs to fk-w3-agent
• fk-w3-agent writes logs to files

91. fk-w3-agent:
Connection Pooling

92. fk-w3-agent:
Connection Pooling
• Many service calls / request

93. fk-w3-agent:
Connection Pooling
• Many service calls / request
• Cost of recreating connections high

94. fk-w3-agent:
Connection Pooling
• Many service calls / request
• Cost of recreating connections high
• PHP does not support connection pools*

95. fk-w3-agent:
Connection Pooling

96. fk-w3-agent:
Connection Pooling
• fk-w3-agent maintains persistent connection pools
to various services
• PHP accesses services by communicating with fk-
w3-agent

97. fk-w3-agent:
Parallelization

98. fk-w3-agent:
Parallelization
• Service calls can be parallelized

99. fk-w3-agent:
Parallelization
• Service calls can be parallelized
• Multi-threading in PHP is ...

100. fk-w3-agent:
Parallelization

101. fk-w3-agent:
Parallelization
• PHP sends request to fk-w3-agent
• fk-w3-agent uses multiple threads to hit services in
parallel

102. fk-w3-agent:
Config & Discovery

103. fk-w3-agent:
Config & Discovery
• Change config without deployment?

104. fk-w3-agent:
Config & Discovery
• Change config without deployment?
• Simple: store config in a database

105. fk-w3-agent:
Config & Discovery
• Change config without deployment?
• Simple: store config in a database
• But: reloading config on each request: expensive

106. fk-w3-agent:
Config & Discovery

107. fk-w3-agent:
Config & Discovery
• fk-w3-agent loads config from database
• Caches it in memory
• Keeps it up-to-date
• PHP gets config from fk-w3-agent

108. fk-w3-agent:
Running Statistics

109. fk-w3-agent:
Running Statistics
• PHP sends basic stats to fk-w3-agent for every
request
• fk-w3-agent aggregates these stats
• Used for monitoring

110. How Flipkart Scales
• Horizontal Scaling
• Load-balancing
• Web-server optimizations
• Caching

111. To Cache or not to
Cache?

112. To Cache or not to
Cache?
• “There are only two hard things in Computer
Science: cache invalidation and naming things”. --
Tim Bray quoting Phil Karlton

113. To Cache or not to
Cache?
• “There are only two hard things in Computer
Science: cache invalidation and naming things”. --
Tim Bray quoting Phil Karlton
• “There are only two hard problems in Computer
Science: cache invalidation, naming things and off-by-1
errors.”

114. Caches Invalidation

115. Caches Invalidation
• Objects updated => Notification Message

116. Caches Invalidation
• Objects updated => Notification Message
• Remove updated objects from cache

117. Caches Invalidation
• Objects updated => Notification Message
• Remove updated objects from cache
• Replace updated objects in cache

118. Caches Invalidation
• Objects updated => Notification Message
• Remove updated objects from cache
• Replace updated objects in cache
• Aggressively populate cold cache

119. Caches at Flipkart

120. Caches at Flipkart
• Memcached -- in memory caching

121. Caches at Flipkart
• Memcached -- in memory caching
• Redis -- persistent store/cache

122. Caches at Flipkart
• Memcached -- in memory caching
• Redis -- persistent store/cache
• Varnish -- HTTP caching

123. Caches at Flipkart
• Memcached -- in memory caching
• Redis -- persistent store/cache
• Varnish -- HTTP caching
• fk-w3-agent -- Config caching

124. Let’s continue the
discussion
• @sids on Twitter
• siddhartha@flipkart.com
• #phpcloud on irc.freenode.net