Presentation from DC PHP 2009

1. Clustered PHPDC PHP 2009 by Marcel Esser – marcel.esser@croscon.com Senior Developer & Systems Analyst, CROSCON

2. what is clustering? “A computer cluster is a group of linked computers, working together closely so that in many respects they form a single computer.” - Wikipedia

3. what is clustering? “A computer cluster is a group of linked computers, working together closely so that in many respects they form a single computer.” - Wikipedia …in other words, it’s really hard.

4. what is clustering not? Clustering is not high availability. … but you often achieve high availability with clustering.

5. what is clustering not? Clustering is not high availability. … but you often achieve high availability with clustering. Clustering is not high volume. … but you often achieve high volume with clustering.

6. what is clustering not? Clustering is not high availability. … but you often achieve high availability with clustering. Clustering is not high volume. … but you often achieve high volume with clustering. We want both.

7. also… clustering is really, really hard

8. some do it well

9. some … eh well, nevermind.

10. why cluster? service more users

11. why cluster? service more users service users faster

12. why cluster? service more users service users faster increase reliability

13. why cluster? service more users service users faster increase reliability get rich

14. objectives linear capacity increase

15. 1 server - 100 lolcats per second

16. 2 servers - 200 lolcats per second

17. 3 servers - 300 lolcats per second

18. objectives linear capacity increase linear cost increase

19. 100 lolcats - $100

20. 200 lolcats - $200

21. 300 lolcats - $300

22. objectives linear capacity increase linear cost increase exponential reliability increase

24. Database Scaling

25. Replicated Storage

26. Backups

27. Data Caches

28. Distributed Sessions

29. Staging Strategies

30. Debugging

33. balance by performance

34. balance by weight

35. route by geolocation

36. support sticky sessions

38. Big IP

39. Perlbal

40. nginx

43. users don’t share data between each other (between servers)

44. when you need more capacity, add another database server

45. works for some apps, doesn’t work for othersimplementation example: invoice and timesheet management app

46. database replication (mysql) master - master master - slave master - many slave

48. complete nightmare when replication gets desynchronized

49. doesn’t actually improve write performance

51. makes you re-write your application to consider slave queries

52. doesn’t increase write performance

53. de-synchronization is relatively painless

55. makes you re-write your application to consider slave queries

56. doesn’t increase write performance

57. de-synchronization is relatively painless

59. logical index is kept of what data is where (for example, a mathematical index or a lookup chart)

60. you have to grab, parse, and correlate data across servers

61. theoretically limitless scalability

62. complicatedimplementation example: digg, facebook, etc

63. replicated storage common things you can do: replicated file system lookup tables storage services huge NAS arrays

65. various replication modes

66. nothing to keep track of in your app

67. easy to implement

69. limitless mode; entirely up to you

70. entirely dependent on your application logic

72. theoretically limit-less capacity

73. easy to use

74. data must be pulled back first if used locally

76. insanely expensive

77. insanely expensive

78. bullet-proof fault tolerance… at a price

80. backups all-RAID doesn’t work Why? RAID won’t keep your application from deleting data everywhere

82. easy if you do it from the beginning

83. requires some planning

85. easy if you do it from the beginning

86. requires some planning

88. fast

89. scalable

90. proven

91. use itGot configuration data? Small, high-TTL data sets? Use APC. Large, high-TTL data sets? Use files. Mind the race condition.

93. database

96. do profile your to loop for obvious performance issues

97. do use phpdoc tags

98. do make your dev environment as similar to live as practical (i.e., don’t develop on Windows and run live on UNIX)

99. do document all your changes

101. do create data fixtures that are representative of real-life data

102. do create functional tests for the user interface (Selenium)

104. do connect to the live database

105. do have test ‘users’ to perform destructive operations against

107. do automate pushing updates

108. do take nodes out rotation when you push updates

110. do prepare an automated action that can turn xdebug and profiling on/off on 1 of the live nodes. you can and will run into errors that only exist on live.

111. do write a test case to replicate the the bug first and then fix the bug, whenever possible

112. do first look if bugs are explainable by platform differences between development and production systems (i.e., don’t develop on Windows and deploy on UNIX)

114. PHP doesn’t have a native message queue, so (many) people wrote some. example, gearmand. do use a message queue.

115. do check for memory leaks in background tasks! many php libraries and also many php versions themselves still leak memory. try to write a loop in bash for a background task rather than in php. recycle the process often.

116. do plan your message format carefully

118. shameless plug CROSCON Bespoke Application Development Consulting Security Marcel Esser marcel.esser@croscon.com (202) 730-9728 Personal? marcel.esser@gmail.com