1. Scaling for Web
Architecting HootSuite for 5 Million+ Users

2. About Us
Beier Cai
Chief Architect
beier.cai@hootsuite.com
@beiercai
Bill Monkman
Lead Developer, Web
bill.monkman@hootsuite.com
@bmonkman

3. Hoo’s HootSuite

4. Some Numbers
• 5+ million Members
• 10+ million Social Networks
• 25+ million Streams
• 350+ million Messages Scheduled
• 1.3+ Billion Messages Sent
• 2+ Billion Page Views per month
• 2,000 Web Requests/sec

6. Technologies
PHP
MySQL

7. Technologies
PHP
Memcached
MySQL

8. Technologies
PHP
Gearman
Memcached
MySQL

9. Technologies
PHP
Gearman
Memcached
MySQL MongoDB

10. Technologies
PHP
Gearman ZeroMQ
Memcached
MySQL MongoDB

11. Technology Stack
• LAMP (Linux, Apache, MySQL, PHP)
• Scala
• Akka
• Python
• Clojure
• MongoDB
• Redis
• Memcached
• Gearman
• ZeroMQ
• RabbitMQ/AMQP
• jQuery/Backbone
• EC2 Cloud / CloudStack
• Jenkins / CruiseControl
• CloudFront CDN
• HAProxy/Nginx/Varnish
• Zend
• Statsd/Graphite
• LogStash
• ElasticSearch/Lucene
• Nagios/Munin/Sensu
• ZooKeeper

12. Optimization
• Push processing up to the front-end, make your users’ computers into a
distributed processing network. e.g. stream rendering
• Cache as much as possible, invalidate only when necessary.
• Sometimes features have to take a back-seat to performance/scalability.
• Look at profiler data.
• Watch query execution time, optimize when necessary with indexes,
partitions, de-normalization, etc.
• Know what your weak spots are and keep a close eye on them.

13. Optimization

14. Optimization Case Study

15. Optimization Case Study

17. Scaling Dev Team
• Passion
• Commitment
• Dedication
• Responsibility
• General Awesomeness
Besides technical competency, we also value

18. Scaling Dev Team
• Passion
• Commitment
• Dedication
• Responsibility
• General Awesomeness
Besides technical competency, we also value

19. Scaling Dev Team

20. Team Work
• Entrepreneurial
• Communication
• Respect
• Trust

21. Coding Practice
• Clean and Readable
• Maintainable
• Reusable
• Scalable
• Commented
Have a well defined coding standard within your dev team, and
follow it!

22. Monitoring, Profiling & Logging

23. Monitoring, Profiling & Logging

24. Case Study

25. Case Study

26. Thank You!
Beier Cai
beier.cai@hootsuite.com
twitter.com/beiercai
Bill Monkman
bill.monkman@hootsuite.com
twitter.com/bmonkman