This document discusses lessons learned from building the Wakoopa software. It outlines how the company scaled its infrastructure from one server to support 200 million users. Key strategies included splitting the server into staging, production and database roles, using background job queues, optimizing database queries, archiving old hourly usage data, and plans to implement MySQL partitioning and other technologies to improve performance.
20. Overrides default PK
class HourlyUsage < ActiveRecord::Base
set_primary_keys :user_id, :software_id, :used_at
...
end
Original idea by Dr Nic
Enhanced for Rail 2.1 by Darrin Holst
http://compositekeys.rubyforge.org/
22. HourlyUsage
to
HourlyUsageStorage
SELECT ... FROM hourly_usage WHERE used_at < '...' INTO OUTFILE '/path/to/file'
LOAD DATA INFILE '/path/to/file' REPLACE INTO TABLE hourly_usage_storage
avoids long locks
24. MySQL 5.1
Partitioning
CREATE TABLE hourly_usage_storage ( ... )
PARTITION BY RANGE (software_id) (
PARTITION p0 VALUES LESS THAN (...),
PARTITION p1 VALUES LESS THAN (...),
PARTITION p2 VALUES LESS THAN (...),
PARTITION p3 VALUES LESS THAN (...),
....
PARTITION pn VALUES LESS THAN MAXVALUE
);
Makes ‘WHERE software_id = ...’ very fast
Available in MySQL 5.1
25. And many more plans
Designing a new infrastructure
MySQL 5.1
Memcached
Sphinx
Rails 2.1
Amazon EC2