This document summarizes Berk D. Demir's design for a content addressable storage system to store and serve large amounts of static assets with low latency, high availability, and without data duplication. The key aspects of the design are: 1) Using HBase as the underlying distributed database to store immutable rows of metadata and blob content in a single table with different column families based on access patterns. 2) Addressing content via a cryptographic hash of the content rather than a database key to allow immutable and deduplicated storage. 3) Serving the stored content via HTTP using common verbs and headers to provide a simple interface for clients.

1. Content
Addressable
Storages
for fun and profit.

2. Berk D. Demir
@bddemir

3. I break and fix things at
@StumbleUpon.

4. Problem.

5. Serve lots of static assets
with low latency
and high availability.

6. Understanding
data.

7. A lot. 100 million.
Small. 19 kilobytes.
Frequent. Updates.

8. BLOBs don’t change.
They get replaced.
We want to keep all
without duplicating.

9. Content
Addressable
Store

10. Store immutable content and
authoritatively address it with
a cryptographic hash.

11. We had ideas.

12. Very bad ideas.

13. Very bad.
Shared Storage, i.e., NFS.

14. Bad ideas.

15. Bad.
AWS S3, RS Cloud Files, ...
Distributed: AFS, Gluster
HDFS (Oh my!)

16. Bad ideas.
Take 2

17. o_O
Write a distributed, fault
tolerant, replicating, multi
datacenter, fast, CAS for
BLOBs.

18. Reimplementing a lot of
things is generally not a
good sign.

19. Reuse.
Don’t reimplement.

20. HBase
Distributed,
Fault tolerant,
Replicating,
Multi datacenter,
Fast.

21. Immutable rows with compact
keys, separated into different
column families based on their
access patterns.

22. One table to rule them all.
m: d:
MD5 16 bytes Metadata BLOB
(SHA-1 20 bytes) 9 bytes Many bytes
MAX_FILESIZE => 20G,
VERSION => 1,
BLOCKCACHE => true,
BLOOMFILTER => ROW
Pre-split into 512 regions at table creation time.

23. Scala,
Finagle,
asynchbase,
Varnish

24. HTTP has a lot to offer.

25. Verbs
GET HEAD PUT DELETE
GET /KwIEec5utYGrKmzXYLgFzg HTTP/1.1
Host: b9.sustatic.com

26. Headers
Cache-Control: max-age=<1 year>
Last-Modified: <cell timestamp>
Content-MD5: <row key: base64>
Content-Disposition: attachment; filename=su.xpi

27. HBase and HTTP are the
perfect tools to build
simple, reliable, fast
data services.

28. Get excited and build
things!

29. Thanks.

30. Like the design of this slide deck?
Direct your positive feedback to Coda Hale (@coda)