Facebookが開発した写真ストレージ "Haystack" の説明プレゼンです。 本家の説明文を和訳してるだけです。 ソースURL: http://www.facebook.com/note.php?note_id=76191543919

1. About Haystack
Tomohiro MITSUMUNE
2009/11/26
2009 12 2

2. Agenda
• about Facebook
• Old Photo Store system
• Haystack Store system
2009 12 2

3. Facebook
• SNS( : 250,000,000)
• 1
2009 12 2

4. Photo App in Facebook
• 150
• 4
• 600 1.5PB
•1 2 2 25TB
• 550,000
2009 12 2

5. Old Facebook System
2009 12 2

6. NFS Photo Infrastructure
• Upload
•
NFS
• Photo serving
• HTTP NFS
• NFS storage
•
2009 12 2

7. NFS Photo Infrastructure
•
• i
• NFS
multiple I/O
Operation
2009 12 2

8. NFS Photo Infrastructure
• NFS
• CDN
2009 12 2

9. Additonal Optimization
• Cachr
• Facebook
• NFS file handle cache
• NFS
photo serving
2009 12 2

10. New Facebook System
2009 12 2

11. Haystack Photo Infrastructure
• photo serving storage 1
• HTTP
•
• I/O
2009 12 2

12. Haystack Photo Infrastructure
• HTTP Server
• Photo Store
• Haystack Object Store
• Filesystem
• Storage
2009 12 2

13. Storage
2009 12 2

14. Storage
• 2U
• 2 x quad-core CPUs
• 16GB - 32GB
• RAID (256MB
- 512MB NVRAM cache )
• 12 + 1 TB SATA HDD
2009 12 2

15. Storage
• 1 10TB
• RAID-6
•
• NVRAM write-back
2009 12 2

16. Storage
•
NVRAM
•
•
2009 12 2

17. Filesystem
2009 12 2

18. Filesystem
• 10TB 1
• inode
• 2
•
•
2009 12 2
• Haystack XFS

19. Block based filesystem
•
•
2009 12 2

20. Extent based filesystem
•
•
•
2009 12 2

21. Haystack Object Store
2009 12 2

22. Haystack Object Store
• Needle
•2
• Haystack Store file ( )
•
2009 12 2

23. Haystack Store File
8KB
2009 12 2

24. Needles
2009 12 2

25. Needles
• <Needle Offset, Key, Alternate Key,
Cookie>
• Offset haystack store
• Key OK
2009 12 2

26. Index
2009 12 2

27. Index
•
•
• haystack store file
2009 12 2

28. Haystack Write
Operation
• Needle Store file
• Needle
•
2009 12 2

29. Haystack Write
Operation
•
flush
• <Key,
Alternate Key, Cookie>
• Needle
2009 12 2

30. Haystack Read
Operation
• Needle offset, Key, Alternate Key, Cookie,
Data size
• Key Alaternate Key Cookie
2009 12 2

31. Haystack Delete
Operation
• ‘deleted’
• Needle
•
2009 12 2

32. Photo Store
2009 12 2

33. Photo Store
• HTTP
•
Haystack Store Object /
2009 12 2

34. Photo Store
• I/O
•
• Google Sparse Hash
2009 12 2

35. Photo Store
• 64bit ID
4
• Cookie 64bit
• (large, medium, small,
thumbnail) Alternate
2009 12 2

36. in-memory Index
2009 12 2

37. Photo Store Write/
Modify Operation
• haystack
•
2009 12 2

38. Photo Store Read
Operation
• haystack id, photo key, , cookie
• photo key
Needle
2009 12 2

39. Photo Store Delete
Operation
• Haystack Delete Operation
0
2009 12 2

40. Compaction
• ( ) Needle
•
haystack
2009 12 2

41. HTTP Server
2009 12 2

42. HTTP Server
• libevent evhttp
• 1 1
• I/O bound
2009 12 2

43. • HTTP
•
•
2009 12 2

44. Thank you for
your kind attention
2009 12 2

45. References
• http://www.facebook.com/note.php?
note_id=76191543919
• http://www.niallkennedy.com/blog/2009/04/
facebook-haystack.html
• http://perspectives.mvdirona.com/
2008/06/30/
FacebookNeedleInAHaystackEfficientStorag
eOfBillionsOfPhotos.aspx
2009 12 2