20. Useful building block: ZooKeeper
● Centralized coordination service
o configuration, service discovery
o locking, queues, barriers
o failure detection
o leader election, membership
● Reliable
● Source of truth
26. BookKeeper : durability service
durability
replication consistency
on commodity hardware
recovery
user library
A building block for reliable systems
27. The ledger abstraction
op op op op op op op op op op opop opop opop op
add
read
checkpoint
Ledger 1
Ledger 2
Ledger 3
28. Guarantees
If an entry
has been acknowledged,
it must be readable
If an entry
is read once,
it must always be readable
29. History
Initial use case : Hadoop name node recovery
2008: open sourced contrib of ZooKeeper
2011: sub-project of ZooKeeper
2012: Production
34. Reliable writes
● store digest along with entry
● fsync each entry before
returning
● ACK when:
○ all previous
entries
○ this entry
accepted
by
quorum
50. Performance considerations
I/O bound
- disk IOPS: ~ 120/s HDD, 500 000/s SSD
- network: 1Gb/s ~ 100MB/s max or less in practice
~ 1KB msgs: 100 000/s
per node