1. Andy Schwerin
Lead Engineer, MongoDB

2. • Goals of Replication
• Replication Architecture
• A representative write

3. • High availability for processing reads and writes
– Automatic leader election
• Support many network topologies
– Tag sets
• Accessible consistency model
– Ordered operation log
• Client can trade latency for durability
– Write concern

5. { ts: 4,
op: “i”,
ns: “d.c”,
o: { _id: 10, name: “john” }
}
OPLOG
…

6. PRIMARY OPLOG
4
SECONDARY OPLOG
8 9
SECONDARY OPLOG
4 5
When a secondary oplog is not a prefix
of the primary oplog…

7. w:?

8. w:1
Could lose write when
primary disappears,
without notification.

9. w:majority
Over half of nodes must
fail to lose the write.
And, an outside
operator must intervene
before new writes are

10. w:all
All nodes have the
write before primary
responds.
But, cannot complete
writes if any nodes are

11. OPLOG
d.c
OPLOG
P TS:6
S1 TS:6
S2 TS:2
d.c. insert ({_id:10,name:’john’}, wC: {w:2}})
1. Fetch oplog entries
2. Apply to collections
3. Write to local oplog
4. Notify primary
5. Repeat

12. OPLOG
OBSERVER
BATCH
BATCH
PREFETCH
APPLIER
BATCH
x.y d.cd.c
OPLOG
d.c. insert ({_id:10,name:’john’}, wC: {w:2}})
P TS:6
S1 TS:6
S2 TS:2

13. OPLOG
d.c.insert ({_id:10,name:’john’}, wC: {w:2}})
d.c
{ ts: 4,
op: “i”,
ns: “d.c”,
o: { _id: 10, name: “john” }
}
P TS:4
S1 TS:2
S2 TS:2

14. OPLOG
d.c.insert ({_id:10,name:’john’}, wC: {w:2}})
OBSERVER
BATCH
d.c
OPLOG
P TS:6
S1 TS:2
S2 TS:2

15. OPLOG
d.c.insert ({_id:10,name:’john’}, wC: {w:2}})
BATCH
d.c
OPLOG
OBSERVER
P TS:6
S1 TS:2
S2 TS:2

16. OBSERVER
BATCH
BATCH
PREFETCH
OPLOG
• Split batch into arbitrary work units
• Assign work to prefetch threads
• Entries processed in any order
• All while admitting readers
Allow readers

17. OBSERVER
BATCH
BATCH
PREFETCH
OPLOG
BATCH
x.y d.c
APPLIER
• Assign entries to workers by
target collection
• Disable schema constraints
Allow readers

18. OBSERVER
BATCH
BATCH
PREFETCH
OPLOG
BATCH
x.y d.c
APPLIER
• Concurrency control excludes
readers
• Oplog entries applied in
timestamp order
Exclude readers

19. OBSERVER
BATCH
BATCH
PREFETCH
OPLOG
BATCH
x.y d.c
APPLIER
Exclude readers• Concurrency control excludes
readers
• Oplog entries applied in
timestamp order

20. OBSERVER
BATCH
BATCH
PREFETCH
OPLOG
BATCH
x.y d.c
APPLIER
Exclude readers• Concurrency control excludes
readers
• Oplog entries applied in
timestamp order

21. OBSERVER
BATCH
BATCH
PREFETCH
APPLIER
BATCH
x.y d.c
OPLOG
• Readmit readers
• Move entries from batch to oplog
• Begin processing next batch
Allow readers

22. OPLOG
OBSERVER
BATCH
BATCH
PREFETCH
APPLIER
BATCH
x.y d.cd.c
OPLOG
Allow readers
P TS:6
S1 TS:6
S2 TS:2

23. OPLOG
d.c
P TS:6
S1 TS:6
S2 TS:2
• Consults list of waiting clients
• Looks for those waiting for ts:6 or
earlier on S1
• Sends acknowledgement!