Consensus in distributed systems has been a debated topic every since programmers discovered they could run the same program on multiple machines. Researchers have been studying consensus for decades, resulting in numerous algorithms and white papers. Unfortunately, many of these algorithms are flawed and unreliable. However, in 2011, a team of researchers published a paper on a novel approach to distributed consensus using Conflict-free Replicated Data Types (https://hal.inria.fr/inria-00609399v1...). This paper created quite a buzz as it showed that CRDTs were mathematically proven to guarantee consensus through "Strong Eventual Consistency." They also claimed to have solved the CAP conundrum. This presentation dives into this seminal paper in order to answer the hard questions. What are CRDTs? How do they work? And most importantly, does it actually solve CAP? By the end of this talk, everyone in the audience will have a foundational understanding of CRDTs and how they can be applied to their own work. Best of all, I will be explaining all of this is as simple language as possible. No advanced math degree required! Sound too good to be true? You'll just have to come see for yourself!

1. Guaranteeing Consensus in
Distributed Systems with CRDTs
Sun-Li Beatteay
Paper: http://bit.ly/2011-crdt-paper
@SunnyPaxos

2. About Me
● Software Engineer @
DigitalOcean
● Technical Writer:
@SunnyB on Medium
● GitHub: sunny-b
● Twitter: @SunnyPaxos
@SunnyPaxos

3. @SunnyPaxos

4. CRDTs often explained using dense language
and complicated proofs
@SunnyPaxos

5. Explaining CRDTs in simple terms
@SunnyPaxos

6. But first, some history
@SunnyPaxos

7. Consensus in Distributed Systems
Image Credit: Shubheksha@SunnyPaxos

8. Image Credit: Martin Kleppmann@SunnyPaxos

9. @SunnyPaxos

10. Eventual
Consistency
@SunnyPaxos

11. Image Credit: Martin Kleppmann@SunnyPaxos

12. @SunnyPaxos

13. Possible consensus algorithms?
@SunnyPaxos

14. Possible consensus algorithms
● Manual override conflicts/rollback (Git)
@SunnyPaxos

15. Possible consensus algorithms
● Manual override conflicts/rollback (Git)
● Pick one set of changes, throw out the rest (Blockchain)
@SunnyPaxos

16. Possible consensus algorithms
● Manual override conflicts/rollback (Git)
● Pick one set of changes, throw out the rest (Blockchain)
● Dynamically merge conflicts based on user intention
@SunnyPaxos

17. Image Credit: Martin Kleppmann
@SunnyPaxos

18. Algorithms for (dynamic) convergence
Image Credit: Martin Kleppmann
@SunnyPaxos

19. Image Credit: Martin Kleppmann@SunnyPaxos

20. Algorithms for (dynamic) convergence
Image Credit: Martin Kleppmann
@SunnyPaxos

21. What are CRDTs?
@SunnyPaxos

22. CRDT -> “Data Types”
● Objects that store and
model data
● Any data structure can
be a CRDT
@SunnyPaxos

23. CRDT -> “Replicated”
● Designed for decentralization
● Maintain the same state
● Async communication
@SunnyPaxos

24. CRDT -> “Conflict-free”
● All updates merge without conflict
● No need for central authority
@SunnyPaxos

25. Types of CRDT communication
@SunnyPaxos

26. State-Based CRDT (Convergent)
@SunnyPaxos

27. State-Based CRDT (Convergent)
@SunnyPaxos
Node 1
State: 4
Node 1 Node 2
State: 4
Add(1)

28. State-Based CRDT (Convergent)
@SunnyPaxos
Node 1
State: 5
Node 1 Node 2
State: 4
Add(1) 5

29. State-Based CRDT (Convergent)
@SunnyPaxos
Node 1
State: 5
Node 1 Node 2
State: 5
Add(1) 5

30. Operation-Based CRDT (Commutative)
@SunnyPaxos

31. @SunnyPaxos
Node 1
State: 4
Node 1 Node 2
State: 4
Add(1)
Operation-Based CRDT (Commutative)

32. @SunnyPaxos
Node 1
State: 5
Node 1 Node 2
State: 4
Add(1) Add(1)
Operation-Based CRDT (Commutative)

33. @SunnyPaxos
Node 1
State: 5
Node 1 Node 2
State: 5
Add(1) Add(1)
Operation-Based CRDT (Commutative)

34. How do CRDTs guarantee
consensus?
@SunnyPaxos

35. Strong Eventual Consistency (SEC)
@SunnyPaxos

36. To understand SEC, we
need to know EC
@SunnyPaxos

37. Eventual Consistency
@SunnyPaxos

38. Eventual Consistency
@SunnyPaxos

39. Eventual Consistency
@SunnyPaxos

40. Eventual Consistency
Node 1 Node 2
State: {}State: {}
@SunnyPaxos

41. Eventual Consistency
Node 1 Node 2
State: {}1. Add(“a”)
State: {“a”}
@SunnyPaxos

42. Eventual Consistency
Node 1 Node 2
State: {}1. Add(“a”)
2. Add(“b”)
State: {“a”, “b”}
@SunnyPaxos

43. Eventual Consistency
Node 1 Node 2
State: {}1. Add(“a”)
2. Add(“b”)
3. Remove(“a”)
State: {“b”}
@SunnyPaxos

44. Eventual Consistency
Node 1 Node 2
State: {}1. Add(“a”)
2. Add(“b”)
3. Remove(“a”)
4. Remove(“b”)
State: {}
@SunnyPaxos

45. Eventual Consistency
Node 1 Node 2
1. Remove(“a”)
State: {}
1. Add(“a”)
2. Add(“b”)
3. Remove(“a”)
4. Remove(“b”)
State: {}
@SunnyPaxos

46. Eventual Consistency
Node 1 Node 2
1. Remove(“a”)
2. Add(“a”)
State: {“a”}
1. Add(“a”)
2. Add(“b”)
3. Remove(“a”)
4. Remove(“b”)
State: {}
@SunnyPaxos

47. Eventual Consistency
Node 1 Node 2
1. Remove(“a”)
2. Add(“a”)
3. Add(“b”)
State: {“a”, “b”}
1. Add(“a”)
2. Add(“b”)
3. Remove(“a”)
4. Remove(“b”)
State: {}
@SunnyPaxos

48. Eventual Consistency
Node 1 Node 2
1. Remove(“a”)
2. Add(“a”)
3. Add(“b”)
4. Remove(“b”)
State: {“a”}
1. Add(“a”)
2. Add(“b”)
3. Remove(“a”)
4. Remove(“b”)
State: {}
@SunnyPaxos

49. Eventual Consistency
Node 1 Node 2
Consensus needed@SunnyPaxos
1. Add(“a”)
2. Add(“b”)
3. Remove(“a”)
4. Remove(“b”)
State: {}
1. Remove(“a”)
2. Add(“a”)
3. Add(“b”)
4. Remove(“b”)
State: {“a”}

50. @SunnyPaxos

51. @SunnyPaxos

52. Strong Eventual Consistency
All replicas that have seen the same set of updates will
have the same state.
@SunnyPaxos

53. 1. Add(“a”)
2. Add(“b”)
3. Remove(“a”)
4. Remove(“b”)
State: {}
Strong Eventual Consistency
Node 1 Node 2
@SunnyPaxos
=
1. Remove(“b”)
2. Add(“a”)
3. Remove(“a”)
4. Add(“b”)
State: {}

54. @SunnyPaxos

55. @SunnyPaxos

56. In order to learn how CRDTs
guarantee SEC...
@SunnyPaxos

57. Let’s build a CRDT!
@SunnyPaxos

58. Building a distributed Set
● A list of elements where each element is unique
@SunnyPaxos

59. Building a distributed Set
● A list of elements where each element is unique
● Operations
○ Query
○ Add
○ Remove
@SunnyPaxos

60. Building a distributed Set
● A list of elements where each element is unique
● Operations
○ Query
○ Add
○ Remove
{ 1, 2, 3, … }
@SunnyPaxos

61. ● Operation-based replication (better network
scalability)
Building a distributed Set
@SunnyPaxos

62. How do we ensure SEC?
@SunnyPaxos

63. @SunnyPaxos
● “...semilattice property is SEC.”

64. @SunnyPaxos

65. @SunnyPaxos

66. @SunnyPaxos

67. @SunnyPaxos

68. @SunnyPaxos

69. @SunnyPaxos

70. Three Requirements for SEC
@SunnyPaxos

71. Three Requirements for SEC
● Commutativity
A given set of operations will always result in same state, no matter their
order.
Example: 1 + 5 = 5 + 1
@SunnyPaxos

72. Three Requirements for SEC
● Commutativity (1 + 5 = 5 + 1)
● Idempotency
Duplicate operations result in the same state
Example: x * 1 = x
@SunnyPaxos

73. ● Commutativity (1 + 5 = 5 + 1)
● Idempotency (x * 1 = x)
● Associativity or Causality
Three Requirements for SEC
@SunnyPaxos

74. Associativity (State-based)
CRDTs merge to the same state no matter how they
are grouped.
@SunnyPaxos

75. Causality (Operation-based)
● Operations are received in the correct Cause-Effect order
● Example: Create must come before Remove
@SunnyPaxos

76. Three Requirements for consensus
● Commutativity (1 + 5 = 5 + 1)
● Idempotency (x * 1 = x)
● Associativity or Causality
○ Associativity -> State-based replication
○ Causality -> Operation-based replication
@SunnyPaxos

77. Back to our CRDT
@SunnyPaxos

78. Three Requirements for SEC
● Commutativity?
@SunnyPaxos

79. {1, 2, 3}{1, 2, 3}
Node 1 Node 2Commutativity
@SunnyPaxos

80. {1, 2, 3}{1, 2, 3}
Node 1 Node 2
Remove(2)Add(4)
Commutativity
{1, 2, 3, 4} {1, 3}
@SunnyPaxos

81. {1, 2, 3}{1, 2, 3}
Node 1 Node 2
Remove(2)Add(4)
Commutativity
{1, 2, 3, 4} {1, 3}
Add(4)Remove(2)
{1, 3, 4} {1, 3, 4}
@SunnyPaxos

82. {1, 2, 3}{1, 2, 3}
Node 1 Node 2
Remove(2)Add(4)
Commutativity
{1, 2, 3, 4} {1, 3}
Add(4)Remove(2)
{1, 3, 4} {1, 3, 4}
@SunnyPaxos

83. Three Requirements for SEC
● Commutativity
● Idempotency?
@SunnyPaxos

84. {1, 2, 3}{1, 2, 3}
Node 1 Node 2Idempotency
@SunnyPaxos

85. {1, 2, 3}{1, 2, 3}
Node 1 Node 2
Remove(2)Remove(2)
Idempotency
{1, 3} {1, 3}
@SunnyPaxos

86. {1, 2, 3}{1, 2, 3}
Node 1 Node 2
Remove(2)Remove(2)
Idempotency
{1, 3} {1, 3}
Remove(2)Remove(2)
{1, 3} {1, 3}
@SunnyPaxos

87. {1, 2, 3}{1, 2, 3}
Node 1 Node 2
Remove(2)Remove(2)
Idempotency
{1, 3} {1, 3}
Remove(2)Remove(2)
{1, 3} {1, 3}
@SunnyPaxos

88. Three Requirements for SEC
● Commutativity
● Idempotency
● Casuality?
@SunnyPaxos

89. Node 1 Node 2
@SunnyPaxos
Causality
1. Remove(“b”)
2. Add(“a”)
3. Remove(“a”)
4. Add(“b”)
State: {“b”}
1. Add(“a”)
2. Add(“b”)
3. Remove(“a”)
4. Remove(“b”)
State: {}

90. Node 1 Node 2
@SunnyPaxos
Causality
1. Remove(“b”)
2. Add(“a”)
3. Remove(“a”)
4. Add(“b”)
State: {“b”}
1. Add(“a”)
2. Add(“b”)
3. Remove(“a”)
4. Remove(“b”)
State: {}

91. Adding Lamport Clocks and Version
Vectors
@SunnyPaxos

92. Lamport Clocks
A: 0
Node A
NodeID Timestamp
@SunnyPaxos

93. Lamport Clocks
A: 1
Node A
Add(5)
@SunnyPaxos

94. Lamport Clocks
A: 2
Node A
Remove(5)
@SunnyPaxos

95. Version Vector
Node A Node B
@SunnyPaxos
Node Count
A 0
B 0
Node Count
A 0
B 0

96. Version Vector
Node A Node B
@SunnyPaxos
Node Count
A 1
B 0
Node Count
A 0
B 0
Add(1)
NodeID: A
Count: 1

97. Version Vector
Node A Node B
@SunnyPaxos
Node Count
A 1
B 0
Node Count
A 1
B 0
Add(1)
NodeID: A
Count: 1

98. { 10, 11, ... }
So instead of a Set like this...
@SunnyPaxos

99. { (10, {A: 1}), (11, {B: 1}), ... }
...it will look like this
@SunnyPaxos

100. { (10, {A: 1}), (11, {B: 1}), ... }
@SunnyPaxos
Element Value
...it will look like this

101. { (10, {A: 1}), (11, {B: 1}), ... }
@SunnyPaxos
Element Value Lamport
Timestamp
...it will look like this

102. Node A
Node B Node C
A: 0
B: 0
Version Vector
Node Count
A 0
B 0
@SunnyPaxos
State: {}

103. Node A
Node B Node C
Add(5)
- NodeID: A
- Counter: 1A: 1
B: 0
Version Vector
Node Count
A 0
B 0
@SunnyPaxos
State: { (5, {A: 1} }

104. Node A
Node B Node C
Remove(5)
- NodeID: B
- Counter: 1
A: 1
B: 1
Add(5)
- NodeID: A
- Counter: 1 Version Vector
Node Count
A 0
B 0
@SunnyPaxos
State: {}

105. Node A
Node B Node C
Add(5)
- NodeID: A
- Counter: 1
Remove(5)
- NodeID: B
- Counter: 1
Version Vector
Node Count
A 0
B 0
A: 1
B: 1
@SunnyPaxos
State: {}

106. Node A
Node B Node C
Version Vector
Node Count
A 0
B 0
Remove(5)
- NodeID: B
- Counter: 1
Add(5)
- NodeID: A
- Counter: 1
A: 1
B: 1
@SunnyPaxos
State: {}

107. Node C
Version Vector
Node Count
A 0
B 0
Remove(5)
- NodeID: B
- Counter: 1
@SunnyPaxos

108. Node C
Version Vector
Node Count
A 0
B 0
Remove(5)
- NodeID: B
- Counter: 1
@SunnyPaxos

109. Node C
Version Vector
Node Count
A 0
B 0
Remove(5)
- NodeID: B
- Counter: 1
@SunnyPaxos

110. Node C
Version Vector
Node Count
A 0
B 0
Remove(5)
- NodeID: B
- Counter: 1
(5, {A: 1})
@SunnyPaxos

111. Node C
Version Vector
Node Count
A 0
B 0
Remove(5)
- NodeID: B
- Counter: 1
(5, {A: 1})
@SunnyPaxos

112. Node C
Version Vector
Node Count
A 1
B 0
Remove(5)
- NodeID: B
- Counter: 1
(5, {A: 1})
@SunnyPaxos

113. Node C
Version Vector
Node Count
A 1
B 1
Remove(5)
- NodeID: B
- Counter: 1
(5, {A: 1})
@SunnyPaxos

114. Node A
Node B Node C
Version Vector
Node Count
A 1
B 1
Add(5)
- NodeID: A
- Counter: 1
A: 1
B: 1
@SunnyPaxos
State: {}

115. Node A
Node B Node C
Version Vector
Node Count
A 1
B 1
Add(5)
- NodeID: A
- Counter: 1
A: 1
B: 1
@SunnyPaxos
State: {}

116. Node A
Node B Node C
Version Vector
Node Count
A 1
B 1
Add(5)
- NodeID: A
- Counter: 1
A: 1
B: 1
@SunnyPaxos
State: {}

117. Node A
Node B Node C
Version Vector
Node Count
A 1
B 1
A: 1
B: 1
Consensus State: {}
@SunnyPaxos

118. Causality achieved!
@SunnyPaxos

119. Commutativity
+ Idempotency
+ Causality
= Strong Eventual Consistency!
@SunnyPaxos

120. CRDT Drawbacks
● Scale
○ Inefficient memory
● Network
○ Heavy traffic
○ Causal delivery “required”
● Effort
○ Lack of available libraries
@SunnyPaxos

121. CRDTs in the Wild
@SunnyPaxos

122. CRDTs in the Wild
@SunnyPaxos

123. CRDTs in a nutshell
● Decentralized data store
● Strong Eventual Consistency
● 3 requirements for SEC
○ Commutativity
○ Idempotency
○ Associativity/Causality
● Any data structure can be a CRDT (w/ enough effort)
@SunnyPaxos

124. References
Papers
● http://bit.ly/2011-crdt-paper
● http://bit.ly/crdt-types-paper
Videos
● http://bit.ly/kleppman-crdt-talk
● http://bit.ly/shapiro-crdt-talk
@SunnyPaxos @SunnyB http://www.sunli.co
Thanks!