Beyond Strong Consistency

1. Beyond Strong
Consistency
Models for Consistency in High Performance
Distributed Systems
John L. Singleton
University of Central Florida

2. A Motivating Example
“Represents”
What Happens if we delete a friend from my social graph?

3. The Problem of Scalability
del(fn)
query(f)
Strong Consistency: With a single database,
all users see the update after it is committed

4. The Problem of Scalability (continued)
Strong Consistency: As more
database servers are added to
handle the load, update latency
increases
DB1
query(f)
del(fn)
query(f)
DBn
sync(f)

5. The Problem: Scaling a Model of
Consistency
• Strong Consistency requires that every update is immediately
propagated to all nodes.
• All users have the same view of the data at any moment.
• However, this is bad for performance and is hard to scale.
How can we implement a model of consistency that scales?

6. The CAP “Theorem”
• Uttered by Eric Brewer at PODC circa 2000
• Consistency: All users see the same view of the data
• Availability: The system is always available
• Partition Tolerance: The system can operate in the presence
of network failures between nodes.
• Not really a theorem because no proof of its validity exists.

8. Some Criticism of the CAP Theorem
• In the world of NoSQL databases, CAP has been used as an
excuse to give up consistency
• However, the cases they are optimizing for (for example,
partition tolerance) are in fact the most rare
• Network partition in a local cluster
• Failure of a single node is much more common
“One should not throw out the C so quickly, since there are real error
scenarios where CAP does not apply and it seems like a bad tradeoff in
many of the other situations.” – Michael Stonebraker

9. One Possible Solution: Eventual
Consistency
Asks the question: “What if we omit the C?”
• Do updates locally and propagate the changes
• No foreground synchronization
• Resists partitioning
• Requires consensus
• Consistency is eventual, no definite point in the future.

10. Some Preliminaries: State-Based Objects
• Two Views of an Object: State Based and Operation Based
• State Based: Object is a collection of state transitions

11. The State-Based View

12. Causal History of State-Based Objects
• We need an abstraction to reason about the state an object
may be in (or may have come from)
• Each replica can maintain a copy of the causal history.

13. Definition of Eventual Consistency
Eventual Delivery: An update delivered at some correct replica
is eventually delivered to all correct replicas:
Convergence: Correct replicas that have delivered the same
updates eventually reach equivalent state:
Termination: All method executions terminate

14. Problems with Eventual Consistency
The Strong Consistency Model
• In the strong consistency model, users get a
single view in a specific order, therefore no
conflicts
a
s2.add(a,b) s1.del(a)
b
time (increasing)

15. Problems with Eventual Consistency
The Eventually Consistency Model
• This update is invalid!
• Measures must be taken to handle conflicts
and perform rollbacks to ensure consistency
• This results in overhead to the system and
should be avoided
a
s2.del(a) s1.add(a,b)
b
time (increasing)

16. Eventual Delivery: An update delivered at some correct replica
is eventually delivered to all correct replicas:
Convergence: Correct replicas that have delivered the same
updates eventually reach equivalent state:
Termination: All method executions terminate
Strong Convergence: Correct replicas that have delivered the
same updates have equivalent state:

17. Strong Eventual Consistency
• Strong Eventual Consistency is a stronger subset of Eventual
Consistency
• By placing restrictions on the way in which replicated types are
constructed, we can provide stronger guarantees.
• Most importantly: SEC is a solution to the CAP problem
• Can tolerate up to n-1 concurrent crashes
• Does not require consensus
• Is weaker than strong consistency, but provides performance benefits
and a well defined consistency model.
• Removes conflicts

18. Conflict Free Replicated Data Types
• The key to Strong Eventual Consistency is the Convergent
Replicated Data Type (CvRDT)
• Constructed with a monotonic semi-lattice object
• A set with a partial order
• And an operation that can find the least upper bound on any two
objects in the set such that:

19. CvRDTs and Monotonic Semi-Lattices
If these conditions are satisfied, you have a CvRDT
(technical note: State-Based RDTs are equivalent to
Operation Based RDTs)

20. Sample CvRDT: A Integer Vector

21. Set-Based CvRDT
• How could we apply this idea to a more complex object like a
set?
• Create one set, the Addition set, and record all elements added
• Create another set, the Remove set, and record all elements
removed.
• Both sets are monotonic
• Define the query function as:

22. Conclusions
• Introduced the idea of multiple models of consistency:
• Strong Consistency
• Eventual Consistency
• Strong Eventual Consistency
• Discussed the CAP Theorem and showed how
performance problems with Strong Consistency lead to
Eventual Consistency
• Demonstrated how problems with Eventual Consistency
can arise
• Showed how Strong Eventual Consistency is a solution to
the CAP problem, and can improve the performance and
reliability of distributed data store systems.

23. References
[1] Deftu, Andrei, and Jan Griebsch. "A Scalable Conflict-Free Replicated Set Data Type."
In Distributed Computing Systems (ICDCS), 2013 IEEE 33rd International Conference on, pp. 186-
195. IEEE, 2013.
[2] Shapiro, Marc, Nuno Preguiça, Carlos Baquero, and Marek Zawirski. "Conflict-free replicated data
types." In Stabilization, Safety, and Security of Distributed Systems, pp. 386-400. Springer Berlin
Heidelberg, 2011.
[3] Bouajjani, Ahmed, Constantin Enea, and Jad Hamza. "Verifying eventual consistency of optimistic
replication systems." In Proceedings of the 41st annual ACM SIGPLAN-SIGACT symposium on
Principles of programming languages, pp. 285-296. ACM, 2014.
[4] Vogels, Werner. "Eventually consistent." Communications of the ACM 52, no. 1 (2009): 40-44.
[5] Stonebraker, Michael. "Errors in database systems, eventual consistency, and the cap
theorem." Communications of the ACM, BLOG@ ACM (2010).
[6] Brewer, Eric A. "Towards robust distributed systems." In PODC, p. 7. 2000.
[7] Lamport, Leslie. "Time, clocks, and the ordering of events in a distributed
system." Communications of the ACM 21, no. 7 (1978): 558-565.
[8] Gilbert, Seth, and Nancy Lynch. "Brewer’s Conjecture and the Feasibility of Consistent." Available,
Partition-Tolerant Web Services. doi 10, no. 564585.564601 (2002).

24. Questions?