Akka Persistence and Eventuate

10.384 Aufrufe

Veröffentlicht am

A CQRS/ES tools comparison

Veröffentlicht in: Software
0 Kommentare
36 Gefällt mir
Statistik
Notizen
  • Als Erste(r) kommentieren

Keine Downloads
Aufrufe
Aufrufe insgesamt
10.384
Auf SlideShare
0
Aus Einbettungen
0
Anzahl an Einbettungen
4.092
Aktionen
Geteilt
0
Downloads
144
Kommentare
0
Gefällt mir
36
Einbettungen 0
Keine Einbettungen

Keine Notizen für die Folie

Akka Persistence and Eventuate

  1. 1. Akka Persistence and Eventuate A CQRS/ES tool comparison Martin Krasser (@mrt1nz) Freelance software developer/architect
  2. 2. History Eventsourced Akka Persistence Eventuate 7.2012 9.2013 1.2015
  3. 3. Similarities
  4. 4. Basis technologies Scala Akka actors Akka streams (Pluggable) storage backends
  5. 5. Akka actor Actor State Behavior Mailbox transient non-durable
  6. 6. Akka Persistence and Eventuate Persist actor state via event sourcing
  7. 7. Event sourcing Capture all changes to application state as sequence of events Store events in event log (append- only) Replay events to recover application state
  8. 8. Event sourcing Commands - actions, can fail Events - immutable facts
  9. 9. Event-sourced actor Actor State Behavior Mailbox Command handler Event handler
  10. 10. Command processing Event-sourced actor State Command handler Event handler Event log
  11. 11. Command processing Event-sourced actor State Command handler Event handler Command Event log
  12. 12. Command processing Event-sourced actor State Command handler Event handler Command Event log Events
  13. 13. Command processing Event-sourced actor State Command handler Event handler Command Event log Events Events
  14. 14. Command processing Event-sourced actor State Command handler Event handler Command Event log Events Events
  15. 15. Command processing Event-sourced actor State Command handler Event handler Command Reply Event log Events Events
  16. 16. State recovery Event-sourced actor State Command handler Event handler Event log
  17. 17. State recovery Event-sourced actor State Command handler Event handler Event log Events (replay)
  18. 18. State recovery Event-sourced actor State Command handler Event handler Event log Events (replay)
  19. 19. Akka Persistence and Eventuate Separate command processing from query processing
  20. 20. CQRS Command Query
 Responsibility Segregation Different data models for command processing and query processing
  21. 21. CQRS Write model for command processing Read model for query
 processing
  22. 22. CQRS Can be well combined with event sourcing (CQRS/ES)
  23. 23. CQRS/ES Write model Event log Command Query Events (r/w) Events (r) Command side Query side Read model
  24. 24. Abstractions Command side Query side Akka Persistence PersistentActor PersistentView Source[E,M] Eventuate EventsourcedActor EventsourcedView EventsourcedProcessor Source[E,M] *) work in progress
  25. 25. PersistentActor EventsourcedActor
  26. 26. PersistentActor EventsourcedActor State
  27. 27. PersistentActor EventsourcedActor Command
 handler
  28. 28. PersistentActor EventsourcedActor Event
 handler
  29. 29. PersistentActor EventsourcedActor Snapshot
 handler
  30. 30. Differences Command side
  31. 31. At a glance https://twitter.com/mrt1nz/status/573382831889653760
  32. 32. Consistency Akka Persistence and Eventuate support strong consistency Eventuate additionally supports relaxation to causal consistency
  33. 33. Akka Persistence Enforces strong consistency on command side PersistentActors must be global singletons No actor/state replication
  34. 34. Eventuate EventsourcedActors can be replicated State replication via reliable, asynchronous event replication Replication across locations (availability zones)
  35. 35. Replication Location A Location B Location C
  36. 36. Replication Location A Application state Location B Location C
  37. 37. Replication Events Events Location A Application state Location B Location C
  38. 38. Replication Events Events Location A Application state Location B Location C
  39. 39. Replication Command Location A Location B Location C
  40. 40. Replication Command X Location A Location B Location C
  41. 41. Replication Command X Events Events Location A Location B Location C
  42. 42. Replication Command X X X Events Events Location A Location B Location C
  43. 43. Eventuate Replicas writeable at all locations = multi-master Write-availability during network partitions
  44. 44. Eventuate Write conflicts detection tracking automated resolution interactive resolution
  45. 45. Partition Location A Location B Location C
  46. 46. Partition Command Location A Location B Location C
  47. 47. Partition Command X Location A Location B Location C
  48. 48. Partition Command X Events Location A Location B Location C
  49. 49. Partition Command X XEvents Location A Location B Location C
  50. 50. Partition Command Command X XEvents Location A Location B Location C
  51. 51. Partition Command Command X X Y Events Location A Location B Location C
  52. 52. Partition Command Command X X Y Events Location A Location B Location C
  53. 53. Partition Command Command X X Y EventsEvents Events Location A Location B Location C
  54. 54. Conflict XY XY XY Location A Location B Location C
  55. 55. Resolution XY XY XY Interactive resolution XY => Z Location A Location B Location C
  56. 56. Resolution XY XY Interactive resolution XY => Z Z Location A Location B Location C
  57. 57. Resolution XY XY Interactive resolution XY => Z Z Events Events Location A Location B Location C
  58. 58. Resolution Interactive resolution XY => Z Z Z Z Events Events Location A Location B Location C
  59. 59. Eventuate Strong consistency within location*) Causal consistency across locations Causality is tracked with vector clocks *) relaxation to causal consistency also possible within location
  60. 60. Causal consistency example Distributed chat application FIFO reliable broadcast vs. Causal reliable broadcast
  61. 61. Inspired by ACM article “Don’t Settle for Eventual Consistency” http://queue.acm.org/detail.cfm? id=2610533 Causal consistency example
  62. 62. FIFO rel. broadcast DC1 DC2 DC3
  63. 63. FIFO rel. broadcast - Alice: Lost my wedding ring Alice DC1 DC2 DC3
  64. 64. FIFO rel. broadcast - Alice: Lost my wedding ring - Alice: Lost my wedding ring - Alice: Lost my wedding ring Alice DC1 DC2 DC3
  65. 65. - Alice: Lost my wedding ring - Alice: Lost my wedding ring - Alice: Lost my wedding ring Alice - Alice: Whew, found it upstairs DC1 DC2 DC3 FIFO rel. broadcast
  66. 66. - Alice: Lost my wedding ring - Alice: Lost my wedding ring - Alice: Lost my wedding ring Alice - Alice: Whew, found it upstairs - Alice: Whew, found it upstairs DC1 DC2 DC3 FIFO rel. broadcast
  67. 67. - Alice: Lost my wedding ring - Alice: Lost my wedding ring - Alice: Lost my wedding ring Alice - Alice: Whew, found it upstairs - Alice: Whew, found it upstairs DC1 DC2 DC3 FIFO rel. broadcast
  68. 68. - Alice: Lost my wedding ring - Alice: Lost my wedding ring - Alice: Lost my wedding ring Bob - Alice: Whew, found it upstairs - Alice: Whew, found it upstairs - Bob: I’m glad to hear that DC1 DC2 DC3 FIFO rel. broadcast
  69. 69. - Alice: Lost my wedding ring - Alice: Lost my wedding ring - Alice: Lost my wedding ring Bob - Alice: Whew, found it upstairs - Alice: Whew, found it upstairs - Bob: I’m glad to hear that - Bob: I’m glad to hear that DC1 DC2 DC3 FIFO rel. broadcast
  70. 70. - Alice: Lost my wedding ring - Alice: Lost my wedding ring - Alice: Lost my wedding ring - Alice: Whew, found it upstairs - Alice: Whew, found it upstairs - Bob: I’m glad to hear that - Bob: I’m glad to hear that - Bob: I’m glad to hear that DC1 DC2 DC3 Bob FIFO rel. broadcast
  71. 71. - Alice: Lost my wedding ring - Alice: Lost my wedding ring - Alice: Lost my wedding ring - Alice: Whew, found it upstairs - Alice: Whew, found it upstairs - Bob: I’m glad to hear that - Bob: I’m glad to hear that - Bob: I’m glad to hear that DC1 DC2 DC3 FIFO rel. broadcast Causality violation!
  72. 72. - Alice: Lost my wedding ring - Alice: Lost my wedding ring - Alice: Lost my wedding ring - Alice: Whew, found it upstairs - Alice: Whew, found it upstairs - Bob: I’m glad to hear that - Bob: I’m glad to hear that - Bob: I’m glad to hear that - Alice: Whew, found it upstairs DC1 DC2 DC3 FIFO rel. broadcast Retry Causality violation!
  73. 73. Causal rel. broadcast - Alice: Lost my wedding ring - Alice: Lost my wedding ring - Alice: Lost my wedding ring - Alice: Whew, found it upstairs - Alice: Whew, found it upstairs DC1 DC2 DC3
  74. 74. Causal rel. broadcast - Alice: Lost my wedding ring - Alice: Lost my wedding ring - Alice: Lost my wedding ring Bob - Alice: Whew, found it upstairs - Alice: Whew, found it upstairs - Bob: I’m glad to hear that DC1 DC2 DC3
  75. 75. Causal rel. broadcast - Alice: Lost my wedding ring - Alice: Lost my wedding ring - Alice: Lost my wedding ring Bob - Alice: Whew, found it upstairs - Alice: Whew, found it upstairs - Bob: I’m glad to hear that Causality
 check DC1 DC2 DC3
  76. 76. Causal rel. broadcast - Alice: Lost my wedding ring - Alice: Lost my wedding ring - Alice: Lost my wedding ring Bob - Alice: Whew, found it upstairs - Alice: Whew, found it upstairs - Bob: I’m glad to hear that - Bob: I’m glad to hear that Causality
 check DC1 DC2 DC3
  77. 77. Causal rel. broadcast - Alice: Lost my wedding ring - Alice: Lost my wedding ring - Alice: Lost my wedding ring Bob - Alice: Whew, found it upstairs - Alice: Whew, found it upstairs - Bob: I’m glad to hear that - Bob: I’m glad to hear that Causality
 check Causality
 check DC1 DC2 DC3
  78. 78. Causal rel. broadcast - Alice: Lost my wedding ring - Alice: Lost my wedding ring - Alice: Lost my wedding ring Bob - Alice: Whew, found it upstairs - Alice: Whew, found it upstairs - Bob: I’m glad to hear that - Bob: I’m glad to hear that - Bob: I’m glad to hear that Buffer Causality
 check Causality
 check DC1 DC2 DC3
  79. 79. - Alice: Lost my wedding ring - Alice: Lost my wedding ring - Alice: Lost my wedding ring - Alice: Whew, found it upstairs - Alice: Whew, found it upstairs - Bob: I’m glad to hear that - Bob: I’m glad to hear that - Bob: I’m glad to hear that Buffer Causality
 check Retry DC1 DC2 DC3 Causal rel. broadcast
  80. 80. - Alice: Lost my wedding ring - Alice: Lost my wedding ring - Alice: Lost my wedding ring - Alice: Whew, found it upstairs - Alice: Whew, found it upstairs - Bob: I’m glad to hear that - Bob: I’m glad to hear that - Bob: I’m glad to hear that Buffer Causality
 check - Alice: Whew, found it upstairs Retry DC1 DC2 DC3 Causal rel. broadcast
  81. 81. - Alice: Lost my wedding ring - Alice: Lost my wedding ring - Alice: Lost my wedding ring - Alice: Whew, found it upstairs - Alice: Whew, found it upstairs - Bob: I’m glad to hear that - Bob: I’m glad to hear that - Bob: I’m glad to hear that Buffer Causality
 check - Bob: I’m glad to hear that - Alice: Whew, found it upstairs Retry DC1 DC2 DC3 Causal rel. broadcast
  82. 82. - Alice: Lost my wedding ring - Alice: Lost my wedding ring - Alice: Lost my wedding ring - Alice: Whew, found it upstairs - Alice: Whew, found it upstairs - Bob: I’m glad to hear that - Bob: I’m glad to hear that - Bob: I’m glad to hear that - Alice: Whew, found it upstairs Causal re-ordering DC1 DC2 DC3 Causal rel. broadcast
  83. 83. Causal consistency Causally related events are delivered in same order at all locations Concurrent events can be delivered in any order at different locations
  84. 84. Differences Event log
  85. 85. “Local” event log Eventuate - LevelDB - Cassandra Location A Location B Location C
  86. 86. “Local” event log “Replicated” event log Eventuate - LevelDB - Cassandra Location A Location B Location C - LevelDB - Cassandra EventsEvents Events
  87. 87. Local event log Storage order consistent with
 causal order Given the potential causality relation ➞ of events Local event logs are linear extensions of ➞
  88. 88. Replicated event log Same causal event storage order 
 at different locations Different total event storage order 
 at different locations
  89. 89. Akka Persistence - LevelDB - Cassandra - EventStore - MongoDB - Kafka - … Location A Location B Location C
  90. 90. Differences Event collaboration
  91. 91. Akka Persistence PA PA PA PA Event log: private for writing shared for reading (query side)
  92. 92. Eventuate EA EA EA Event log: shared for writing shared for reading EA
  93. 93. Event routing EA EA EA
  94. 94. Event routing EA EA EA e1
  95. 95. Event routing EA EA EA e1 e1
  96. 96. Event routing EA EA EA e1 e1 All Topic Aggregate id …
  97. 97. Event routing EA EA EA e1 e1 EAEA e1 EA e1 Location A Location B Location C
  98. 98. Event collaboration State replication Service interaction
  99. 99. Event collaboration S2 S1 S1 S3 Location A Location B Location C
  100. 100. Event collaboration S2 S1 e1 S1 S3 Location A Location B Location C
  101. 101. Event collaboration S2 S1 e1 S1 e1 S3 Location A Location B Location C
  102. 102. Event collaboration S2 S1 e1 e1 S1 e1 S3 e1 Location A Location B Location C
  103. 103. Event collaboration S2 S1 e1 e1 S1 e1 S3 e1 Location A Location B Location C e2
  104. 104. Event collaboration S2 S1 e1 e1 S1 e1 S3 e1 Location A Location B Location C e2 e2
  105. 105. Event collaboration Reliable, distributed and partition- tolerant business processes Event-driven (micro-)service architectures
  106. 106. Event collaboration Reliable event delivery Causal event delivery De-duplicated event stream
  107. 107. Differences Query side
  108. 108. Akka Persistence 2.3 PA PA PV PV PV PersistentViews on single PersistentActor
 (default)
  109. 109. Akka Persistence 2.4 PA PA S S S Stream Sources for multiple PersistentActors
 (requires storage plugin support)
  110. 110. Eventuate EV EV EventsourcedViews on multiple EventsourcedActors
 (default, shared local log) EA EA EA
  111. 111. Eventuate EV EV EventsourcedViews on multiple EventsourcedActors
 (default, shared replicated log) EA EA EA EA EAEA Location A Location B Location C
  112. 112. Consistency in CQRS Eventual consistency Causal consistency Akka Persistence usually per PA (plugin-specific) Eventuate
  113. 113. Causal consistency Single EA/EV: default Across EA/EV: conditional requests
  114. 114. Example scenario Update a write model (EA) Query a read model (EV) Query result should include effect that was caused by the update
  115. 115. Conditional request EA Location A EV Location B
  116. 116. Conditional request EA Location A EV Location B cmd
  117. 117. Conditional request EA Location A EV Location B evt cmd
  118. 118. Conditional request EA Location A EV Location B evt evt(vts) cmd vts = vector timestamp (a “condition”)
  119. 119. Conditional request EA Location A EV Location B evt evt(vts) cmd reply(vts) vts = vector timestamp (a “condition”)
  120. 120. Conditional request EA Location A EV Location B evt evt(vts) cmd CR(query, vts)reply(vts) vts = vector timestamp (a “condition”)
  121. 121. Conditional request EA Location A EV Location B evt evt(vts) cmd CR(query, vts)reply(vts) vts = vector timestamp (a “condition”) delay
  122. 122. Conditional request EA Location A EV Location B evt evt(vts) cmd CR(query, vts)reply(vts) vts = vector timestamp (a “condition”) delay
  123. 123. Conditional request EA Location A EV Location B evt evt(vts)evt(vts) cmd CR(query, vts)reply(vts) vts = vector timestamp (a “condition”) delay
  124. 124. Conditional request EA Location A EV Location B evt evt(vts)evt(vts) cmd CR(query, vts)reply(vts) vts = vector timestamp (a “condition”) deliver
  125. 125. Conditional request EA Location A EV Location B evt evt(vts)evt(vts) cmd CR(query, vts)reply(vts) vts = vector timestamp (a “condition”) reply deliver
  126. 126. Akka Distributed Data and Eventuate CRDTs
  127. 127. CRDTs Conflict-free Replicated Data Types Automated resolution of write conflicts
  128. 128. CRDTs Akka distributed data: CvRDTs
 (convergent or state-based) Eventuate: CmRDTs
 (commutative or operation-based)
  129. 129. CRDTs CvRDTs CmRDTs Message payload Current state Operations Communication middleware No guarantees Reliable broadcast (idempotent) Change history CvRDT intern Persistent log
  130. 130. Specification “A comprehensive study of Convergent and Commutative Replicated Data Types” http://hal.upmc.fr/docs/00/55/55/88/ PDF/techreport.pdf
  131. 131. Akka CvRDTs 10 implemented from specification
 (incl. counters, registers, sets, maps) In-memory only (non-durable) Custom data types can be defined
  132. 132. Eventuate CmRDTs 4 implemented from specification
 (Counter, MV/LWW-register, OR-Set) Durable via event sourcing Eventuate CRDT framework
  133. 133. Documentation http://rbmhtechnology.github.io/eventuate/ http://doc.akka.io/docs/akka/2.4.0/scala/ persistence.html http://doc.akka.io/docs/akka/2.4.0/scala/ persistence-query.html http://doc.akka.io/docs/akka/2.4.0/scala/ distributed-data.html
  134. 134. Thank you!

×