Microservices are seen as the way to simplify complex systems, until you need to coordinate a transaction across services, and in that instant, the dream ends. Transactions involving multiple services can lead to a spaghetti web of interactions. Protocols such as two-phase commit come with complexity and performance bottlenecks. The Saga pattern involves a simplified transactional model. In sagas, a sequence of actions are executed, and if any action fails, a compensating action is executed for each of the actions that have already succeeded. This is particularly well suited to long-running and cross-microservice transactions. In this talk we introduce the new Simple Sagas library (https://github.com/simplesourcing/simplesagas). Built using Kafka streams, it provides a scalable fault tolerance event-based transaction processing engine. We walk through a use case of coordinating a sequence of complex financial transactions. We demonstrate the easy to use DSL, show how the system copes with failure, and discuss this overall approach to building scalable transactional systems in an event-driven streaming context.
2. SIMPLE MACHINES SAGAS IN KAFKA
!2
Journey to Sagas
Coordinating microservices Sagas
Kafka as a database
Events
A protocol for distributed and long running transactions
3. SIMPLE MACHINES SAGAS IN KAFKA
!3
Microservice Transactions
www.
Flight Booking
Hotel Booking
Car Rental Payment
Trips
?
4. SIMPLE MACHINES SAGAS IN KAFKA
!4
Two Phase Commit
www.
Flight Booking
Hotel Booking
Car Rental Payment
Trips
Prepare
Response
Confirm
Acknowledgement
• Too strict
• Too slow
• Too complex
Not that popular in practice…
5. SIMPLE MACHINES SAGAS IN KAFKA
!5
Saga Transaction
www.
Flight Booking
Hotel Booking
Car Rental Payment
Trips
Action
Success Response
• Much simpler protocol
• Minimises latency and comms
• Services don’t need to care
about sagas
6. SIMPLE MACHINES SAGAS IN KAFKA
!6
Saga Failure
www.
Flight Booking
Hotel Booking
Car Rental Payment
Trips
Action
Success Response
Compensation
Failure
work backwards, executing
compensating actions
Compensation Response
7. SIMPLE MACHINES SAGAS IN KAFKA
!7
Saga Data Consistency
SQL Database
A Atomicity
C Consistency
I Isolation
D Durability
Sagas
A Atomicity
C Consistency
I Isolation
D Durability
We give up isolation, but we
can live without it…
8. SIMPLE MACHINES SAGAS IN KAFKA
!8
Kafka and Event Sourcing
Kafka as a database?
Event sourcing
Event processingKafka
Events
WHY NOT STORE
THE DATA LIKE WE
USE IT?
WE’RE ALREADY
USING KAFKA FOR
MESSAGING
9. SIMPLE MACHINES SAGAS IN KAFKA
!9
Event Log Consistency
e.g. Bank Account
Events
Simple Sourcing
• Balance always positive
• Account never debited twice
Taking advantage of
• Kafka exactly once processing
• Per topic-partition ordering guarantee
10. An API to support
these consistency
requirements
11. SIMPLE MACHINES SAGAS IN KAFKA
!11
Simple Sourcing Concepts
Type Represents Example
C Command Intent debit Bob’s account $100
E Event Outcome Bob’s account debited $100
A Aggregate State Bob’s account balance=$700, available=$650
K Key Identity Bob’s account number
Based on CQRS (Command query responsibility segregation)
12. SIMPLE MACHINES SAGAS IN KAFKA
!12
Simple Sourcing Data Flow
Kafka
Command request
Aggregate state
Command response
Event
Bob’s
balance
>$100?
Debit Bob’s account $100
Return Success
Publish event
AccountDebited(bob, 100)
Return error response
Debit Bob’s account $100
Yes
No
Everything happens in a single atomic transaction
13. SIMPLE MACHINES SAGAS IN KAFKA
!13
Simple Sourcing Programming Model
In Scala types
(Command, Aggregate) => Either[Error, List[Event]]
(Event, Aggregate) => Aggregate
Key => Aggregate
COMMAND HANDLER
EVENT AGGREGATOR
INITIAL VALUE
14. SIMPLE MACHINES SAGAS IN KAFKA
!14
Event Sourced Apps
Coordinating between aggregates: Bike auction scenario
Account Aggregate
Key Total Reserved
Bob’s a/c 400 200
Charlie’s a/c 500 –
Auction Aggregate
Key Bidder Amount
Bicycle Bob 200
Reserve Charlie’s account $250
Bid $250 for Charlie
Release $200 for Bob
$200 Current bid: Bob $200
Bob
$250 Next bid: Charlie $250
Charlie
250
Auction Aggregate
Key Bidder Amount
Bicycle Charlie 250
–
15. SIMPLE MACHINES SAGAS IN KAFKA
!15
Auction App
Coordinating between aggregates
Charlie
$250
Account Aggregate
Key Total Reserved
Bob’s a/c 400 –
Charlie’s a/c 500
Alice’s a/c 500
Auction Aggregate
Key Bidder Amount
Bicycle Charlie 250
Reserve Bob’s account $300
Bid $300 for Bob
Undo Bob’s reservation
Reserve Alice’s account $350
Bid $350 for Alice
Release $250 for Charlie
Charlie has the current best bid of $250
Bob
$300 Bob bid’s again, this time $300
Alice
$350 Meanwhile Alice arrives and bids $350
300
350
300
350Alice
Sagas are suitable for synchronising between
multiple aggregates in an event sourced application
-
-250
16. SIMPLE MACHINES SAGAS IN KAFKA
!16
Representing Sagas
A stateful directed acyclic graph (DAG)
a undo
A
b c undo
C
d undo
D
e
a
undo
A
Saga actions
Undo (compensation) actions
18. SIMPLE MACHINES
(Event, Aggregate) => Aggregate
(Command, Aggregate) => Either[Error, List[Event]]
Key => Aggregate
COMMAND HANDLER
EVENT AGGREGATOR
INITIAL VALUE
(_, _) => true
(_, aggregate) => if (aggregate) Error("email taken") else List(EventClaimed)
_ => false
SAGAS IN KAFKA
!18
Useful Patterns
Enforcing uniqueness
Claim Email
Type Example
C Command
claim email address, rejects command if
aggregate is set
A Aggregate either true or false
K Key email address
*Each user must have a unique email
Claim email address:
bob@xyz.com
Create user with email address:
bob@xyz.com
Claim
email
Create
user
release
email
19. SIMPLE MACHINES SAGAS IN KAFKA
!19
Useful Patterns
Pessimistic lock
Claim lock
Perform action on resource
Release lock
Claim lock Create user
release lock
Create userDo actions Release lock
20. SIMPLE MACHINES SAGAS IN KAFKA
!20
Saga State a undo
A
b c undo
C
d undo
D
e
Pending
In progress
Completed
Failed
N/A
• Saga state (in progress, in failure)
• Action state (pending, in progress)
• The actions themselves (for dynamic actions)
a undo
A
b c undo
C
d undo
D
e
a undo
A
b c undo
C
d undo
D
e
a undo
A
b c undo
C
d undo
D
e
a undo
A
b c undo
C
d undo
D
e
a undo
A
b c undo
C
d undo
D
e
a undo
A
b c undo
C
d undo
D
e
a undo
A
b c undo
C
d undo
D
e
a undo
A
b c undo
C
d undo
D
e
a undo
A
b c undo
C
d undo
D
e
21. SIMPLE MACHINES SAGAS IN KAFKA
!21
Saga State a undo
A
b c undo
C
d undo
D
e
Pending
In progress
Completed
Failed
N/A
Failure mode
saga into switches into undo mode
a undo
A
b c undo
C
d undo
D
e
undo
A
b undo
C
undo
D
undo
E
undo
A
b undo
C
undo
D
undo
E
undo
A
b undo
C
undo
D
undo
E
undo
A
b undo
C
undo
D
undo
E
undo
A
b undo
C
undo
D
undo
E
22. SIMPLE MACHINES SAGAS IN KAFKA
!22
Saga DSL
Scala DSL loosely based on Akka Streams
a
b
c
a ~> b ~> c
23. SIMPLE MACHINES SAGAS IN KAFKA
!23
Saga DSL
with undo (compensation) actions
a ~> b.withUndo(undoB) ~> c
a
b
c
undo
B
24. SIMPLE MACHINES SAGAS IN KAFKA
!24
Saga DSL
parallel execution
a undo
A
b c undo
C
d undo
D
e
a.withUndo(undoA) ~> parallel(b, c.withUndo(undoC)) ~> d.withUndo(undoD) ~> e
25. SIMPLE MACHINES SAGAS IN KAFKA
!25
Saga DSL
These are all equivalent:
(a ~> b ~> c) ~> d ~> e
a ~> (b ~> c ~> d) ~> e
a ~> List(b, c, d).inSeries ~> e
(a ~> b) ~> c ~> (d ~> e)
~> is associative
This enables a building
block approach to
constructing sagas
26. SIMPLE MACHINES SAGAS IN KAFKA
!26
Simple Sagas
A simple Kafka-based Sagas framework
Saga coordinator
Action processors
Client
30. SIMPLE MACHINES SAGAS IN KAFKA
!30
Action Processor
{
"sagaId": 1,
"commandId": 34,
"command": "BookFlight(...)",
"actionType": "book_flight"
}
{
"sagaId": 1,
"commandId": 34,
"success": true,
"undo": {
"command": "CancelFlight(...)",
"actionType": "cancel_flight"
}
}
Action request Action response
Action processor
Idempotent effect
Use the built in ones, or roll your own
?
31. SIMPLE MACHINES SAGAS IN KAFKA
!31
Event Sourcing Action Processor
Action request
Key Value
Saga ID DebitAccount(amount=100)
Saga Coordinator
Command processor Event Aggregator
Action request
Simple Sourcing
Command request
Aggregate state
Event
Action response
Command request
Key Value
Bob’s a/c DebitAccount(amount=100)
Event
Key Value
Bob’s a/c DebitAccount(amount=100)
Aggregate state
Key Value
Bob’s a/c Account(balance=500)
Bob’s a/c DebitAccount(amount=100)
Command response
Key Value
Bob’s a/c DebitAccount(amount=100)
Action response
Key Value
Bob’s a/c DebitAccount(amount=100)
Command response
Event sourcing action
processor
saga-long per-aggregate
consistency guarantee
32. SIMPLE MACHINES SAGAS IN KAFKA
!32
Async Action Processor
Action request
Key Value
Saga id ThirdPartyPayment(amount=50)
Saga Coordinator
Action request Action response
Async output
Key Value
? Payment(token=XXX)
Action response
Key Value
Saga id Response(success=true,
undoCommand=CancelPayment(X
XX))
Action output
Async action processor
Microservice
{
"success": true,
“booking_reference": “XXX"
}
Callback
Async / http invoke
undo actions can also be
defined dynamically from
web service result
update saga with new
undo action