How Events Are Reshaping Modern Systems

How Events
Are Reshaping
Modern Systems
Jonas Bonér
@jboner

1. Events drive autonomy
2. Events help reduce risk
3. Events help you move faster
4. Events Increase Loose coupling
5. Events increase stability
6. Events increase scalability
7. Events increase resilience
8. Events increase traceability
9. Events allow for time-travel
Why Should you care?

1. Cloud and multicore architectures
2. Microservices and distributed systems
3. Data-centric applications
4. “We want more, of everything, and we
want it now.” - Your Customers

Why Now?
1. Cloud and multicore architectures
2. Microservices and distributed systems
3. Data-centric applications
4. “We want more, of everything, and we
want it now.” - Your Customers

“Event-Driven Architecture (EDA) is a design paradigm
in which a software component executes in response
to receiving one or more event notifications.”
- Gartner
Event Driven
Architecture
To The Rescue

Event
“Something that happens.”
- Merriam Webster

✴Events represent Facts of information
➡ Facts are immutable
➡ Facts Accrue - Knowledge can only grow
The Nature of Events

✴ Events/Facts can be disregarded/Ignored

✴ Events/Facts Can not be retracted (once accepted)

✴ Events/Facts Can not be deleted (once accepted)
➡ Might be needed for legal or moral reasons

✴ Events/Facts Can not be deleted (once accepted)
➡ Might be needed for legal or moral reasons
✴ Events/Facts (new) can invalidate existing Facts

Event Loop
What Makes Event Tick

✴ Queue + anonymous Callbacks
✴ The Hollywood principle
✴ At-Most-Once delivery guarantees
✴ Local context only
✴ Sync or AsynC
✴ Limited and quite powerless
The Event Loop

✴ Queue + anonymous Callbacks
✴ The Hollywood principle
✴ At-Most-Once delivery guarantees
✴ Local context only
✴ Sync or AsynC
✴ Limited and quite powerless
The Event Loop
*
* Too limited as a general model of computation, and programming model for modern distributed systems. But is a useful building block.

✴ Ephemeral and anonymous
✴ No sane failure model
Error Channel - console.log(‘4/0=err’, err)
✴ No composition
Signature: Input => Side-effect - (Any => Unit)
✴ Hard to Debug
✴ Hard to reason about
✴ Leads to Callback hell
The Problem With
Event Callbacks

https://techbrunch.gousto.co.uk/2016/04/22/callback-hell/

✴Patterns
Observer, Pub-Sub, Broadcast, …
✴Futures & PRomises
✴Dataflow Variables
✴Stream Processing
✴ Event-driven Microservices
✴ FaaS
We Need High Level
Abstractions

Unified Event Driven Abstractions
Powered By
Message Driven Fabric
Ladder of
abstraction

Powered By
Network Boundary
Ladder of
abstraction

Powered By
Unified API (Event-driven Microservices, Patterns, Streams, Futures, FaaS,…)
Network Boundary
Ladder of
abstraction

Powered By
Network Boundary
Event 1Ladder of
abstraction

Powered By
Local Event Loop
Network Boundary
Event 1Ladder of
abstraction

Powered By
Local Event Loop
Distribution Fabric
Network Boundary
Event 1Ladder of
abstraction

Powered By
Local Event Loop
Distribution Fabric Distribution Fabric
Network Boundary
Message Passing
Event 1Ladder of
abstraction

Powered By
Local Event Loop
Distribution Fabric
Local Event Loop
Distribution Fabric
Network Boundary
Message Passing
Event 1Ladder of
abstraction

Powered By
Local Event Loop
Distribution Fabric
Local Event Loop
Distribution Fabric
Network Boundary
Message Passing
Event 1 Event 1Ladder of
abstraction

Powered By
Local Event Loop
Distribution Fabric
Local Event Loop
Distribution Fabric
Network Boundary
Message Passing
Event 1 Event 1
Reactive Systems is doing the heavy lifting
Ladder of
abstraction

Powered By
Local Event Loop
Distribution Fabric
Local Event Loop
Distribution Fabric
Network Boundary
Message Passing
Event 1 Event 1
Reactive Systems is doing the heavy lifting
Ladder of
abstraction Reactive Programming

1. REceive and react to facts (or not), that
are coming its way
2. Publish new facts (as events) to the rest
of the world
3. Invert the control flow to minimize
coupling and increase autonomy
Event Driven
Services

Mutable State
Needs To Be
Contained
And Non Observable

Publish Facts
To Outside World

Event
Stream
Use The
as the integration fabric

Event Driven
Services
Command
Event
Event Stream

Event Driven
Services
Command
Event
EventEvent
Event Stream

Event Driven
Services
Eventual
Consistency
Command
Event
EventEvent
Event Stream

Eventual
Consistency
No One Wants
It’s a necessery evil

But Relax
It Is How The
World Works

Information Is Always
From the Past

Welcome To The Wild Ocean Of
Non Determinism
Distributed Systems

“In a system which cannot count on
distributed transactions, the
management of uncertainty must be
implemented in the business logic.”
- Pat Helland
Life Beyond Distributed Transactions - An Apostate’s Opinion, Pat Helland (2007)
We Need To Model
Uncertainty

Exploit
Reality
We need to
In our design

Events Can Lead To Greater
Certainty

Events Can Help Us Craft
Autonomous Islands
Of Determinism

What does it mean to be
Automonous?

From greek: Auto-nomos
✴ Auto meaning self
✴ Nomos meaning Law
Autonomy

Promise
Theory
Let
Lead the way

✴Impositions diverge
into unpredictable outcomes
from definite beginnings
Distributed information
Decreased certainty
Convergence
vs.
Divergence
of information
✴Promises converge
towards a definite outcome
from unpredictable beginnings
Local information
Improved certainty

“Autonomy makes information local,
leading to greater certainty and stability.”
“An autonomus component can only
promise its own behavior.”
- Mark Burgess
In Search of Certainty, Thinking in Promises - Mark Burgess

Benefits of Thinking in Promises

✴ If a service only promises its own behavior
Then all information needed to
➡ Resolve a conflict
➡ Repair under failure
➡ Is available within the service itself

✴ If a service only promises its own behavior
Then all information needed to
➡ Resolve a conflict
➡ Repair under failure
➡ Is available within the service itself
✴ Promises remove the need for needless
1. Communication
2. Coordination
3. Consensus

“The first principle of successful
scalability is to batter the consistency
mechanisms down to a minimum.”
- James Hamilton
As transcribed in: Toward a Cloud Computing Research Agenda. SIGACT News, 40(2):68–80, June 2009.

Events First Design
Drives Autonomy

Events First
Domain Driven
Design

“When you start modeling events, it
forces you to think about the behavior
of the system. As opposed to thinking
about the structure of the system.”
- Greg Young
A Decade of DDD, CQRS, Event Sourcing, Greg Young (Presentation from 2016)

✴ Don’t focus on the things
The Nouns
The Domain Objects
✴ Focus on what happens
The Verbs
The Events

✴ IntentS
➡ Communication
➡ Conversations
➡ Expectations
➡ Contracts
➡ Control Transfer
Event Driven Design

✴ IntentS
➡ Communication
➡ Conversations
➡ Expectations
➡ Contracts
Event Driven Design
✴ Facts
➡ State
➡ History
➡ Causality
➡ Notifications
➡ State Transfer

✴ IntentS
➡ Communication
➡ Conversations
➡ Expectations
➡ Contracts
Event Driven Design
✴ Facts
➡ State
➡ History
➡ Causality
➡ Notifications
➡ State Transfer
Commands

✴ IntentS
➡ Communication
➡ Conversations
➡ Expectations
➡ Contracts
Event Driven Design
✴ Facts
➡ State
➡ History
➡ Causality
➡ Notifications
➡ State Transfer
Commands Events

✴Commands
➡ Object form of method/Action request
➡ Imperative: CreateOrder, ShipProduct
Event Driven Design

✴Commands
✴Reactions
➡ Represents side-effects
Event Driven Design

✴Commands
✴Reactions
➡ Represents side-effects
✴Events
➡ Represents something that has happened
➡ Past-tense: OrderCreated, ProductShipped
Event Driven Design

Commands Eventsvs
1. All about intent 1. Intentless

Commands Eventsvs
1. All about intent
2. Directed
1. Intentless
2. Anonymous

Commands Eventsvs
1. All about intent
2. Directed
3. Single addressable
destination
1. Intentless
2. Anonymous
3. Just happens - for
others (0-N) to observe

Commands Eventsvs
1. All about intent
2. Directed
destination
4. Models personal
communication
1. Intentless
2. Anonymous
4. Models broadcast
(speakers corner)

Commands Eventsvs
1. All about intent
2. Directed
destination
4. Models personal
communication
5. Distributed focus
1. Intentless
2. Anonymous
4. Models broadcast
(speakers corner)
5. Local focus

Commands Eventsvs
1. All about intent
2. Directed
destination
4. Models personal
communication
5. Distributed focus
6. Command & Control
1. Intentless
2. Anonymous
4. Models broadcast
(speakers corner)
5. Local focus
6. Autonomy

Let the Events Define the
Bounded Context

Principles For
Promise Theory Driven
Protocol Design

1. Convergency
Promises should reach a desired state,
a stable outcome - idempotent behavior
Principles For
Protocol Design

1. Convergency
2. Embracing Failure
Expect things to go wrong
Promises may or may not be kept
Principles For
Protocol Design

1. Convergency
2. Embracing Failure
Expect things to go wrong
Promises may or may not be kept
3. Autonomy
You can’t rely on other services
Principles For
Protocol Design

✴ Maintains Integrity & consistency
✴ Is our Unit of Consistency
✴ Is our Unit of Failure
✴ Is our Unit of Determinism
✴ Is fully Autonomous
The Aggregate

Enough Talk
Show Me The
Code
Sample in Java: https://gist.github.com/jboner/4361c710d2e2386be8bddc39edb0528a
Sample in Scala: https://gist.github.com/jboner/dea4aa819e127d543d684208d74081b5

Are we done now?
Perhaps. We have come a long way.

Are we done now?
But events can also be used for:

Are we done now?
➡ Persistence

Are we done now?
➡ Persistence
➡ Managing time

“Update-in-place strikes systems
designers as a cardinal sin: it violates
traditional accounting practices that have
been observed for hundreds of years.”
- jim Gray
The Transaction Concept, Jim Gray (1981)

Event Sourcing
A Cure For the Cardinal Sin

“The truth is the log.
The database is a cache
of a subset of the log.”
- Pat Helland
Immutability Changes Everything, Pat Helland (2015)

Happy Path
Event
Sourced
Services

Happy Path
Event
Sourced
Services
1) Receive and verify Command
(“ApprovePayment”)

Happy Path
Event
Sourced
Services
2) Create new Event
(“PaymentApproved”)

Happy Path
Event
Sourced
Services
2) Create new Event
3) Append Event
to Event Log

Happy Path
Event
Sourced
Services
2) Create new Event
3) Append Event
to Event Log
4) Update internal
component state

Happy Path
Event
Sourced
Services
5) Run side-effects
(approve the payment)
2) Create new Event
3) Append Event
to Event Log
4) Update internal
component state

SAD Path - recovering from failure
Happy Path
Event
Sourced
Services
5) Run side-effects
2) Create new Event
3) Append Event
to Event Log
4) Update internal
component state

Happy Path
Event
Sourced
Services
5) Run side-effects
2) Create new Event
3) Append Event
to Event Log
4) Update internal
component state
1) Rehydrate Events
from Event Log

Happy Path
Event
Sourced
Services
5) Run side-effects
2) Create new Event
3) Append Event
to Event Log
4) Update internal
component state
1) Rehydrate Events
from Event Log
2) Update internal
component state

Happy Path
Event
Sourced
Services
5) Run side-effects
2) Create new Event
3) Append Event
to Event Log
4) Update internal
component state
1) Rehydrate Events
from Event Log
2) Update internal
component state
Memory Image

Event Sourcing
✴ One single Source of Truth with All history
✴ Allows for Memory Image - durable in-memory state
✴ Avoids the Object-relational mismatch
✴ Allows others to Subscribe to state changes
✴ Mechanical sympathy (Single Writer Principle etc.)

✴ Unfamiliar model
✴ Versioning of events
✴ Deletion of events (legal or moral reasons)
Disadvantages
Of Using Event Sourcing

“Modeling events forces you to have a temporal
focus on what’s going on in the system.
Time becomes a crucial factor of the system.”
- Greg Young
A Decade of DDD, CQRS, Event Sourcing, Greg Young (Presentation from 2016)

Causality
How We Make Sense Of The World

Time Is the Succession Of
Causally Related Events

✴ Event is a snapshot in time
✴ Event ID is an index for time
✴ Event Log is our full history
The database of Our past
The path to Our present
Event Sourcing
Allows Us To
Model Time

Event Sourcing
Allows For
Time Travel

Event Sourcing
Allows For
Time Travel
✴Replay the log for historic debugging
✴Replay the log for auditing & traceability
✴Replay the log on failure
✴Replay the log for replication

“The future is a function of the past.”
- A. P. Robertson

“The (local) present is a merge function
of multiple concurrent pasts.”
- me

val newLocalPresent = observedPasts.
foldLeft(oldLocalPresent) { _ merge _ }

We need to explicitly model the
local present as
facts derived from the
merging of multiple
concurrent pasts

We Can Even Fork the Past
...Or Join Two Distinct Pasts

Let Make Our Demo
Event Sourced

Untangle Your
Read and Write Models
With CQRS

CQRS
Write Side Model
Commands

CQRS
Write Side Model
Write to Event Log
Commands

CQRS
Write Side Model
Events
Write to Event Log
Commands

CQRS
Write Side Model
Events
Events
Write to Event Log
Commands

CQRS
Read Side Model
Write Side Model
Events
Events
Read Data Store
Write to Event Log
Commands

CQRS
Read Side Model
Write Side Model
Events
Queries
Events
Read Data Store
Write to Event Log
Commands

CQRS
Read Side Model
Write Side Model
Events
Queries
Events
Read Data Store
Write to Event Log
Eventual
Consistency
Commands

✴ Temporal Decoupling
✴ Increased Resilience
✴ Increased Scalability
✴ Allows for Multiple Event subscribers
✴ Allows for Polyglot Persistence
Advantages
Of Using CQRS

✴ More infrastructure to manage
✴ 2 data models (or more) to maintain
✴ Eventual Consistency
Disadvantages
Of Using CQRS

Key Takeaways
Event driven design helps you to:
✴ Design autonomous services
✴ Move Faster towards a Reactive architecture
✴ reduce risk when modernizing applications
✴ Balance Certainty and Uncertainty

Key Takeaways
Event driven design helps you to:
✴ Design autonomous services
✴ Move Faster towards a Reactive architecture
✴ reduce risk when modernizing applications
✴ Balance Certainty and Uncertainty
Event Logging allows you to:
✴ take control of your system’s history
✴ time-travel
✴ Balance Strong and eventual consistency

Learn More
Download my latest book for free at:
bit.ly/reactive-microsystems

How Events Are Reshaping Modern Systems

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to How Events Are Reshaping Modern Systems

Similar to How Events Are Reshaping Modern Systems (20)

More from Jonas Bonér

More from Jonas Bonér (17)

Recently uploaded

Recently uploaded (20)

How Events Are Reshaping Modern Systems