Over the years I've spoken many times about what Event Sourcing is and shared many of the good, the bad and the ugly parts of it in blog posts and various talks. However, I've never talked about how to actually build a system based on this architecture style. I keep getting the same questions over and over again. Like when to apply Event Sourcing and at what architectural level. How to deal with transactional boundaries within and outside the domain. How to build projections that are autonomous, reliable and self-supporting. How to deal with upgrades and blue-green deployments. But also on how to handle bugs, design mistakes and crashing projections. Having made a lot of these mistakes myself over these years, it's time to share my current thoughts and opinions about this. Since the .NET space has a pretty rich set of open-source projections to support this, the examples and code will be .NET. But the concepts are universal, so don't let that scare you off.

1. Dennis Doomen | @ddoomen | Aviva Solutions | The Continuous Improver

5. NOT

14. Application
Command
Service
Correct
Customer
Email Handler
Customer
Unit of
Work
Projector
Data
Access
Layer
Read
Database Write
Database
CorrectCustomerEmailCommand
HTTP API / In-process invocation
Get<Customer>(identity)
Correct Email
Event Store
Load(events)
Apply
Get changes
CustomerEmailCorrectedEvent
Submit changes
History
Dennis Doomen | @ddoomen | The Continuous Improver
NES
Aggregates.NET
SimpleDomain
NStore
EventStore
NEventStore (*)
SQLStreamStore (*)
NStore
NES
Marten
Handelier
Brighter
MediatR (*)
Projac
LiquidProjections (*)
EventStore

15. Dennis Doomen | @ddoomen | The Continuous Improver
Application
Domain
NoSQL / RDBMS
OR/M / DAL
Web UI, HTTP API,
etc
Lucene Index
Document Projector
Web UI, HTTP
API, etc
Web UI, HTTP API, etc
Domain
Commands
Events
Event StoreProjections
Projectors
Uses Event
Sourcing
Uses traditional
CRUD
architecture
Indexing-based
architecture
Subcribe
to
webhooks
Coarse-
grained
HTTP
requests.
Bus
Subscribe
Publish coarse-
grained event

16. Customer #123
Correct Customer
Shipping Address
Command
Customer Shipping
Address Corrected
Event
Event Store
Treat as a
message, not a
type
Order #456
Correct Shipping Address
Command Handler
Application
Customer Shipping Address
Corrected Handler
Order Redirected
Event.
Identity !=
natural key
Identify
partition key
Primitive types
only
Avoid terms like
Create, Update,
Delete, Change.
Avoid property
change events
Don’t use as
inter-domain
contracts
Don’t expose
outside the
domain
Dennis Doomen | @ddoomen | The Continuous Improver

17. Customer #123
Correct Customer
Shipping Address
Command
Customer Shipping
Address Corrected
Event
Event Store
Order #456
Correct Shipping Address
Command Handler
Application
Order Redirected
Event
Dennis Doomen | @ddoomen | The Continuous Improver

18. Customer #123
Correct Customer
Shipping Address
Command
Customer Shipping
Address Corrected
Event
Event Store
Order #456
Correct Shipping Address
Command Handler
Application
Order Redirected
Event
Dennis Doomen | @ddoomen | The Continuous Improver

19. Command Service
Some Command
Handler
Customer #123
Event Store
Customer Created
Event
Get<Customer>(“123”)
Customer
Created Event
Converter
Customer Enrolled
Event
May split or
merge events
Can also run as
part of migration
May change the
identity
Unaffected Events
Dennis Doomen | @ddoomen | The Continuous Improver

20. Event
Store
Projector Projector
RDBMS
Subscribe
Subscribe
Document
DB
Projector
RDBMS
Subscribe
Raw SQLNHibernate RavenDB
Lookup
Autonomous &
independent
Storage technique
optimized for
projection
No joining of
projections
Avoid reusing the
projections for
multiple purposes
Use aggressive
caching during
rebuilds.
Caching strategy
optimized for
projector
Owned by
projector
Dennis Doomen | @ddoomen | The Continuous Improver

21. var mapBuilder = new EventMapBuilder<MyContext, DocumentProjection, string>();
mapBuilder
.Map<WarrantAssignedEvent>()
.AsCreateOf(anEvent => anEvent.Number)
.Using((document, anEvent) =>
{
document.Type = "Warrant";
document.Kind = anEvent.Kind;
document.Country = anEvent.Country;
document.State = anEvent.InitialState;
});
mapBuilder
.Map<StateTransitionedEvent>()
.When(anEvent => anEvent.State != "Closed")
.AsUpdateOf(anEvent => anEvent.DocumentNumber)
.Using((document, anEvent) => document.State = anEvent.State);
Dennis Doomen | @ddoomen | The Continuous Improver

22. Transaction
Event Event
Transaction
Event Event
Transaction
Event Event
RDBMS
INSERT/UPDATE … WHERE …
Transaction
Event Event
Transaction
Event Event
Transaction
Event Event
RDBMS
INSERT/UPDATE … WHERE …
INSERT/UPDATE … WHERE …
INSERT/UPDATE … WHERE …
Unit of Work
Dennis Doomen | @ddoomen | The Continuous Improver

23. private async Task<ExceptionResolution> OnException(ProjectionException exception, int attempts)
{
if (IsTransient(exception))
{
if (attempts < 3)
{
await Task.Delay(TimeSpan.FromSeconds(attempts ^ 2));
return ExceptionResolution.Retry;
}
return ExceptionResolution.Abort;
}
else
{
if (exception.TransactionBatch.Count > 1)
{
return ExceptionResolution.RetryIndividual;
}
// Mark projection as corrupt
return ExceptionResolution.Ignore;
}
}
Dennis Doomen | @ddoomen | The Continuous Improver

24. Event Store
Projector
Enrichment
Projector
Subscription
Subscription
Storage
(persistent)
Cache
Event
Enricher
Extends events
with lookup data
Receives
extended events
Dennis Doomen | @ddoomen | The Continuous Improver

25. var stats = new ProjectionStats(() => DateTime.UtcNow);
stats.StoreProperty("CountByDocument", "some setting key", "some value");
stats.LogEvent("CountByDocument", "some significant thing that happened");
stats.TrackProgress("CountByDocument", currentCheckpoint);
float? transactionsPerSecond = stats.GetSpeed("CountByDocument");
TimeSpan? eta = stats.GetTimeToReach("CountByDocument", targetCheckpoint);
public void Configure(IAppBuilder builder)
{
builder.UseStatistics(stats);
}
Dennis Doomen | @ddoomen | The Continuous Improver

26. IAppBuilder builder = ….
builder.UseStatistics(stats);
GET http://localhost/projectionStats/CountByDocument
{
"projectorId": "CountByDocument",
"lastCheckpoint": 1000,
"lastCheckpointUpdatedUtc": "2018-05-10T10:39:00Z",
"properties": [{
"key": "some setting key",
"value": "some value",
"lastUpdatedUtc": "2018-05-10T10:39:00Z"
}],
"eventsUrl": "http://localhost/projectionStats/CountByDocument/events"
}
Dennis Doomen | @ddoomen | The Continuous Improver

27. Document #1
Created Event
Event Store
Graph
Projector
Document #1
Closed Event
(other events)
Projector with
active
projections
Archiving
Projector
Start archiving
Document #1
Marked As
Archivable Event
Mark all events
as archivable
Tracks dependencies
between documents
Deletes
projections
related to
Document #1
Can skip all
archivable events
during next
rebuild.
Dennis Doomen | @ddoomen | The Continuous Improver

28. Event Store
Lucene
Projector
Document #1
Marked As
Archivable Event
Allows projectors
to clean up
Lucene Index
Take snapshot
Purge events
Tombstone
$tombstone
stream
Application
Document
Projector
Search
Tracks deleted streams
for future references
Stream Tombstoned
Event
Search
Dennis Doomen | @ddoomen | The Continuous Improver

29. Event Store
Projector
Application
Version 2
Owns schema
(e.g. FluentMigrator)
Version 1
X
Involves down-time until
projections are rebuild
Dennis Doomen | @ddoomen | The Continuous Improver

30. Event Store
Projector
Application Application
Network Load Balancer
Event Store
Version 1 Version 2
events
Projection ProjectorProjection
bring off-line
Returns HTTP 503
(Service Unavailable)
Dennis Doomen | @ddoomen | The Continuous Improver
Returns HTTP 503
(Service Unavailable)

33. Dennis Doomen | @ddoomen | The Continuous Improver

34. Dennis Doomen
@ddoomen
Dennis Doomen | @ddoomen | The Continuous Improver