Event Driven Architecture

Wildfire Interactive, Inc. | 167 Hamilton Ave, Palo Alto CA 94301 Suite 200 | (888) 274-0929
Event Driven Architecture
Lourens Naudé – SAPO Codebits 2010

About
• Developer @ http://www.wildfireapp.com
• Ruby / C
• Well versed full stack
• Active interest in Event Driven technologies
and related business processes

Agenda
• Business and systems growth
• Events
• (Lack of) control
• Event Driven Architecture
• Domain Driven Design primer
• Command and Query Responsibility
Segregation
• Complex Event Processing

Maintenance cost is
multiple times more than
development cost

http://en.wikipedia.org/wiki/Open/closed_principle
“Systems should ideally be
open for extension, but
closed for modification”

Driving forces for change
• Fulfillment of business requirements
• Business expansion: rollout of new
processes to drive sales, market penetration etc
• Mergers and acquisitions
• Legislation

Modification risks
• Introduces regression testing and a QA
burden – never compromise a working
system for a functional requirement
• Profiling overhead for (soft) realtime
systems
• Excessive “refactoring” of existing stable
interfaces
• May fracture a business domain

EDA imposes a system
design that's geared for
extension, interoperability
and unanticipated use.

Interoperability VS Integration
• Two or more systems performing a
task / process together
• Integration implies a formal link / bridge
between two systems - high coupling
• Building with interoperability in mind is
the easiest way for long-term integration
• Should be accounted for in system lifecycle as
well: delays, cost, communication etc.

Governance (lay down the law)
• Technology standards, SLAs, development
processes
• Cannot achieve interoperation without
governance
• Poor governance: inappropriate policies,
people (habits) and business structures
• Policies should be enforced at runtime
in an auditable manner
• Otherwise like criminal law without
police and courts

Business layers

Business Layers
• Physical world: sensors and users as
actors
• Transactional: physical world interactions
with the business / domain layer
• Intelligence: reporting tier and insights,
most notably sales and marketing
performance
• This is where EDA really shines: visibility
into business processes
• Transparent operation for easier
compliance and lower support costs

Events

Event Definition
• Anything that happened (or didn't
happen)
• A change in state
• A condition that triggers a notification
• An event is always named in the past tense
and is immutable
CustomerAddressChanged
InventoryRelocated
OrderShipped

Three levels of detail
• Basic fact that an event occurred
• Event definition required in order to
recognize an event
• Event detail / context
• Always attempt to capture events at the point
of greatest information value (context)
• This reduces downstream lookup and
correlation overhead

Event Types
• Notification: informational
• Alert: notification that should cause a
response
• Event Object: record of an event - just data,
no behavior
• Business Event: meaningful for conducting
trade activities
• Complex: abstraction of one or more other
events that represents summary level insights

Notification Types
• Observational
• Reports an event
but
does not change
anything
LowStock
CartCleared
• Transactional
• Reports an event
and
causes change
• Reliable transport
• Originates from
within the org.
OrderShipped
CardProcessed

(Lack of) Control

Event Driven Programming
• Control flow is driven by external
events
• NO central controller (main) - contrary
to what we're taught starting off as devs
• No well defined flow of data

EDA Components
• Event producers
• Event consumers / listeners
• Event processors
• Event reactions
• Messaging infrastructure

Lack of control in EDA
• An EDA is driven by extensibility and
not controllability
• Transaction state is managed by
events, not some central controller
• Loosely coupled, asynchronous and stateless
• Tell downstream components what
happened, but not what to do.
• Business events enable a system to operate
under control, but without central
control

Reduced coupling
• There's always some coupling
• We strive to reduce the degree of
coupling
• Always coupled to data

Stateless Order process
• Modifying processes managed by a central
controller requires changes to a component
as well as the controller
• Move state / data from the central controller
to events

Event Driven Architecture

Use cases
• Government: security, border control,
taxes etc.
• Health: stream processing + research
• Compliance: guard against violations in
cross country laws
• Track and trace: Fedex, UPS etc.
• Service Level Agreements

When not to use an EDA
• If the potential for reuse is very little
• Problem being solved is lower down the
application stack
• Network bandwith and capacity is
limited – serialization overheads and tends
to be chatty
• Security boundaries: immutability of
events that carry sensitive data

EDA Workloads
• Simple event processing: narrow design
• Stream processing: high message
volume, but only relevant events is filtered
• Complex Event Processing (CEP): able
to correlate multiple streams

Interaction Types
• Time-driven: actions initiated at specified
times, eg. batch workflows etc.
• Request-driven: initiated by client and
completes when the provider replies, eg.
client server
• Event-driven: initiated by events and
participants is open ended

Hybrid sytems
Most event driven systems
also include time and request
driven components.

Components
• Event producers
• Event consumers / listeners
• Event processors
• Event reactions
• Messaging infrastructure

Nervous System Analogy
• Hand is the event producer
• Spinal cord is the ESB
• Brain is the event listener
• Brain is the event processor
• Hand is the event reaction – we pull away

Event Producers
• Generates event data OR transforms it
into a format that the EDA can
understand
• Data isn't always generated to feed an
EDA, especially if the producer is a legacy
component

Event Consumers / Listeners
• Knows how to differentiate an event
from data it receives
• Can only “hear” events it's supposed
to
• Should be able to detect and interpret an
event

Event Processors
• Able to asses event importance and
derives the next action to take
• No value in not handling a perceived event
• "Do nothing" is a valid reaction as
well

Event Reactions
• Automated responses
• Notifications to participants or people
• Human reactions / interventions for
offline processes

Messaging Infrastructure
• Communication link between EDA
participants
• Doesn't have to be a single piece of
infrastructure - distinct backbones
should be able to share information
• Embrace standards and be as universal as
possible
• Promote decoupling between
participants - pub/sub

Data in Event Driven
Systems

Event Data
• Business event data held in an event
object
• Context of what happened
ProductCategorized('dsaed32wed', 'Coffee')
OrderCancelled('ko90323', 'No funds')

State Data
• Core system data that changes as a
result of operational activity
Order('ko90323', 'Tom Jones', 13.45 'open')
Product('dsaed32wed', 'Delta', 'Coffee', 5.8)

Reference Data
• Stable / somewhat permanent - could
change, but a lot less frequently than
state data
• Reference data would be replaced whereas
state data would change incrementally.
SupportedCurrency('USD')
ProductCategory('Coffee')
Address('Some Street', 'CA', 'USA')

Lifecycle of Data
• New state data = state data + event
data applied
• State data is the accumulation of event
data over time
• Event data as deltas
Product('dsaed32wed', 'Delta', 'Coffee', 5.8)
ProductCategorized('dsaed32wed', 'Tea')
Product('dsaed32wed', 'Delta', 'Tea', 5.8)

Domain Driven Design
primer

Domain Driven Design
• Suitable for behavior driven systems
• Modeling is often data and not
behavior driven
• A Domain Model is a representation of
relationships between Entities and Values
• Try to have a business analyst review
processes before introducing new
architecture

Common Language
• Participants: Developers, Experts and
Stakeholders
• Nouns: things with / without identity
• Verbs: actions, hints at state changes
• Domain models evolve through language
• We often write different to how we speak and
think
• Most failures of systems is communication
failures of people

Entities
• Has an identity within the system
• State can change
Product
Customer
Order

Value Objects (Values)
• No identity
• Immutable
• Represented by it's structure - structural
equivalence
Money
OrderStatus
Timezone

Domain Services
• Actions, operations and activities
• Business driven intent
• Stateless
• Should handle behavior that doesn't map
well to any particular entity
ShippingQuotesService
PaymentProcessingService

Aggregates
• A group of Entities that belong
together
• An instance of a process
• A single unit of work – consistency /
transactional boundary
• eg. Order and Order items
• Order item is useless without an Order
• Remove Order item: reconstitute the Order,
find item via Order, remove item
• Usually identified by cascading delete
requirements

Rental administration example

Command and Query
Responsibility Segregation

Either perform an action,
or return a result
Not both.

Overview
• CAP theorem: only can have 2 of 3
• Consistency steps aside for Availability and
Partitionability
• Somewhat stale data is often acceptable
• Business requirements define by how much
• Split systems into distinct command and query
components (read and write)

Conceptual view

Benefits
• Split teams: high level + good communicators
on the write side, junior devs on the read side
• Ability to scale command and reporting
sides independently
• Auditable system (dependent on history)
• Selectively disable functionality during
upgrades on the write side whilst the reporting
side is still available
• Lends itself well to behavior driven extension
• Easy to test

CQRS Components
• Commands: drives change in the system
• Internal events: captures intent (event
store)
• External events: republished to let other
components know
• Queries: examines state changes

Eventual Consistency
• All data is stale, unless the whole
system uses pessimistic locking
• Most web apps request data *before*
they render screens
• Add 250ms to 500ms on average for
rendering and additional server side
blocking operations
• Users don't react immediately to UI
changes
• Strictly consistent on the write side, relaxed
(eventual) consistency on the read side

Rethinking UI / adaptors
• Users don't think in data, but in tasks
and processes
• Cannot get benefits of CQRS without
considering the UI first
• Focus on capturing user intent
explicitly
• User experience should be an integral part of
any business process

Ambigious UI
UpdateComment / CommentUpdated

Task Driven UI
ApproveComment / CommentApproved

Commands
• A well defined task with parameters
• Represents a process, not an entity
• Distinct and transactional unit of work:
wholly accepted or rejected
• The only way to change data in a system
• Just the right amount of granularity and intent
to not cause concurrency conflicts of updates
on large entities
• Rejectable with exception or fault event
• Idempotent – should never be reprocessed

Rethinking Validations
• There's a fundamental difference
between a validation and a business rule
• Validation: well structured and formatted etc.
• Business rule: uniqueness constraints etc.
• Validate in client and in command handler -
not all clients are well behaved
• Valid commands can fail, most often
through optimistic concurrency conflicts
• Common for offline processes to reject requests

Queries and Reports
• Read only view of data - examines state
in the system
• Never goes through the domain model
• Table per UI view (assuming relational
storage)
• Single call to the reporting store returns
all (or most) data for a screen
• Minimal / no transformation - reporting
tier is a cache

Aggregates and Events
---------- begin consistency boundary ----------
=> Commands::CategorizeProduct("blends",
"1517fd7efbffacdf928056656bbb61cf")
rebuild
Aggregates::Product(1517fd7efbffacdf928056656bbb61cf)
from Events::ProductCreated({:category=>"coffee",
:product_id=>"1517fd7efbffacdf928056656bbb61cf",
:sku=>"XYZ", :name=>"Delta"})
Aggregates::Product @ version 1
<= Events::ProductCategorized({:category=>:blends,
:product_id=>"1517fd7efbffacdf928056656bbb61cf"})
Aggregates::Product @ new version 2
---------- end consistency boundary ----------

Code example
class Aggregates::Product
......
def rename(name)
apply Event(:ProductRenamed, uuid, name)
end
private
def apply_product_renamed(event)
@name = event.name
end
......
end

Internal Events
• Domain / Event store is append only: no
deletes, ad hoc reads.
• Compensating events to delete
• Event store is a history of intent
• Replay: events allow for reproducing error
conditions by rebuilding an aggregate via
snapshot up to just before a problem occurred
• Republishes any handled events to other
components

External Events
• Events handled by our domain that's
been republished to the rest of our
system
• Communicates any state changes to
the reporting tier
• Integration / interoperation hook for
business growth

ORM Issues
• Suitable for structural models only
• Structural models tend to loose
information
• Mapping objects to structed data is difficult
to optimize
• CRUD – what changes on UPDATE ?

Testing
• Command -> Domain -> [Event, Event …]
• Able to assert behaviors in two ways
• What happened ?
• What didn't happen ?
• Same pattern for testing the reporting side
• Event -> View -> assert state from getters

Assertion example
def test_relocate_inventory
Domain << Command(:CreateInventory, uuid,
123, 'Delta', 'A', 'X',10)
Domain << Command(:RelocateInventory, uuid,
'B', 'Y')
assert_published :InventoryCreated,
:InventoryRelocated
end

Complex Event Processing

Overview
• Connecting the dots
• Summary level information gathered
from base events
• Patterns identified by business experts, but
enforced at runtime by CEP software
• Value is proportional to accuracy and
relevance
• Accurate data can however still be
irrelevent
• Allows for Management by Exception

Consciousness / awareness
• Without consciousness can't react to
situations without human intervention
• SOA's the nervous system
• EDA brings awareness
• The core of Military systems – situational
awareness on the battlefield derived from input
streams

Event Stream
• Linear (ordered by time) sequence of events
• Produced by a single component / system
• Little aggregation required
• High throughput
• Minimal insights into what happened
• Example: market data feed

Event Cloud
• Collection of Event Streams within a system
• Aggregates streams into event sets
• Much lower throughput
• Very good situational awareness
• Example: trading strategies that's
dependent on multiple market feeds,
realtime news sources etc.

Event Cloud layout

CEP in Retrospective
• Adds value to Event Driven systems
• “Complex” is misleading – it simplifies input
and streams
• Matches real-time events from an Event
Cloud with patterns from historical data
and detect or predict situations
• Event -> correlate -> assess -> decide -> action

Summary
• Restist modification
• Tasks and intent – behavior matters
• Communicate
• Reduce coupling and avoid central
control

@methodmissing
http://www.github.com/methodmissing
CQRS Example
https://github.com/methodmissing/aftermath
Thanks !
Questions ?

Event Driven Architecture

Empfohlen

Empfohlen

Weitere ähnliche Inhalte

Was ist angesagt?

Was ist angesagt? (20)

Ähnlich wie Event Driven Architecture

Ähnlich wie Event Driven Architecture (20)

Mehr von Lourens Naudé

Mehr von Lourens Naudé (9)

Kürzlich hochgeladen

Kürzlich hochgeladen (20)

Event Driven Architecture