Microservices are an essential enabler of agility but developing and deploying them is a challenge. In order for microservices to be loosely coupled,each service must have its own datastore. This makes it difficult to maintain data consistency across services.
Deploying microservices is also a complex problem since an application typically consists of 10s or 100s of services, written in a variety of languages and frameworks. In this presentation, you will learn how to solve these problems by using an event-driven architecture to maintain data consistency and by using Docker to simplify deployment.
Microservices + Events + Docker = A Perfect Trio (dockercon)
1. @crichardson
Microservices + Events +
Docker = A Perfect Trio
Chris Richardson
Founder of Eventuate.io
Founder of the original CloudFoundry.com
Author of POJOs in Action
@crichardson
chris@chrisrichardson.net
http://eventuate.io
5. @crichardson
About Chris
Founder of a startup that is creating
a platform that makes it easier for
developers to write transactional
microservices
(http://eventuate.io)
15. @crichardson
Apply functional
decomposition
X axis
- horizontal duplication
Z
axis
-data
partitioning
Y axis -
functional
decomposition
Scale
by
splitting
sim
ilar
things
Scale by
splitting
different things
3
19. @crichardson
Drawbacks
Complexity of developing a distributed system
Implementing inter-process communication
Handling partial failures
Complexity of implementing business transactions that span multiple
databases (without 2PC)
Complexity of testing a distributed system
Complexity of deploying and operating a distributed system
Managing the development and deployment of features that span
multiple services
Fortunately solutions exists
21. @crichardson
Issues to address
How to deploy the services?
How do the services communicate?
How do clients of the application communicate with the
services?
How to partition the system into services?
How to deal with distributed data management problems?
âŠ.
25. @crichardson
Database per service
Order Service Customer Service
Order Database Customer Database
Order table
Customer
table
orderTotal creditLimit
Loose coupling đ but more complex đ andâŠ.
27. @crichardson
Customer management
How to maintain data consistency
without 2PC?
Order management
Order Service
placeOrder()
Customer Service
updateCreditLimit()
Customer
creditLimit
...
has ordersbelongs toOrder
total
Invariant:
sum(open order.total) <= customer.creditLimit
?
33. @crichardson
Replay events to recreate
state
Order
state
OrderCreated(âŠ)
OrderAccepted(âŠ)
OrderShipped(âŠ)
Events
Periodically snapshot to avoid loading all events
34. @crichardson
BeneïŹts of event sourcing
Solves data consistency issues in a Microservice/NoSQL based
architecture
Reliable event publishing: publishes events needed by predictive
analytics etc, user notiïŹcations,âŠ
Eliminates O/R mapping problem (mostly)
ReiïŹes state changes:
Built in, reliable audit log
temporal queries
Preserved history More easily implement future requirements
35. @crichardson
Drawbacks of event sourcing
Requires application rewrite
Weird and unfamiliar style of programming
Events = a historical record of your bad design decisions
Must handle duplicate events: idempotent handlers or
duplicate detection
Querying the event store can be challenging
37. @crichardson
Find recent, valuable
customers
SELECT *
FROM CUSTOMER c, ORDER o
WHERE
c.id = o.ID
AND o.ORDER_TOTAL > 100000
AND o.STATE = 'SHIPPED'
AND c.CREATION_DATE > ?
Customer
Service
Order Service
What if event
sourcing is
used?âŠ. is no longer easy
38. @crichardson
Command Query Responsibility
Segregation (CQRS)
Command side
Commands
Aggregate
Event Store
Events
Query side
Queries
Materialized
View
Events
POST
PUT
DELETE
GET
MongoDB
Redis
Neo4j
SQL
ElasticSearch
âŠ
More complex đ but high performance, scalable views đ
41. @crichardson
Forces
Services are written using a variety of languages, frameworks,
and framework versions
Each service consists of multiple service instances for
throughput and availability
Building and deploying a service must be fast
Services must be deployed and scaled independently
Service instances need to be isolated
Deployment must be reliable and cost-effective
44. @crichardson
VM
VM
Pattern: Service per Container
host
Service
Container
image
Container
Service
Container
Service
Container
Service
packaged as
deployed as
45. @crichardson
BeneïŹts of containers
Great isolation
Great manageability
Container encapsulates implementation technology
EfïŹcient resource utilization
Fast deployment
50. @crichardson
Smoke testing docker images
Smoke test
Docker
daemon
Service
containerGET /health
POST /containers/create
creates
POST /containers/{id}/start
Docker daemon must listen on
TCP port
51. @crichardson
Running on Docker!
EC2 Instance
Jenkins
Container
Artifactory
container
EBS volume
/jenkins-
home
/gradle-home
/artifactory-
home
52. @crichardson
Summary
Use microservices to accelerate development
Use an event-driven architecture to maintain data consistency
Use Docker to simplify development and deployment