Just over a year ago (before becoming the full time chair and advocate of QCon London, San Francisco, and New York), my main role was with HPE as the principal architect for a client in the US public sector.
The systems we supported were responsible for personnel information, scholarships decisions, and record management. Like so many others, we were also faced with legacy applications, COTS product integrations, polyglot code bases, and often brittle deployments. In an effort to decouple code bases and address some of these issues, we started advocating for a Microservice architecture and trying to distinguish it from the SOA practices of the past.
Now, it’s a year later. I have had the incredible opportunity to have access to architects, engineers, and leaders from some of the world’s more respected software companies. These are companies like Uber, Microsoft, Netflix, Apple, Google, Slack, Pinterest, and Etsy. I’ve had the chance to have one-on-one discussions with Chief Architects, developers, and engineers building the apps I most admire and use every day (some leveraging Microservices, some embracing Monoliths, and others falling somewhere in between).
Patterns & Practices of Microservices is some of the things I wish I knew before beginning a push towards Microservices just over a year ago. It’s the practices of companies leveraging Microservices, it’s the technology tradeoffs when deciding between Monoliths and Microservices, and it’s the advice I’ve heard in interviewing, podcasting, and iterating on presentations from software giants like Adrian Cockcroft, Matt Ranney, Josh Evans, Martin Thompson, and literally hundreds of other engineers who drop knowledge at QCons around the world.
2. Proxy /
Load Balancer
Web Server
Database Server
Application Server
Web Server
~4 YEARS
3. Proxy /
Load Balancer
Web Server
Database Server
Application Server
Web Server Web Server
Application Server
~3 YEARS
4. Proxy /
Load Balancer
Web Server
Database Server
Application Server
Web Server Web Server
Application Server
Proxy /
Load Balancer
Web Server
Application Server
Web Server Web Server
Application Server
Hot
Cold
Database Server
~2 YEARS
5. Proxy /
Load Balancer
Web Server
Database Server
Application Server
Web Server Web Server
Application Server
Proxy /
Load Balancer
Web Server
Application Server
Web Server Web Server
Application Server
Hot
Cold
GLB / DNS
Reporting
Database Server
~2 YEARS
6. Proxy /
Load Balancer
Web Server
Database Server
Application Server
Web Server Web Server
Application Server
Proxy /
Load Balancer
Web Server
Application Server
Web Server Web Server
Application Server
Reporting
Database Server
Hot
Hot
Database Server
GLB / DNS
~2 YEARS
10. W E S L E Y R E I S Z
San Francisco / London / New York
CHAIR QCON
11.
12.
13. “Loosely coupled Service Oriented
Architecture with Bounded Contexts”
-Adrian Cockcroft, Microservices Thoughtleader
IF EVERY SERVICE HAS TO BE
UPDATED AT THE SAME TIME
IT’S NOT LOOSELY COUPLED
IF YOU HAVE TO KNOW TOO
MUCH ABOUT SURROUNDING
SERVICES YOU DON’T HAVE A
BOUNDED CONTEXT. SEE THE
DOMAIN DRIVEN DESIGN BOOK
BY ERIC EVANS.
15. CONS
PROS
• Monitoring/Logging becomes more
important than ever.
• You trade code simplicity for operational
complexity.
• Easy to chase the latest shiny technology / mix
new languages & tech in.
• Another dimension to scale
the application.
• Easier to have different services use
data persistence most appropriate
for the use case.
• Easier to reason about moving applications
to the cloud.
• Easier to have a developer understand
the portion of the system and contribute
• Easier to test & deploy a single service.
• Cascading failures can cause
problems.
• Orchestration can
become complex
17. DEVOPS CULTURE IS A MUST
• Rapid provisioning
• Basic monitoring
• Rapid Application
Deployment
http://martinfowler.com/bliki/MicroservicePrerequisites.html
18. “Signing up for Microservices is signing up
for evolutionary architecture. There’s no point
where you’re just done.”
-Josh Evans,
Director of Operations Engineering @Netflix
19. Proxy /
Load Balancer
Web Server
Database Server
Application Server
Web Server Web Server
Application Server
Proxy /
Load Balancer
Web Server
Application Server
Web Server Web Server
Application Server
Reporting
Database Server
Hot
Hot
Database Server
GLB / DNS
21. SCALE CUBE
X axis - horizontal Scaling
(Scale by Cloning)
Z
axis-DataPartitioning
(ScalebySharding)
Y axis - Functional
Decomposition
(Scale by Splitting)
22. The CAP theorem is the idea that a distributed computing system is
not able to provide partition tolerance, consistency and availability at
the same time.
23. MICROSERVICESPATTERNS & PRACTICES
AVAILABILITY/PARTITION TOLERANCE
CONSISTENCY/PARTITION TOLERANCE
CP: Choose Consistency over Availability when your business
requirements dictate atomic reads and writes. Wait for a response
from the partitioned node which could result in a timeout error. The
system can also choose to return an error, depending on the
scenario you desire.
AP: Choose Availability over Consistency when your business
requirements allow for some flexibility around when the data in the
system synchronizes. Return the most recent version of the data you
have, which could be stale. This system state will also accept writes that
can be processed later when the partition is resolved. Availability is also
a compelling option when the system needs to continue to function in
spite of external errors (shopping carts, etc.)
24. Proxy /
Load Balancer
Web Server
Database Server
Application Server
Web Server Web Server
Application Server
Proxy /
Load Balancer
Web Server
Application Server
Web Server Web Server
Application Server
Reporting
Database Server
Hot
Hot
Database Server
GLB / DNS
25. Evolu&onary Architecture
Scale Cube
CAP Theorem
Microservices is a architecture that conFnues to change
and adapt
Scale Cube consists of an X, Y and Z axes – each
addressing a different approach to scale a service.
The idea that a distributed compuFng system is not able
to provide parFFon tolerance, consistency and availability
at the same Fme.
FIRST PRINCIPLES
Things we need to
understand about as we
plan/scale systems
DevOps is a Must
Microservices is the first architecture post devops culture
shiP
28. SPINE MODEL
• Needs:
We are here to satisfy ________
• Values before Principles
We optimize for ________
• Principles before Practices
We leverage _____ to change the system
• Practices before tools
We do _____ to create value
• We have always been Tool users and
makers. Tools should answer the question:
We use ____ to get our work done.
http://spinemodel.info/explanation/introduction/
29. Proxy /
Load Balancer
Web Server
Database Server
Application Server
Web Server Web Server
Application Server
Proxy /
Load Balancer
Web Server
Application Server
Web Server Web Server
Application Server
Reporting
Database Server
Hot
Hot
Database Server
GLB / DNS
30. Hot
Hot
GLB / DNS
User Facing Availability
Concerns
Internal Consistency
Concerns
User Facing Availability
Concerns
Internal Consistency
Concerns
36. The fundamental idea of Event Sourcing is that of
ensuring every change to the state of an application
is captured in an event object, and that these event
objects are themselves stored in the sequence they
were applied for the same lifetime as the application
state itself.
- M Fowler
38. Immutable Events
Recreate the exact state
Increase in complexity
Performance / Load tes&ng
Provides a model to think about scalability
Because you have all of the state changes, you have all of
the changes that affected state
No longer doing CRUD. You have calculate the result.
Leverage things like snapshots
You an reply the events across the Enterprise
EVENT SOURCING
The fundamental idea of Event
Sourcing is that of ensuring
every change to the state of an
application is captured in an
event object, and that these
event objects are themselves
stored in the sequence they
were applied for the same
lifetime as the application state
itself.
41. Command and Query Responsibility Segregation
(CQRS) is a pattern that segregates the operations
that read data (Queries) from the operations that
update data (Commands) by using separate
interfaces.
45. Domain Specific Models
Scale
Tradeoff
Performance
Clean Domain models queried
Scale the reads above the writes
Increased the operaFonal complexity of the data stores
Tune different datastore for different performance needs
CQRS
Command-Query-Responsibility-
Separation (CQRS) separates
query processing from command
processing, which updates
entities using business logic that's
commonly implemented using
event sourcing.
Security
Updates to the model controlled different from the reads
47. Service A called B.
TRIPPED
Service B is
Sick, let’s flag is and
try back in 5min
Service C
Different Instance of
Service B
Service B is
sick. Don’t
send traffic
there for 5
minute
(Tripped)
48. Service A called B.
OPEN
Service B is
still sick
Service C
Different Instance of
Service B
Service B is
still sick.
Isolate it and
remove from
ELB
50. Prevents Doomed to Fail
Three states
Prevents Cascading Failures
purpose is to detect external service failures and prevent
the applicaFon from trying to perform an acFon that is
doomed to fail.
Closed (Normal), Open(Tripped), & Half-Open state
Helps prevent cascading failures and improves resiliency.
CIRCUIT BREAKER
Circuit breaker is used to
detect failures and
encapsulates logic of
preventing a failure to reoccur
constantly (during
maintenance, temporary
external system failure or
unexpected system
difficulties).
Part of Hysterix
Part of Nelix OSS library for latency & fault tolerance
52. Bad Service
Good Service
Edge Service
3s
1s 1s
Fails fast @ 2s
“Upstream timeout must always be
longer than total downstream
timeout * retries delay”
-Adrian Cockcroft
Budgeted timeout, one retry
53. Tools
Resiliency PaIerns
Architectural PaIerns to Consider
Don’t start with the tool.
Circuit Breaker, Retry
Event Sourcing & CQRS
TECHNICAL
These are some interesting
technical considerations
with Microservices
55. OBSERVABILITY
In control theory, observability is a
measure for how well internal states of a
system can be inferred by knowledge of
its external outputs.
56. At a minimum setup
log stash, look at going
an extra step to setup
ELK.
OBSERVABILITY: THINGS TO THINK ABOUT
LOGS
• Need consistent, structured logging
• Logs are not for humans
• Logstash, Elasticsearch, Kibania works
great
57. OBSERVABILITY: THINGS TO THINK ABOUT
PROTOCOLS/
TRANSPORT
RPC/HTTP
vs
JSON vs Binary
RPC vs HTTP
JSON vs Binary
• Speed/Performance vs Readability
• Measure serialization, transmission, deserialization
costs
58. OBSERVABILITY: THINGS TO THINK ABOUT
How do you
Recreate a failure in
a distributed resilient
environment?
TRACING
Look into:
• Zipkin
• Prometheus
Best way to understand Fanout
62. Observability
Polyglot
Containers
Logging, Service CommunicaFon, Tracking
Containers makes it easier, but observability, logging,
tracking can be more challenging
Low fricFon deployment, Finer-grained execuFon
environments, speed, isolaFon
OPERATIONAL
Microservices simplifies
code. It trades code
complexity for operational
complexity
64. CONWAY'S LAW
“Organizations which design systems ... are
constrained to produce designs which are
copies of the communication structures of these
organizations”
-Conway
65. Model
One size does not fit all
Tradeoff
Organiza&onal Lifecycle Stages
Provides a model to think about scalability
Microservices (funcFon decomposiFon) is AN approach
Each approach has different tradeoffs (For example: as
we’ll find with Y-axis scaling, you may trade code
complexity for organizaFonal complexity)
Different needs at different points of an org’s lifecycle
CONWAY’S LAW
Organizations which design
systems ... are constrained
to produce designs which
are copies of the
communication structures of
these organizations
66. So what do I wish I knew before
starting the push to Microservices?
67. Proxy /
Load Balancer
Web Server
Database Server
Application Server
Web Server Web Server
Application Server
Proxy /
Load Balancer
Web Server
Application Server
Web Server Web Server
Application Server
Reporting
Database Server
Hot
Hot
Database Server
GLB / DNS
70. FIRST PRINCIPLES
MICROSERVICESPATTERNS & PRACTICES
• Have capabilities in your org to do things
like rapidly provision, monitor, and use
rapid deployment.
(DevOPs Culture is a must)
• Accept that the service going to evolve
(Evolutionary Architecture)
• Handle their own data needs
(Consistency/Availablity Tradeoffs)
• Be able to scale based on it’s needs
(Scale Cube)
Processing
DevOps Culture,
Evolutionary Architecture,
Scale Cube & CAP Theorem
71. TECHNICAL
Spine Model
Event Sourcing/CQRS
Circuit Breaker
Retry
OPERATIONAL
Observability
Containers
Polyglot
CULTURAL
Conway’s Law
MICROSERVICESPATTERNS & PRACTICES Processing
• Have a decision process
(Spine Model)
• Consider using events where it makes sense and
how you might scale commands & queries
(Event Sourcing / CQRS)
• Protect against cascading failures
(Circuit Breaker, Bulkhead)
• “Everything is a tradeoff, do it intentionally” Matt
Ranney (RPC/HTTP, Containers, Polyglot)
• Bootstrap services with organizational capabilities
(Logs, Tracing)
• Beware of Conway’s Law
72. CONS
PROS
• Monitoring/Logging becomes more
important than ever.
• You trade code complexity for operational
complexity.
• Easy to chase the latest shiny technology / mix
new languages & tech in.
• Another dimension to scale
the application.
• Easier to have different services use
data persistence most appropriate
for the use case.
• Easier to reason about moving applications
to the cloud.
• Easier to have a developer understand
the portion of the system and contribute
• Easier to test & deploy a single service.
• Cascading failures can cause
problems.
• Orchestration can
become complex
73. “If you don’t end up regretting your early
technology decisions, you probably over
engineered.” - Randy Shoup, Engineering Director @StitchFix
MICROSERVICESPATTERNS & PRACTICES
