2. What Are Microservices?
“is a software architecture style in which complex
applications are composed of small, independent
processes communicating with each other using language-
agnostic APIs. These services are small, highly
decoupled and focus on doing a small task, facilitating a
modular approach to system-building.” - Wikipedia
https://en.wikipedia.org/wiki/Microservices
3. Monolithic vs. SOA vs. Microservices
SOA
Coarse-
grained
Microservices
Fine-grained
Monolithic
Single Unit
4. Order UI User UI
Shipping
UI
Order
Service
User
Service
Shipping
Service
Data
Access
Monolithic Architecture
7. Order UI User UI
Shipping
UI
Order
Service
User
Service
Shipping
Service
Microservices Architecture
8. Order UI User UI UI
Order
Service
Service
Shipping
Service
Order UI
Order UI
User UI UIShipping
UI
Order
ServiceOrder
Service
Service
Service
Service
Service
User
Service
Shipping
Service
Microservices Architecture – Scaling
12. Microservice Challenge #1 – Resource Management
Managing a large fleet by hand is impossible:
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
AZ 1 AZ 2
AZ 3
13. Microservices Challenge #2 – Monitoring
A microservices architecture will have 10s, 100s, 1000s,
maybe even 10,000s of individual services:
• How do you know if an individual service is healthy?
• How do you measure the performance of an individual
service?
• How do you troubleshoot and debug an individual
service?
14. Microservices Challenge #3: Service Discovery
Each microservice scales up and down independently of
one another:
• How does Service A know the URLs for all instances of
Service B?
• How do you allow services to scale independently while
still using load balancers?
• How does a new instance of a service announce itself to
other services?
15. Microservices Challenge #4: Deployment
A microservices architecture will have 10s, 100s, 1000s,
maybe even 10,000s of individual services:
• Each service will be developed, tested, and deployed on
its own timeline – How do you manage this across large
numbers of services?
• Services are polyglot – different languages, frameworks
– how do you efficiently deploy them?
• How do you decide which hosts to deploy a service on?
17. Containers Are Natural for Services
Simple to model
Any app, any language
Image is the version
Test & deploy same artifact
Stateless servers decrease change risk
18. Easily Manage Clusters for Any Scale
Nothing to run
Complete state
Control and monitoring
Scale
19. Designed for Use with Other AWS Services
Elastic Load Balancing
Amazon Elastic Block Store
Amazon Virtual Private Cloud
AWS Identity and Access Management
AWS CloudTrail
22. Services
Good for long-running applications
Load balance traffic across containers
Automatically recover unhealthy containers
Discover services
23. Monitoring with Amazon CloudWatch
Metric data sent to CloudWatch in 1-minute periods and recorded for a period
of two weeks
Available metrics: CPUReservation, MemoryReservation, CPUUtilization,
MemoryUtilization
Available dimensions: ClusterName, ServiceName
25. Scheduling Containers on ECS
Batch Jobs
ECS task scheduler
Run tasks once
Batch jobs
RunTask (random)
StartTask (placed)
Long-Running Apps
ECS service scheduler
Health management
Scale-up and scale-down
AZ aware
Grouped containers
27. Scheduling Containers: Long-Running App
Deploy quickly without reducing service capacity:
minimumHealthyPercent = 100%, maximumPercent =
200%
28. Scheduling Containers: Long-Running App
Blue-Green Deployments
• Define two ECS services
• Each service is associated w/ ELB
• Both ELBs in Route 53 record set
with weighted routing policy, 100%
primary, 0% secondary
• Deploy to blue or green service and
switch weights
TaskTask
Route 53
record set
with
weighted
routing
policy
0%
100%
30. Servers
Which users should have
access to my servers?
How will the application
handle server hardware failure?
How many users create
too much load for my servers?
How can I control
access from my servers?
When should I decide to
scale out my servers?
When should I decide to
scale up my servers?
How will I keep my server
OS patched?
Which OS should my
servers run?
What size servers are
right for my budget?
How can I tell if a
server has been
compromised?
Which packages should
be baked into my server images?
How will new code be
deployed to my servers?
How should I implement dynamic
configuration changes on my servers
How much remaining
capacity do my servers have?
(AAHHHHHHHHH!!)
32. What is serverless computing?
EC2
• Machine as the
unit of scale
• Abstracts the
hardware
• I configure
machines,
storage,
networking, OS
ECS
• Application as
the unit of scale
• Abstracts the
OS
• I run servers,
configure apps,
control scaling
AWS Lambda
• Functions as
the unit of scale
• Abstracts the
language
runtime
• I run my code
when it’s
needed
34. The serverless compute manifesto
Functions are the unit of deployment and scaling.
No machines, VMs, or containers visible in the programming model.
Permanent storage lives elsewhere.
Scales per request. Users cannot over- or under-provision capacity.
Never pay for idle (no cold servers/containers or their costs).
Implicitly fault-tolerant because functions can run anywhere.
BYOC – Bring your own code.
Metrics and logging are a universal right.
36. What is AWS Lambda?
EVENT SOURCE FUNCTION SERVICES (ANYTHING)
Changes in
data state
Requests to
endpoints
Changes in
resource
state
Node.js
Python
Java
C# NEW!
Triggers Uses
37. Continuous scalingNo servers to
manage
Never pay for idle
– No cold servers
(only happy
accountants)
Benefits of AWS Lambda
41. Using AWS Lambda
Bring your own code
• Node.js, Java, Python, C#
• Bring your own libraries
(even native ones)
Simple resource model
• Select power rating from 128
MB to 1.5 GB
• CPU and network allocated
proportionately
• Reports actual usage
Flexible authorization
• Securely grant access to
resources, including VPCs
• Fine-grained control over
who can call your functions
Flexible use
• Call or send events
• Integrated with other AWS
services
• Build whole serverless
ecosystems
42. Using AWS Lambda
Programming model
• AWS SDK built in (Python and
Node.js)
• Eclipse & Visual Studio plugins
• AWS Lambda is the “webserver”
• Use processes, threads, /tmp,
sockets normally
Stateless
• Persist data using Amazon
DynamoDB, S3, or
ElastiCache
• No affinity to infrastructure
(can’t “log in to the box”)
Authoring functions
• Author directly using the
console WYSIWYG editor
• Package code as a .zip and
upload to AWS Lambda or
S3
• Command line tools
Monitoring and logging
• Built-in metrics for requests,
errors, latency, and throttles
• Built-in logs in Amazon
CloudWatch Logs
43. Stream processing use case
• High ingest rate
• Near real-time processing (low latency from ingest to
process)
• Spiky traffic (lots of devices with intermittent network
connections)
• Message durability
• Message ordering
44. Serverless stream processing architecture
Sensors
Amazon Kinesis:
Stream
Lambda:
Stream Processor
S3:
Final Aggregated Output
Lambda:
Periodic Dump to S3
CloudWatch Events:
Trigger every 5 minutes
S3:
Intermediate Aggregated
Data
Lambda:
Scheduled Dispatcher
KPL:
Producer
45. Reference Case: bustle.com
Amazon
API Gateway
AWS Lambda Amazon Kinesis AWS Lambda
Redis
Amazon Mobile
Analytics
Amazon CloudWatch
Amazon Elasticsearch
Service
Amazon S3
Amazon RedshiftAmazon QuickSight
Engineering
Marketing & Operations
Design
Bustle.com users
• 52 million monthly users
• 100 million events daily
• 84% cost savings
• 0 servers
• 0 operating system patches
• Automatic scaling
47. Extending AWS Services using AWS Lambda
AWS
CloudFormation
custom
resources
Amazon Echo skills Amazon SWF tasks Customized
notifications with
Amazon SNS
Amazon Cognito
triggers
Amazon S3
triggers
Amazon
Dynamo DB
triggers
Amazon
Kinesis
processors
Microservices
with API
Gateway
Alexa, do
my expense
report
And the list
continues
to grow!