Yan Cui, an AWS Serverless Hero, will talk about the learnings from using serverless at scale.
He will cover the challenges for observability in serverless asynchronous workloads and the patterns to address those challenges, like using centralized logging, correlation IDs, tracing, lambda extensions.
5. @theburningmonk theburningmonk.com
In control theory, observability is a measure of how well
internal states of a system can be inferred from
knowledge of its external outputs.
https://en.wikipedia.org/wiki/Observability
6. @theburningmonk theburningmonk.com
In control theory, observability is a measure of how well
internal states of a system can be inferred from
knowledge of its external outputs.
https://en.wikipedia.org/wiki/Observability
including non-
functional outputs
7. @theburningmonk theburningmonk.com
These are the four pillars of the Observability Engineering
team’s charter:
• Monitoring
• Alerting/Visualization
• Distributed systems tracing infrastructure
• Log aggregation/analytics
“
” http://bit.ly/2DnjyuW
- Observability Engineering at Twitter
35. @theburningmonk theburningmonk.com
API Gateway Lambda
Worker
Worker
Worker
Worker
Worker
Worker
Worker
Worker
Worker
Busy
Busy
Busy
Busy
Busy
Busy
Only processes one
request at a time
Runs in an AWS-managed
baremetal EC2 instance
42. @theburningmonk theburningmonk.com
user request
user request
user request
user request
user request
user request
user request
critical paths:
minimise user-facing latency
handler
handler
handler
handler
handler
handler
handler
43. @theburningmonk theburningmonk.com
user request
user request
user request
user request
user request
user request
user request
critical paths:
minimise user-facing latency
StatsD
handler
handler
handler
handler
handler
handler
handler
rsyslog
background processing:
batched, asynchronous, low
overhead
82. @theburningmonk theburningmonk.com
S3 Lambda DynamoDB Lambda API Gateway
Lambda
EventBridge
Lambda
Lambda
How to inject
correlation IDs?
How to inject
correlation IDs?
How to inject
correlation IDs?