Serverless Architecture Patterns

1. Serverless architectural patterns
Ric Harvey, Technical Developer Evangelist
@ric__Harvey

2. What does serverless mean?
No servers to provision or
manage
Scale with your usage
Built in availability and fault-
tolerance
Never pay for idle/unused
capacity

3. Serverless runs on functions
• Functions are the unit of deployment and scale
• This scales per request!
• Skip the boring parts, skip the hard parts

4. Serverless applications
FUNCTION SERVICES (ANYTHING)
Changes in
data state
Requests to
endpoints
Changes in
resource state
Node
Python
Java
C#
EVENT SOURCE

5. Example event sources
Data stores Endpoints
Configuration repositories Event/message sources
Amazon S3 Amazon
DynamoDB
Amazon
Kinesis
Amazon
Cognito
Amazon IoT AWS Step
Functions
Amazon
Alexa
AWS
CloudTrail
AWS
CodeCommit
Amazon
CloudWatch
Amazon SES Amazon SNS Cron events
Amazon
API Gateway
AWS
Cloudformation
…and more!

6. A few Lambda specific best practices
• Lambda is stateless à architect accordingly!
• Assume no affinity with underlying compute infrastructure
• Local filesystem and child processes may not extend beyond the lifetime of
the Lambda request

7. Lambda considerations and best practices
• Can your Lambda functions survive the cold?
• Instantiate AWS clients and database
clients outside the scope of the handler to
take advantage of connection re-use.
• Schedule with CloudWatch Events for
warmth
• ENIs for VPC support are attached during
cold start
import sys
import logging
import rds_config
import pymysql
rds_host = "rds-instance"
db_name = rds_config.db_name
try:
conn = pymysql.connect(
except:
logger.error("ERROR:
def handler(event, context):
with conn.cursor() as cur:
Executes during
cold start
Executes with each
invocation

8. Lambda considerations and best practices
• How about a file system?
• Don’t forget about /tmp (512 MB
of scratch space)
exports.ffmpeg = function(event,context) {
new ffmpeg('./thumb.MP4', function (err,
video)
{
if (!err) { video.fnExtractFrameToJPG('/tmp’)
function (error, files) { … }
…
if (!error)
console.log(files);
context.done();
...

9. Lambda considerations and best practices
• Custom CloudWatch metrics
• 40 KB per POST
• Default Acct Limit of 150 TPS
• Consider aggregating with Kinesis
def put_cstate ( iid, state ):
response = cwclient.put_metric_data(
Namespace='AWSx/DirectConnect',
MetricData=[
{
'MetricName':'ConnectionState',
'Dimensions': [
{
'Name': 'ConnectionId',
'Value': iid
},
],
'Value': state,
'Unit': 'None’

10. A couple kinds of design patterns

11. Pattern one: 3-Tier Web Application

12. 3-Tier web application
Data stored in
Amazon
DynamoDB
Dynamic content
in AWS Lambda
Amazon API
Gateway
Browser
Amazon
CloudFront
Amazon
S3
Browser
Amazon
CloudFront
Amazon S3
Amazon API
Gateway
Dynamic content in
AWS Lambda
Data store in Amazon
DynamoDB

13. Amazon API
Gateway AWS
Lambda
Amazon
DynamoDB
Amazon
S3
Amazon
CloudFront
• Bucket Policies
• ACLs
• OAI
• Geo-Restriction
• Signed Cookies
• Signed URLs
• DDOS
IAM
AuthZ
IAM
Serverless web app security
• Throttling
• Caching
• Usage Plans
Static Content
Browser
Lambda@Edge

14. Amazon API
Gateway AWS
Lambda
Amazon
DynamoDB
Amazon
S3
Amazon
CloudFront
• Bucket Policies
• ACLs
• OAI
• Geo-Restriction
• Signed Cookies
• Signed URLs
• DDOS
IAMAuthZ IAM
Serverless web app security
• Throttling
• Caching
• Usage Plans
Static Content
Browser
Amazon
CloudFront
• HTTPS
• Disable Host Header
Forwarding
AWS WAF

15. Amazon API
Gateway
AWS
Lambda
Amazon
DynamoDB
Amazon
S3
Amazon
CloudFront
• Access Logs in S3 Bucket
• Access Logs in S3 Bucket
• CloudWatch Metrics-
https://aws.amazon.com/clo
udfront/reporting/
Serverless web app monitoring
AWS WAF
• WebACL Testing
• Total Requests
• Allowed/Blocked
Requests by ACL
logslogs
• Invocations
• Invocation Errors
• Duration
• Throttled Invocations
• Latency
• Throughput
• Throttled Reqs
• Returned Bytes
• Documentation
• Latency
• Count
• Cache Hit/Miss
• 4XX/5XX Errors
Streams
AWS
CloudTrail
Static Content
Browser
Custom CloudWatch
Metrics & Alarms

16. Serverless web app lifecycle management
• AWS SAM (Serverless Application Model) - blog
AWS
Lambda
Amazon API
Gateway
AWS CloudFormationAmazon
S3
Amazon
DynamoDB
Package &
Deploy
Code/Packages/Swagger
Serverless
Template
Serverless
Template
w/ CodeUri
package deploy
CI/CD Tools

17. A couple words on Amazon API Gateway
• Use mock integrations
• Signed URL from API Gateway for large or binary file uploads to S3
• Use request/response mapping templates for legacy apps and HTTP
response codes
• Asynchronous calls for Lambda > 30s

18. Pattern two: Batch processing

19. Characteristics of batch processing
• Large data sets
• Periodic or scheduled tasks
• Extract Transform Load (ETL) jobs
• Usually non-interactive and long running
• Many problems fit MapReduce programming model

20. AWS Lambda:
Splitter
Amazon S3
Object
Amazon DynamoDB:
Mapper Results
AWS Lambda:
Mappers
….
….
AWS Lambda:
Reducer
Amazon S3
Results
Serverless batch processing

21. Best practices and things to think about
• Cascade mapper functions
• Lambda languages vs. SQL
• Speed is directly proportional to the concurrent Lambda function limit
• Use DynamoDB/ElastiCache/S3 for intermediate state of mapper
functions
• Lambda MapReduce Reference Architecture

22. Costs of serverless batch processing?
• 200 GB normalized Google Ngram data-set
• Serverless:
• 1000 concurrent Lambda invocations
• Processing time: 9 minutes
• Cost: $7.06

23. Pattern three: stream processing

24. Characteristics of stream processing
• High ingest rate
• Near real-time processing (low latency from ingest to process)
• Spiky traffic (lots of devices with intermittent network connections)
• Message durability
• Message ordering

25. Sensors
Amazon Kinesis:
Stream
Lambda:
Stream Processor
S3:
Final Aggregated Output
Lambda:
Periodic Dump to S3
CloudWatch Events:
Trigger every 5 minutes
S3:
Intermediate Aggregated
Data
Lambda:
Scheduled Dispatcher
KPL:
Producer
Serverless stream processing architecture

26. Fan-out pattern
• Number of Amazon Kinesis Streams shards corresponds to concurrent Lambda invocations
• Trade higher throughput & lower latency vs. strict message ordering
Sensors
Amazon Kinesis:
Stream
Lambda:
Dispatcher
KPL:
Producer Lambda:
Processors
Increase throughput, reduce processing latency

27. More about fan-out pattern
• Keep up with peak shard capacity
• 1000 records / second, OR
• 1 MB / second
• Consider parallel synchronous Lambda invocations
• Rcoil for JS (https://github.com/sapessi/rcoil) can help
• Dead letter queue to retry failed Lambda invocations

28. Some event services options
Amazon Kinesis Streams Amazon SQS Amazon SNS
Message Durability Up to retention period Up to retention period Retry delivery (depends on
destination type)
Maximum Retention Period 7 days 14 days Up to retry delivery limit
Message Ordering Strict within shard Standard - Best effort
FIFO – Strict within Message
Group
None
Delivery semantics Multiple consumers per shard Multiple readers per queue (but
one message is only handled by
one reader at a time)
Multiple subscribers per topic
Scaling By throughput using Shards Automatic Automatic
Iterate over messages Shard iterators No No
Delivery Destination Types Kinesis Consumers SQS Readers HTTP/S, Mobile Push, SMS,
Email, SQS, Lambda

29. Some serverless streaming best practices
• Tune batch size when Lambda is triggered by Amazon Kinesis Streams – reduce
number of Lambda invocations
• Tune memory setting for your Lambda function – shorten execution time
• Use KPL to batch messages and saturate Amazon Kinesis Stream capacity

30. Pattern four: automation

31. Automation characteristics
• Respond to alarms or events
• Periodic jobs
• Auditing and Notification
• Extend AWS functionality
• Highly Available and scalable

32. AWS Lambda:
Update Route53
Amazon CloudWatch Events:
Rule Triggered
Amazon EC2 Instance
State Changes
Amazon DynamoDB:
EC2 Instance Properties
Amazon Route53:
Private Hosted Zone
Tag:
CNAME = ‘xyz.example.com’
xyz.example.com A 10.2.0.134
Automation: dynamic DNS for EC2 instances

33. AWS Lambda:
Resize Images
Users upload photos
S3:
Source Bucket
S3:
Destination Bucket
Triggered on
PUTs
Automation: image thumbnail creation from
S3

34. A few tips from someone who knows what
he’s talking about
• Serverless monolith: frameworks like Zappa or Serverless that just create a single
package and route all requests to the one package
• Easy to port existing applications
• Works well with traditional App level logging and monitoring
• Easy to keep all endpoints warm since everything is hooked up
• Start with a monolith then move out individual endpoints as things break
• If you’re building something greenfield you can do the managed endpoints
pattern with a framework like chalice - each APIGW endpoint gets created, but
it’s still a single app deployment
• Then there’s 1:1 every endpoint gets its own function- makes logging and
introspection a nightmare but gives extreme agility for parallel development

35. A few tips from someone who knows what
he’s talking about
• On the non-web app side: Glue pattern
• Glue is what 99% of lambda deployments are about - taking events from one
service and doing something with them in another
• If services are bricks then lambda is mortar

36. Other resources
• Randall <3s Lambda!
• @jrhunt on Twitter
• Tons of examples and projects here: https://github.com/ranman
• AWS documentation:
http://docs.aws.amazon.com/lambda/latest/dg/welcome.html
• Tons of compute blog posts:
https://aws.amazon.com/blogs/compute/category/aws-lambda/
• Lambda reference architecture: https://github.com/awslabs/lambda-
refarch-webapp

37. Thanks!