Optimizing Your Serverless Applications (SRV401-R2) - AWS re:Invent 2018

© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Optimizing Your Serverless
Applications
Harrell Stiles
Senior Cloud Infrastructure Archictect
AWS
S R V 4 0 1 - R 2

https://secure.flickr.com/photos/mgifford/4525333972
Why are we
here today?

Today’s focus:

Serverless applications
SERVICES (ANYTHING)
Changes in
data state
Requests to
endpoints
Changes in
resource state
EVENT SOURCE FUNCTION
Node.js
Python
Java
C#
Go

Anatomy of a Lambda function
Your
function
Language
runtime
Execution
Environment
Compute
substrate

Your
function
Language
runtime
Execution
Environment
Compute
substrate
Places where
you can
impact
performance

Handler() function
Function to be executed
upon invocation
Event object
Data sent during
Lambda Function
Invocation
Context object
Methods available to
interact with runtime
information (request ID,
log group, etc.)
public String handleRequest(Book book, Context context) {
saveBook(book);
return book.getName() + " saved!";
}

Serverless applications
SERVICES (ANYTHING)EVENT SOURCE FUNCTION

Function myhandler(event, context) {
<Event handling logic> {
result = SubfunctionA()
}else {
result = SubfunctionB()
}
return result;
}
Function subFunctionA(thing){
## logic here
}
Function subFunctionB(thing){
## logic here
}

Import sdk
Import http-lib
Import ham-sandwich
Pre-handler-secret-getter()
Pre-handler-db-connect()
Function myhandler(event, context) {
<Event handling logic> {
result = SubfunctionA()
}else {
result = SubfunctionB()
}
return result;
}
Function Pre-handler-secret-getter() {
}
Function Pre-handler-db-connect(){
}
## logic here
}
## logic here
}

Ephemeral function environment
• Lambda processes a single event per
execution environment
• No need for non-blocking execution
on the frontend
• REMEMBER – execution
environments are reused
• Lazily load variables in the global
scope
• Don’t load it if you don’t need it – cold
starts are affected
Import sdk
Import http-lib
Import ham-sandwich
Pre-handler-secret-getter()
Pre-handler-db-connect()
Function myhandler(event,
context) {
....

Lambda Environment variables
• Key-value pairs that you can dynamically pass to your
function
• Available via standard environment variable APIs such as
process.env for Node.js or os.environ for Python
• Can optionally be encrypted via AWS Key Management
Service (KMS)
• Allows you to specify in IAM what roles have access to the
keys to decrypt the information
• Useful for creating environments per stage (i.e. dev,
testing, production)

AWS Systems Manager – Parameter Store
Centralized store to manage your
configuration data
• supports hierarchies
• plain-text or encrypted with KMS
• Can send notifications of changes
to Amazon SNS/ AWS Lambda
• Can be secured with IAM
• Calls recorded in CloudTrail
• Can be tagged
• Available via API/SDK
Useful for: centralized environment
variables, secrets control, feature flags
from __future__ import print_function
import json
import boto3
ssm = boto3.client('ssm', 'us-east-1')
def get_parameters():
response = ssm.get_parameters(
Names=['LambdaSecureString'],W
ithDecryption=True
)
for parameter in
response['Parameters']:
return parameter['Value']
def lambda_handler(event, context):
value = get_parameters()
print("value1 = " + value)
return value # Echo back the
first key value

Concise function logic
• Separate Lambda handler (entry point) from core logic
• Use functions to TRANSFORM, not TRANSPORT
• Dynamic logic via configuration
•Per function – Environment variables
•Cross function – Amazon Parameter Store/Secrets Manager
• Read only what you need. For example:
•Properly indexed databases
•Query filters in Aurora
•Use S3 select

No orchestration in codeSTARTJOB
JOB#XSTARTED
HTTPPOST
HTTPPOST
AREWETHEREYET?
NOPE!
WE’REDONE!
ZzZz
OR
time.sleep(10)

Keep orchestration out of code.

AWS Step Functions
“Serverless” workflow management with
zero administration:
Makes it easy to coordinate the
components of distributed applications
and microservices using visual workflows
Automatically triggers and tracks each
step, and retries when there are errors, so
your application executes in order and as
expected
Can handle custom failure messages from
Lambda
Task
Choice
Failure capture
Parallel Tasks

Project & repository scoping
1. If functions share an event
source they can go in the same
repo, if not they go in their own
repo as separate “applications”
 Simplifies permissions
2. If functions share an event
source but require varying
different imported packages,
make them their own function
files/jars/etc.
 Keep dependency bloat
minimized per function
Monorepo == anti-pattern for FaaS
Two guiding principles:

Your
function
Language
runtime
Execution
Environment
Compute
substrate
Recap:

The function lifecycle
Bootstrap
the runtime
Start your
code
Full
cold start
Partial
cold start
Warm
start
Download
your code
Start new
Execution
environment
AWS optimization Your optimization

AWS X-Ray Integration with Serverless
• Lambda instruments
incoming requests for all
supported languages and
can capture calls made in
code
• API Gateway inserts a
tracing header into HTTP
calls as well as reports data
back to X-Ray itself
var AWSXRay = require(‘aws-xray-sdk-core‘);
var AWS = AWSXRay.captureAWS(require(‘aws-
sdk’));
S3Client = AWS.S3();

X-Ray Trace Example

Seeing a cold start in AWS X-Ray

Tweak your function’s computer power
Lambda exposes only a memory control, with the % of CPU
core and network capacity allocated to a function
proportionally
Is your code CPU, Network or memory-bound? If so, it could be cheaper
to choose more memory.

Smart resource allocation
Match resource allocation (up to 3 GB!) to logic
Stats for Lambda function that calculates 1000 times all prime numbers
<= 1000000
128 MB 11.722965sec $0.024628
256 MB 6.678945sec $0.028035
512 MB 3.194954sec $0.026830
1024 MB 1.465984sec $0.024638
Green==Best Red==Worst

Smart resource allocation
Match resource allocation (up to 3 GB!) to logic
Stats for Lambda function that calculates 1000 times all prime numbers
<= 1000000
128 MB 11.722965sec $0.024628
256 MB 6.678945sec $0.028035
512 MB 3.194954sec $0.026830
1024 MB 1.465984sec $0.024638
Green==Best Red==Worst
+$0.00001-10.256981sec

Multithreading? Maybe!
• <1.8GB is still single core
• CPU bound workloads won’t see gains – processes share same
resources
• >1.8GB is multi core
• CPU bound workloads will gains, but need to multi thread
• I/O bound workloads WILL likely see gains
• e.g. parallel calculations to return

Lambda execution model
Synchronous
(push)
Amazon
API Gateway
AWS Lambda
function
/order
Asynchronous
(event)
Amazon
SNS
AWS Lambda
function
Amazon
S3
reqs
Poll-based
Amazon
DynamoDB
Amazon
Kinesis
changes
AWS Lambda
service
function

Lambda API
1. Lambda directly invoked
via invoke API
SDK clients
Lambda
function
API provided by the Lambda service
Used by all other services that invoke
Lambda across all models
Supports sync and async
Can pass any event payload structure
you want
Client included in every SDK

If you don’t need a response, execute async
Use the Lambda APIs to start an asynchronous execution
Built-in queue (SQS behind the scenes)
Automatic retries
Dead letter queue for failed events
client = boto3.client("lambda")
client.invoke_async(
FunctionName="test"
InvokeArgs=json_payload
)

The microservices “iceberg”
Common question: “Should every
service of mine talk to another
using an API?”
Maybe not!: Most microservices are
internal only for a given product
supporting their customer facing
features. They may only need to
pass messages to each other that
are simple events and not need a
full fledged interactive API.
Public
interface
Internal
services

Gateways and routers
• Choose suitable entry point for client
applications
• Single, custom client? Use the AWS
SDK
• In region only public API: Use regional
endpoints on API Gateway
• Calls from private microservices in a
VPC: Use private endpoints on API
Gateway
• No need for a custom interface: look at
a non API Gateway sources
• Discard uninteresting events ASAP
• S3 – Event prefix
• SNS – Message filtering

Focusing below the water line
Public
interface
Internal
services

Ways to compare
Pricing
Persistence
Retries
DurabilityScale/Concurrency
controls
Consumption
models

Concurrency across models
SNS/API
No event store
Queue based
Stream based

Lambda Per Function Concurrency controls
• Concurrency a shared pool by default
• Separate using per function concurrency settings
• Acts as reservation
• Also acts as max concurrency per function
• Especially critical for data sources like RDS
• “Kill switch” – set per function concurrency to zero

Scaling/Concurrency Controls
Service Scaling controls
Lambda API Concurrency is point in time, not TPS, can go to 0 up through
maximum for account per region and is shared for all functions in a
region. By default no per function concurrency throttle is set.
SNS Service automatically scales, use Lambda Per Function Concurrency
setting to control downstream consumption.
SQS Service automatically scales, use Lambda trigger Batch size setting
and Per Function Concurrency setting to control downstream
consumption.
Kinesis Streams Shards in a stream: One shard provides ingest capacity of 1MB/sec
or 1000 records/sec, up to 2MB/sec of data output.

Persistence
Service Persistence of requests “in flight”
Lambda API No formal persistence model
SNS No formal persistence model beyond delivery retry logic that extends up
through potentially 13 hours
SQS By default messages are stored for 4 days. This can be modified to as little
as 60 seconds up to 14 days by configuring a queue’s
MessageRetentionPeriod attribute
Kinesis Streams By default data is stored for 24 hours. You can increase this up to 168 hours
(7 days). Extended data retention costs $0.02 per Shard Hour above 24
hours

Retry/failure handling
Service Retry/failure capabilities
Lambda API Retry/failure logic is client dependent for synchronous invocations. For
asynchronous invocations are retried twice by the Lambda service.
SNS If Lambda is not available, SNS will retry 2 times at 1 seconds apart, then 10
times exponentially backing off from 1 seconds to 20 minutes and finally 38
times every 20 minutes for a total 50 attempts over more than 13 hours
before the message is discarded from SNS.
SQS Messages remain in the queue until deleted. They are prevented by being
accessed by other consumers during a period of time known as the “visibility
timeout”. Successful Lambda invocations will cause deletions of messages
automatically. If an invocation fails or doesn’t delete a message during the
visibility timeout window it is made available again for other consumers.
Kinesis Streams When using the Kinesis Client Library (KCL) it maintains a checkpoint/cursor
of processed records and will retry records from the same shard in order
until the cursor shows completion.

Lambda Dead Letter Queues
“By default, a failed Lambda function invoked asynchronously is
retried twice, and then the event is discarded.” –
https://docs.aws.amazon.com/lambda/latest/dg/dlq.html
• Turn this on! (for async use-cases)
• Monitor it via an SQS Queue length metric/alarm
• If you use SNS, send the messages to something durable
and/or a trusted endpoint for processing
•Can send to Lambda functions in other regions
• If and when things go “boom” DLQ can save your invocation
event information
☠️
✉️
Q

Networking

Lambda networking
region
AWS Lambda VPC
Lambda
function
execution
environment

Lambda networking
region
AWS Lambda VPC
Lambda
function
execution
environment
Invocations can only come in
via the AWS Lambda API

Lambda networking
region
AWS Lambda VPC
Lambda
function
execution
environment
Today that API is available
publicly in the region Lambda
is running
Invocations can only come in
via the AWS Lambda API

Lambda networking with a customer configured VPC
region
AWS Lambda VPC Customer VPC
elastic
network
interface
Lambda
function
execution
environment

region
elastic
network
interface
Lambda
function
execution
environment
Completely managed by the
AWS Lambda team
Customer
configured/managed VPC.
Customer controls Security
Groups, Network ACLs, Route
Tables

region
elastic
network
interface
Lambda
function
execution
environment
Invocations still can only
come in via the AWS Lambda
API

region
elastic
network
interface
Lambda
function
execution
environment
Invocations still can only
come in via the AWS Lambda
API
Even with a private API
Gateway endpoint or a VPC
Endpoint provided service

Do I need to put my functions in an Amazon VPC?
Putting your functions inside of a
VPC provides little extra security
benefit to your AWS Lambda
functions

Do I need a VPC?
Should my
Lambda
function be
in a VPC?
Does my function
need to access
any specific
resources in a
VPC?
Does it also need to
access resources or
services in the public
internet?
Don’t put the
function in a
VPC
Put the
function in a
private subnet
Put the
function in a
subnet with a
NAT’d route to
the internet
Yes Yes
No No

Do I need a VPC?
Should my
Lambda
function be
in a VPC?
Do I need to
restrict outbound
access from my
function to the
internet?
Don’t put the
function in a
VPC
Put the
function in a
private subnetYes
No

Basic VPC Design
Lambda
Subnets
--------->
Other
Subnets
--------->
VPC
Availability Zone A Availability Zone B
Subnet Subnet
Subnet Subnet
NAT per
<----- AZ ----->
VPC NAT
gateway
VPC NAT
gateway

Basic VPC Design
• ALWAYS configure a minimum of 2 Availability
Zones
• Give your Lambda functions their own subnets
• Give your Lambda subnets a large IP range to
handle potential scale
• If your functions need to talk to a resource on the
internet, you need a NAT!
• ENIs are a pain, we know, we’re working on it

"Action": "s3:*" makes
puppies cry
Photo by Matthew Henry on Unsplash

Lambda permissions model
Fine grained security controls for both
execution and invocation:
Execution policies:
• Define what AWS resources/API calls can
this function access via IAM
• Used in streaming invocations
• E.g. “Lambda function A can read from
DynamoDB table users”
Function policies:
• Used for sync and async invocations
• E.g. “Actions on bucket X can invoke
Lambda function Z"
• Resource policies allow for cross account
access

Meet
SAM!

https://github.com/awslabs/aws-serverless-samfarm/blob/master/api/saml.yaml
<-THIS
BECOMES THIS->
AWS SAM Templates

AWS SAM Policy Templates
MyQueueFunction:
Type: AWS::Serverless::Function
Properties:
...
Policies:
# Gives permissions to poll an SQS Queue
- SQSPollerPolicy:
queueName: !Ref MyQueue
...
MyQueue:
Type: AWS::SQS::Queue

SAM Policy Templates
45+ predefined policies
All found here:
https://bit.ly/2xWycnj

FIN/ACK
Execution Environment Recap:Your Function Recap:

aws.amazon.com/serverless

Learn more at sessions and breakouts
Thursday, November 29
Leadership Session: Using DevOps, Microservices, and Serverless to
Accelerate Innovation (SRV325)
12:15 – 1:15 PM | Venetian Theatre (Level 2)
A Serverless Journey: AWS Lambda Under the Hood (SRV409-R1)
1:45 PM - 2:45 PM | Numerous overflow rooms
Inside AWS: Technology Choices for Modern Applications (SRV305-R1)
2:30 PM - 3:30 PM | Numerous overflow rooms

Thank you!

Optimizing Your Serverless Applications (SRV401-R2) - AWS re:Invent 2018

Empfohlen

Empfohlen

Weitere ähnliche Inhalte

Mehr von Amazon Web Services

Mehr von Amazon Web Services (20)

Optimizing Your Serverless Applications (SRV401-R2) - AWS re:Invent 2018