AWS Lambda: Serverless Compute for Event-Driven Applications
•
14 gefällt mir•10,324 views
AWS Lambda is a serverless compute platform that allows users to upload code and create functions that can be triggered by events from other AWS services like S3, DynamoDB, SNS, and Kinesis. Lambda handles provisioning and managing servers so users do not have to worry about infrastructure management. It provides a pay-per-use model where users are charged only for the compute time used to process events. The presentation provided examples of using Lambda for image thumbnailing from S3 uploads, sending notifications from DynamoDB updates, and processing streaming data from Kinesis.
Empfohlen
Weitere ähnliche Inhalte
Was ist angesagt?
Was ist angesagt? (20)
Andere mochten auch
Andere mochten auch (20)
Ähnlich wie AWS Lambda: Serverless Compute for Event-Driven Applications
Ähnlich wie AWS Lambda: Serverless Compute for Event-Driven Applications (20)
Mehr von Amazon Web Services
Mehr von Amazon Web Services (20)
Kürzlich hochgeladen
Kürzlich hochgeladen (20)
AWS Lambda: Serverless Compute for Event-Driven Applications
- 1. ©2015, Amazon Web Services, Inc. or its affiliates. All rights reserved A Walk In the Cloud with AWS Lambda Tara E. Walker AWS Technical Evangelist @taraw
- 2. AWS Compute offerings Lambda Server-less compute platform for Stateless, event-driven code execution EC2 Virtual servers in the cloud
- 3. High performance at any scale; Cost-effective and efficient No Infrastructure to manage Pay only for what you use: Lambda automatically matches capacity to your request rate. Purchase compute in 100ms increments. Bring Your Own Code “Productivity focused compute platform to build powerful, dynamic, modular applications in the cloud” Run code in a choice of standard languages. Use threads, processes, files and shell scripts normally. Focus on business logic, not infrastructure. You upload code; AWS Lambda handles everything else. Why Lambda?
- 4. Why Lambda? • You want to thumbnail images as they arrive in an S3 bucket. • You’d like to check that every address stored in Amazon DynamoDB is well formed. “Let’s Examine a Simple Scenario”
- 5. Why Lambda?: • Provision a fleet of proxy machines to capture uploads. • For each upload, enqueue a job to process it. • Provision a second fleet of machines to read and process jobs. • Pick a deployment solution. • Plan capacity, accounting for fault tolerance, good long- term utilization, burst capability, etc. • Monitor 24x7x365 for capacity, health, security, etc. • Migrate to new instance types over time, keeping OS and language runtimes patched and up to date. “For a Simple Scenario there can be complicated solutions”
- 6. The Why of Lambda…. What if there were a better way? Easy to author Easy to scale Easy to manage Easy to maintain Easy to pay for Easy to deploy Easy to maintain
- 7. What is Lambda? • Event-Driven Compute in the Cloud • Launched last November at re:Invent – Lambda functions: Stateless, request-driven code execution – Triggered by events (state transitions) in other services: • PUT to an Amazon S3 bucket • Write to an Amazon DynamoDB table • Record in an Amazon Kinesis stream • Message in an Amazon SQS queue • Transition in an Amazon EC2 instance • Goal: Any API call or resource transition – Makes it easy to… • Transform data as it reaches the cloud • Perform data-driven auditing, analysis, and notification • Kick off workflows
- 8. AWS Lambda or EC2 / ECS? AWS Lambda • Request-driven – Events – Data Triggers • Prioritizes ease of use – One OS – Default hardware choice • AWS owns and manages the infrastructure for you • Implicit scaling; just make requests Amazon EC2 and ECS • Infrastructure rental – Pre-configured AMIs – Build Custom AMIs • Flexible: – Choose instance type – OS – Language, etc.. • You own and configure the infrastructure • Scale by provisioning instances or containers
- 9. Lambda Under the Covers
- 10. How Lambda works S3 event notifications DynamoDB Streams Kinesis events Cognito events SNS events Custom events CloudTrail events LambdaDynamoDB Kinesis S3 Any custom Invoked in response to events Author in familiar language using any libraries; Execute only when needed, automatic scale Redshift SNS Access any service, including your own Any AWS Such as… “Lambda functions”
- 11. Writing Lambda Functions • The Basics – Stock node.js – AWS SDK comes built in and ready to use – Lambda handles inbound traffic • Stateless – Use S3, DynamoDB, or other Internet storage for persistent data – Don’t expect affinity to the infrastructure (you can’t “log in to the box”) • Familiar – Use processes, threads, /tmp, sockets, … – Bring your own libraries, even native ones
- 12. Configuring Lambda functions Simple resource model • Set memory to any size from 128MB to 1GB, in 64MB steps • Receive an equivalent portion of other resources (disk, network, compute power, etc.) • Lambda tells you how much memory you used, so you can tune this setting. Flexible invocation paths • Lambda functions can be invoked “on demand” through CLI and Console • Subscribe to one or many event sources • Reuse the same Lambda function with multiple event sources Granular permissions control (using IAM) • Define what permissions the function has • Uses IAM role (execution role) for granular permission control • Recommended minimum permission – log to CloudWatch • E.g. “read from <X> DDB table only in the context of <Y> function”
- 13. Under the covers – Lambda Functions • Invoke/Call from mobile or web apps – Wait for a response or just send an event and keep going – AWS SDK, AWS Mobile SDK, REST API, CLI • Send events from Amazon S3 or SNS: – One event per Lambda invocation – Unordered model – 3 tries (won’t retry buggy code indefinitely) • Process DynamoDB changes or Amazon Kinesis records as events: – Ordered model with multiple records per event – Unlimited retries (until data expires)
- 14. Under the covers - Invocation models • Push model - Invoke API call – “RequestResponse” mode returns a response immediately – “Event” mode puts function call into a queue, picked up by a poller and then invoked; Used by S3 – Caller (e.g. S3) derives permission through resource policies on Lambda function • Pull model - Subscribe to event source – points poller at a particular data stream; Used by DDB/Kinesis – Poller derived permission from execution role to read from particular DDB update stream
- 15. Under the covers - Invocation permissions • Resource policies – Used in the Push model – Define resource policies attached to a Lambda function – E.g. “User X can invoke on function Y in the context of bucket Z” – Resource policies allow for cross account access! • IAM roles – Used In the pull model – Lambda derives permission from execution role to read from particular Stream – E.g. “User A has permissions to read from Stream B in the context of Function C”
- 16. Monitoring and debugging Lambda Functions • AWS Lambda console includes a dashboard for functions • Lists all Lambda functions • Easy editing of resources, event sources and other settings • At-a-glance metrics • Metrics automatically reported to Amazon CloudWatch for each Lambda function • Requests • Errors • Latency • Throttles • Logs captured by Amazon CloudWatch Logging service
- 17. Real time data processing using Kinesis • Kinesis - A managed service for streaming data ingestion and processing – Data shows up as ordered stream of events supporting multiple readers • All data is stored for 24 hours – Streams are made of Shards • Scale Kinesis streams by adding Shards – Each Shard ingests data up to 1MB/sec and emits data up to 2MB/sec • Using Lambda with Kinesis – Concurrency = Number of Kinesis shards • Increasing shards will cause more Lambda functions invoked concurrently – Batch size = payload size read per shard • Each function call reads serially from a shard • Control how many events are processed per invocation • Increasing batch size will cause fewer Lambda function invocations with more data processed per function
- 20. Scenario #1 - Dynamic data ingestion: Amazon S3 “I want to apply custom logic to process content being uploaded to my data store”. • PDF watermarking • Image thumbnailing and transcoding • Document metadata Indexing • Log aggregation and filtering • RSS feed processing • Media content validation
- 21. Data Triggers: Amazon S3 Amazon S3 Bucket Events AWS Lambda Original image Thumbnailed image 1 2 3
- 22. Data Triggers: Amazon DynamoDB AWS LambdaAmazon DynamoDB Table and Stream Send Amazon SNS notifications Update another table
- 23. Audit and Notify AWS API calls AWS CloudTrail Logs AWS LambdaS3 Bucket events Amazon SNS notifications
- 24. Additional scenarios Cognito CloudFormationLambda Lambda Validate cross device sync data Customized Push notifications SNS Lambda Custom deployment rules Smart workflows (Github, CloudWatch alarms) LambdaDynamoDB DB Triggers (in preview)
- 25. Custom Events AWS LambdaInsert YOUR App Event Here
- 26. AWS Lambda now Generally Available
- 27. Production Release of AWS Lambda • Larger default limits – 100 concurrent executions – 1,000 invokes per second – Increases available via AWS customer service • Preview label removed – Updated API based on feedback during preview – Multiple Lambda functions per stream – Easier to use programming model
- 28. New Features
- 29. Mobile Compute: Building Backends Launching Mobile Compute • Request/response • AWS Mobile SDK – iOS – Android • Easy Personalization …for devices …for end users AWS LambdaMobile App
- 30. Mobile backends • Synchronous invocations – Lambda will return a response as soon as the function finishes executing (JSON in, JSON out) • Mobile SDK integration (Android, iOS) – Can use @LambdaFunciton to annotate app methods and map them to Lambda function calls • Additional information in “context” – When invoked through the mobile SDK, access to information about the device, app and Cognito identity. Client context includes appVersion, appBuild, appPackageName, appName,devicePlatformVersion, devicePlatform, deviceManufacturer,deviceModel,deviceLocale
- 31. New AWS Event Sources for Lambda • Amazon Simple Notification Service (SNS) – Easily target, route, transform, filter, or audit messages – Trigger an AWS Lambda function by sending it notifications – Turn Amazon CloudWatch alarms into actions Lambda FunctionSNS
- 32. New AWS Event Sources for Lambda • Amazon Cognito – Before: Easily synchronize user data across their devices – Now: Take action when that data changes – Examples: Verify and respond to game state updates Lambda FunctionAmazon Cognito
- 33. New Authorization and Auditing Features • Easier setup and configuration – No need for invocation roles – Use resource policies to enable senders to call your function • Cross-account access support – Securely grant execution access to third parties – Send events from S3 buckets in a different AWS account • AWS CloudTrail integration – Track AWS Lambda API calls in AWS CloudTrail logs – Audit access logs easily with a Lambda function
- 34. Improved Metrics and Diagnostics • Discover and take action when you reach your concurrent execution limits – New Amazon CloudWatch throttling metric • Find relevant log entries faster – Sort CloudWatch Logs by time of last entry – See creation time in log stream names and easily filter on it – View the target log stream in your Lambda function
- 35. Java • You can already call Java programs from Lambda functions today… – Java and other languages are automatically included in your filesystem view…don’t wait to start using them! – Freezing ensures you don’t pay repeatedly for JVM boot • We’ll make this even easier with built-in support for AWS Lambda functions written in Java.
- 36. How can you use these features? “I want to send customized messages to different users” SNS + Lambda “I want to send an offer when a user runs out of lives in my game” Amazon Cognito + Lambda + SNS “I want to transform the records in a click stream or an IoT data stream” Amazon Kinesis + Lambda
- 37. Demo: Building a Mobile Backend with AWS Lambda
- 38. Three Next Steps 1. Go to the AWS Management Console to create and test your first Lambda function. The first 1M requests each month are on us! 2. Use the AWS Mobile SDK and Lambda to quickly create an instantly scalable mobile app. 3. Use AWS Lambda to add custom logic to S3, DynamoDB, SNS, Amazon Kinesis, or Amazon Cognito events…no servers required!
- 39. Next steps • Lambda console based walkthroughs – Hello World – Log Amazon S3 update to CloudWatch – Log Amazon Kinesis stream update to CloudWatch – Build a mobile backend • Lambda CLI based walkthroughs – Hello World – Generate thumbnails from S3 uploads – Log Amazon DynamoDB update to CloudWatch logs – Log Amazon Kinesis stream update to CloudWatch – Detect patterns in CloudTrail logs using S3
- 40. Visit http://aws.amazon.com/lambda, the AWS Compute blog, and the Lambda forum to learn more and get started using Lambda. Q & A
- 41. Thank You @taraw
Hinweis der Redaktion
- Amazon EC2 presents a true virtual computing environment, allowing you to use web services interfaces to launch instances with a variety of operating systems, load them with your custom application environment, manage your network’s access permissions, and run your image using as many or few systems as you desire. You use Amazon EC2 simply by: Selecting a pre-configured AMI or creating your own,. Configure security and network access on your Amazon EC2 instance. Choose which instance type(s) you want, then start, terminate, and monitor as many instances of your AMI as needed Determine whether you want to run in multiple locations, utilize static IP endpoints, or attach persistent block storage to your instances. Leverage the ECS service
- Focus on business logic, not infrastructure Design Requirements for Lambda Infrastructure Main consideration was the Infrastructure Challenge So to allow you to focus on the business logic Lambda has to… Upload your code; Lambda handles Capacity Scaling Deployment Fault tolerance Monitoring Logging Web service front end Security patching Automatic Scaling (Have to get his right) So instead of renting compute capacity you give us code and tell us when to run it Looks like S3 or Dynamo’s request model With Lambda it’s a request to say please run my code Each object uploaded to Amazon S3 is an event Each event is a Lambda request Lambda scales to match the event rate of invocation Don’t worry about over or under provisioning just like you do when you use S3 Pay only for what you use; Only do work when you give us a request Bring Your Own Code Lambda is a different type of paradigm, this is not about taking an existing web app and running it in a different way, EC2 is a great existing solution, we have Elastic Beanstalk and lots of other mechanisms that works well today But if you want to run code that doesn’t provide lockin and provides flexibility Create threads and processes, run batch scripts or other executables, and read/write files in /tmp. Design requirement bring any library. Include any library with your Lambda function code, even native libraries. Fine-grained pricing Buy compute time in 100ms increments Low request charge No hourly, daily, or monthly minimums No per-device fees Never pay for idle Free Tier: 1M requests and 400,000 GB-s of compute. Every month, every customer.
- What if you wanted to upload an image convert them into thumbnails or other conversion Or Maybe you have a DynamoDB table and you want to verify that the addresses are correct and if not correct them. Now these problems are probably very few lines of code to execute.
- But turning those few lines of code to a highly scalable, available web service takes some work - Provisioning of Machines Need a durable queue Need some machines to do the actual work of transforms Then you got to deploy the code possible many times How many machines, capacity, how bursty should this be did I provisi What about monitoring and auditing How do I keep it up to date, migration What about security patching, Hardware types This is all to be done for day one
- “Simple problems deserve simple solutions.” What spurred the Lambda creation was a mechanism to take use cases like this, and create one where you could tie the code to the data as it entered the cloud. To have a solution who’s complexity that matches the complexity of the code. So the thought was what if every service in AWS could generate events, could tell you when it’s done something interesting, an API has been called or it’s going through a transition. Having the events are not enough, what if you could respond to these events? Easy to Author Easy to Manage Easy to Deploy Easy to Maintain This is where a full stack solution is so complicated, because if you have 10 lines of code deploying a full stack solution seems like a mismatch
- Compute, sized to the problem at hand Taking stateless request driven code that we call Lambda functions and triggering these on all of these events. So if something happens in an AWS Service you can have an action to respond to that. So you can take a …lifecycle transition into EC2 and turn it into a code execution Making Lambda like connection pipe for all of AWS. The thing that allows you to take these API call, resources changes, events, transition into a code responses.
- So a question that always comes up after I explain what is Lambda is what is different than what we have today - Request vs. infrastructure rental: You tell us when you want work to be done, you call into the API or do something with an AWS Service that triggers Designed with prespective of Easy to use, Productivity in Mind Fixed Operation sys Fixed Language But very simply to get started Contrasting that to EC2 where there are a whole lot of workloads which may require different hardware, Operating systems, and so if you have that flexibility need, EC2 is the right answer. But if you just have some code that needs to run and you are indifferent on what version of Linux you are running them Lambda is the answer EC2 you own and manage the infrastructure with Lambda we own it to take away the complexity/burden for the developer Scaling model is not by instance but by making requests. You ask for the work you want to get done and behind the scenes we handle the compute capacity and where to land the requests.
- So the first thing that was thought of when creating Lambda was
- Launching with Node.JS more language coming down the line Built in is the AWS SDK so we wanted to make it really easy to call any other AWS Service and set it up so you can immediately start calling Graphic library is included, purely for convenience ImageMagic You can bring your own version of ImageMagic or any other Graphics library Key here is that Lambda is stateless vs EC2, different requests make land on different parts of the infrastructure. If you want to persist data you should use DynamoDB or S3 Tried to make to familiar, you can write data to tmp while executing and bring you own library Very few restrictions: - Not able to inbound socket connections we’re doing the frontend for you
- Pick a name for the function and pick a memory size We ask for memory so we are doing are size all the resources at one. So if you ask for 512 vs. 256 you get twice the compute power along with twice the memory Single dial to tune your price performance ratio How do you know? Provide monitoring for error rates in case you have provided too little and latency, stamp into log results how much memory you requested and how much did I actual use. Easy to see if you have allocated too much and turn it down to save money
- So we talked about one Scenario Take images with a few line of code and Thumbnail images, compress files, synchronize object metadata with Amazon DynamoDB tables Turn Amazon S3 into active storage Easily audit AWS CloudTrail logs with AWS Lambda functions Turn Amazon S3 into active storage
- Thumbnail images, compress files, synchronize object metadata with Amazon DynamoDB tables Turn Amazon S3 into active storage Easily audit AWS CloudTrail logs with AWS Lambda functions Turn Amazon S3 into active storage
- Data that you put into DynamoDB can generate change events. An example is the new DynamoDB feature called DynamoDB Streams which was announced at re:Invent. Streams are delta events, they tell you want happened to a table by showing you new data, old data or both as the data is changing in the table. There are lots of usage for this feature …but this can be even more interesting when you hook this up to Lambda As you put data in, you can check an address, correct an address, make entries to another table, in the table that verify data formats, audit out-of-range values, filter and copy data to other tables
- The means that there are lots of interesting things for auditing If you are familiar with CloudTrail, for any API call can generate a Log file. Log AWS API usage with AWS CloudTrail Use Amazon S3 bucket events and AWS Lambda to automate the log analysis and decide what to do with it. Detect suspicious activity or anomalies and notify via Amazon SNS, email, or logs Same technique can be used for any S3 log data
- Compute, sized to the problem at hand
- This can take custom event from any application Call AWS Lambda’s public API like the asyncInvoke API directly from your application, providing a custom event as (json) to kick off a process
- verify data formats, audit out-of-range values, filter and copy data to other tables
- verify data formats, audit out-of-range values, filter and copy data to other tables
- verify data formats, audit out-of-range values, filter and copy data to other tables
- verify data formats, audit out-of-range values, filter and copy data to other tables