Diese Präsentation wurde erfolgreich gemeldet.
Wir verwenden Ihre LinkedIn Profilangaben und Informationen zu Ihren Aktivitäten, um Anzeigen zu personalisieren und Ihnen relevantere Inhalte anzuzeigen. Sie können Ihre Anzeigeneinstellungen jederzeit ändern.

ReInvent 2019 reCap Nordics

103 Aufrufe

Veröffentlicht am

ReInvent 2019 reCap Nordics


Veröffentlicht in: Ingenieurwesen
  • Loggen Sie sich ein, um Kommentare anzuzeigen.

ReInvent 2019 reCap Nordics

  1. 1. N o rd i cs Marcia Villalba Developer Advocate, AWS @mavi888uy
  2. 2. © 2020, Amazon Web Services, Inc. or its affiliates. All rights reserved.
  3. 3. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
  4. 4. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
  5. 5. Agenda Topics for the day: • Compute • Storage • Database and analytics • Networking • Serverless • Infrastructure • AI services • ML services
  6. 6. © 2020, Amazon Web Services, Inc. or its affiliates. All rights reserved. Please fasten your seatbelts!
  7. 7. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
  8. 8. Amazon Braket Introducing Fully managed service that makes it easy for scientists and developers to explore and experiment with quantum computing. DRAFTQuantum Technology Preview – December 2 LEARN MORE CMP213: Introducing Quantum Computing with AWS
  9. 9. AWS Compute Optimizer Introducing Identify optimal Amazon EC2 instances and EC2 Auto Scaling group for your workloads using a ML-powered recommendation engine DRAFTManagement Tools General Availability – December 3 LEARN MORE CMP323: Optimize Performance and Cost for Your AWS Compute
  10. 10. AWS Compute Optimizer
  11. 11. Receive lower rates automatically. Easy to use with recommendations in AWS Cost Explorer Significant savings of up to 72% Flexible across instance family, size, OS, tenancy or AWS Region; also applies to AWS Fargate & soon to AWS Lambda usage Compute/Cost Management LEARN MORE CMP210: Dive deep on Savings Plans Announced – November 6 Simplify purchasing with a flexible pricing model that offers savings of up to 72% on Amazon ECS, AWS Fargate & AWS Lambda usage Savings Plans
  12. 12. DRAFTContainers General Availability – December 3 LEARN MORE CON-326R - Running Kubernetes Applications on AWS Fargate Introducing The only way to run serverless Kubernetes containers securely, reliably, and at scale Amazon EKS for AWS Fargate
  13. 13. The Amazon Builders’ Library Architecture, software delivery, and operations By Amazon’s senior technical executives and engineers Real-world practices with detailed explanations Content available for free on the website
  14. 14. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
  15. 15. Amazon S3 Access Points Introducing Simplify managing data access at scale for applications using shared data sets on Amazon S3. Easily create hundreds of access points per bucket, each with a unique name and permissions customized for each application. DRAFTStorage General Availability – December 3
  16. 16. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
  17. 17. Amazon Managed Apache Cassandra Service Introducing A scalable, highly available, and serverless Apache Cassandra–compatible database service. Run your Cassandra workloads in the AWS cloud using the same Cassandra application code and developer tools that you use today. Apache Cassandra- compatible Performance at scale Highly available and secure No servers to manage DRAFTDatabases Preview – December 3 LEARN MORE DAT324: Overview of Amazon Managed Apache Cassandra Service
  18. 18. DRAFTDatabases Announced – November 26 Amazon Aurora Machine Learning Integration Simple, optimized, and secure Aurora, SageMaker, and Comprehend (in preview) integration. Add ML-based predictions to databases and applications using SQL, without custom integrations, moving data around, or ML experience.
  19. 19. With Comprehend
  20. 20. With Sagemaker 45
  21. 21. Amazon RDS Proxy Introducing Fully managed, highly available database proxy feature for Amazon RDS. Pools and shares connections to make applications more scalable, more resilient to database failures, and more secure. DRAFTDatabases Public Beta – December 3 LEARN MORE DAT368: Setting up database proxy servers with RDS Proxy
  22. 22. UltraWarm for Amazon Elasticsearch Service Introducing A low cost, scalable warm storage tier for Amazon Elasticsearch Service. Store up to 10 PB of data in a single cluster at 1/10th the cost of existing storage tiers, while still providing an interactive experience for analyzing logs. DRAFTAnalytics Public Beta – December 3 LEARN MORE ANT229: Scalable, secure, and cost-effective log analytics
  23. 23. Amazon Redshift Data Lake Export New Feature No other data warehouse makes it as easy to gain new insights from all your data. DRAFTAnalytics General Availability – December 3 LEARN MORE ANT335R: How to build your data analytics stack at scale with Amazon Redshift
  24. 24. AWS Data Exchange Quickly find diverse data in one place Efficiently access 3rd-party data Easily analyze data Reach millions of AWS customers Easiest way to package and publish data products Built-in security and compliance controls For Subscribers For Providers DRAFTAnalytics Announced – November 13 L E A R N M O R E ANT238-R: AWS Data Exchange: Easily find & subscribe to third-party data in the cloud Easily find and subscribe to 3rd-party data in the cloud
  25. 25. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
  26. 26. DRAFTManagement Tools Announced – November 21 Identify unusual activity in your AWS accounts ü Save time sifting through logs ü Get ahead of issues before they impact your business CloudTrail Insights Introducing • Unexpected spikes in resource provisioning • Bursts of IAM management actions • Gaps in periodic maintenance activity L E A R N M O R E MGT420-R: CloudTrail Insights: Identify and Solve Operational Issues
  27. 27. AWS Detective Introducing Quickly analyze, investigate, and identify the root cause of security findings and suspicious activities. Automatically distills & organizes data into a graph model Easy to use visualizations for faster & effective investigation Continuously updated as new telemetry becomes available Preview – December 3 DRAFTSecurity LEARN MORE SEC312: Introduction to Amazon Detective
  28. 28. AWS IAM Access Analyzer Introducing Continuously ensure that policies provide the intended public and cross-account access to resources, such as Amazon S3 buckets, AWS KMS keys, & AWS Identity and Access Management roles. General Availability – December 2 DRAFTSecurity Uses automated reasoning, a form of mathematical logic, to determine all possible access paths allowed by a resource policy Analyzes new or updated resource policies to help you understand potential security implications Analyzes resource policies for public or cross-account access LEARN MORE SEC309: Deep Dive into AWS IAM Access Analyzer
  29. 29. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
  30. 30. L E A R N M O R E SVS401 - Optimizing your serverless applications Provisioned Concurrency on AWS Lambda New Feature • Keeps functions initialized and hyper-ready, ensuring start times stay in the milliseconds • Builders have full control over when provisioned concurrency is set • No code changes are required to provision concurrency on functions in production • Can be combined with AWS Auto Scaling at launch DRAFTServerless General Availability – December 3
  31. 31. Achieve up to 67% cost reduction and 50% latency reduction compared to REST APIs. HTTP APIs are also easier to configure than REST APIs, allowing customers to focus more time on building applications. Reduce application costs by up to 67% Reduce application latency by up to 50% Configure HTTP APIs easier and faster than before HTTP APIs for Amazon API Gateway Introducing DRAFTMobile Services Preview – December 4 L E A R N M O R E CON213-L - Leadership session: Using containers and serverless to accelerate modern application development (incl schema registry demo)
  32. 32. AWS Step Functions Express Workflows Introducing Orchestrate AWS compute, database, and messaging services at rates greater than 100,000 events/second, suitable for high-volume event processing workloads such as IoT data ingestion, streaming data processing and transformation. DRAFTApp Integration General Availability – December 3 L E A R N M O R E API321: Event-Processing Workflows at Scale with AWS Step Functions
  33. 33. 76
  34. 34. Amazon EventBridge Schema Registry Introducing Store event structure - or schema - in a shared central location, so it’s faster and easier to find the events you need. Generate code bindings right in your IDE to represent an event as an object in code. DRAFTApp Integration Preview – December 3 LEARN MORE CON213-L - Leadership session: Using containers and serverless to accelerate modern application development (incl schema registry demo)
  35. 35. Lambda Destinations Introducing
  36. 36. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
  37. 37. Container Support for AWS IoT Greengrass New Feature DRAFTInternet of Things Announced – November 25 Deploy containers seamlessly to edge devices Move containers from the cloud to edge devices using AWS IoT Greengrass, without rewriting any code. Enables both Docker & AWS Lambda components to operate seamlessly together at the edge Use AWS IoT Greengrass Secrets Manager to manage credentials for private container registries.
  38. 38. AWS Outposts Now Available Fully managed service that extends AWS infrastructure, AWS services, APIs, and tools to virtually any connected customer site. Truly consistent hybrid experience for applications across on-premises and cloud environments. Ideal for low latency or local data processing application needs. Same AWS-designed infrastructure as in AWS regional data centers (built on AWS Nitro System) delivered to customer facilities Fully managed, monitored, and operated by AWS as in AWS Regions Single pane of management in the cloud providing the same APIs and tools as in AWS Regions Compute General Availability – December 3 LEARN MORE CMP302-R: AWS Outposts: Extend the AWS experience to on-premises environments Wednesday at 11:30am, Aria Thursday at 3:15pm, Mirage Friday at 10:45am, Mirage
  39. 39. Amazon EC2 Amazon EBS Amazon ECS Amazon EKS Amazon EMR Amazon VPC Amazon RDS Amazon S3
  40. 40. Additional AWS Services Supported Locally on Outposts
  41. 41. Local Zones Introducing Extend the AWS Cloud to more locations and closer to your end-users to support ultra low latency application use cases. Use familiar AWS services and tools and pay only for the resources you use. DRAFTCompute General Availability – December 3 The first Local Zone to be released will be located in Los Angeles.
  42. 42. AWS Wavelength Introducing Embeds AWS compute and storage inside telco providers’ 5G networks. Enables mobile app developers to deliver applications with single-digit millisecond latencies. Pay only for the resources you use. DRAFTCompute Announcement – December 3
  43. 43. AWS Wavelength Introducing Embeds AWS compute and storage inside telco providers’ 5G networks. Enables mobile app developers to deliver applications with single-digit millisecond latencies. Pay only for the resources you use. DRAFTCompute Announcement – December 3
  44. 44. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
  45. 45. VISION SPEECH TEXT SEARCH NEW CHATBOTS PERSONALIZATION FORECASTING FRAUD NEW DEVELOPMENT NEW CONTACT CENTERS NEW Amazon SageMaker Ground Truth Augmented AI SageMaker Neo Built-in algorithms SageMaker Notebooks NEW SageMaker Experiments NEW Model tuning SageMaker Debugger NEW SageMaker Autopilot NEW Model hosting SageMaker Model Monitor NEW Deep Learning AMIs & Containers GPUs & CPUs Elastic Inference Inferentia (Inf2) FPGA Amazon Rekognition Amazon Polly Amazon Transcribe +Medical Amazon Comprehend +Medical Amazon Translate Amazon Lex Amazon Personalize Amazon Forecast Amazon Fraud Detector Amazon CodeGuru AI SERVICES ML SERVICES ML FRAMEWORKS & INFRASTRUCTURE Amazon Textract Amazon Kendra Contact Lens For Amazon Connect SageMaker Studio IDE NEW NEW AWS Machine Learning stack NEW
  46. 46. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
  47. 47. Introducing Amazon Transcribe Medical Easy-to-UseAccurate Affordable
  48. 48. Introducing Amazon Rekognition Custom Labels • Import images labeled by Amazon SageMaker Ground Truth… • Or label images automatically based on folder structure • Train a model on fully managed infrastructure • Split the data set for training and validation • See precision, recall, and F1 score at the end of training • Select your model • Use it with the usual Rekognition APIs
  49. 49. Customers are forced to choose ML only systems are high speed and low cost, but do not support nuanced decision making Human only workflows offer nuanced decision making, but they’re low speed and high cost. OR
  50. 50. Customers need + Machine Learning and humans working together
  51. 51. A2I lets you easily implement human review in machine learning workflows to improve the accuracy, speed, and scale of complex decisions. Introducing Amazon Augmented AI (A2I)
  52. 52. How Amazon Augmented AI works Client application sends input data AWS AI Service or custom ML model makes predictions Results stored to your S3 1 2 4 Low confidence predictions sent for human review 3 High-confidence predictions returned immediately to client application 5 Amazon Rekognition Amazon Textract
  53. 53. Human Review Workforces Amazon Mechanical Turk An on-demand 24x7 workforce of over 500,000 independent contractors worldwide, powered by Amazon Mechanical Turk Private A team of workers that you have sourced yourself, including your own employees or contractors for handling data that needs to stay within your organization Vendors A curated list of third-party vendors that specialize in providing data labeling services, available via de AWS Marketplace
  54. 54. Fraud detection is difficult $$$ billions lost to fraud each year Online business prone to fraud attacks Bad actors often change tactics Changing rules = more human reviews Dependent on others to update detection logic
  55. 55. Fraud detection with ML is also difficult Top data scientists are costly & hard to find One-size-fits-all models underperform Often need to supplement data Data transformation + feature engineering Fraud imbalance = needle in a haystack
  56. 56. Introducing Amazon Fraud Detector A fraud detection service that makes it easy for businesses to use machine learning to detect online fraud in real-time, at scale
  57. 57. Amazon Fraud Detector – Key Features Pre-built fraud detection model templates Automatic creation of custom fraud detection models Models learn from past attempts to defraud Amazon Amazon SageMaker integration One interface to review past evaluations and detection logic
  58. 58. Typical Application Build and Run Process Write + Review Build + Test Deploy Measure Improve 1. Code Reviews require expertise in multiple areas such as knowledge of AWS APIs, Concurrency, etc. 2. Code analyzer tools require high accuracy. 3. Distributed Cloud application are difficult to optimize. 4. Performance engineering expertise is hard to find.
  59. 59. Introducing AWS CodeGuru Built-in code reviews with intelligent recommendations Detect and optimize expensive lines of code before production Easily identify latency and performance improvements production environment CodeGuru Reviewer CodeGuru Profiler LEARN MORE Introduction to Amazon CodeGuru (DOP211)
  60. 60. CodeGuru Reviewer: How It Works Input: Source Code Feature Extraction Machine Learning Output: Recommendations Customer provides source code as input Java AWS CodeCommit Github Extract semantic features / patterns ML algorithms identify similar code for comparison Customers see recommendations as Pull Request feedback
  61. 61. CodeGuru Example – Looping vs Waiting do { DescribeTableResult describe = ddbClient.describeTable(new DescribeTableRequest().withTableName(tableName)); String status = describe.getTable().getTableStatus(); if (TableStatus.ACTIVE.toString().equals(status)) { return describe.getTable(); } if (TableStatus.DELETING.toString().equals(status)) { throw new ResourceInUseException("Table is " + status + ", and waiting for it to become ACTIVE is not useful."); } Thread.sleep(10 * 1000); elapsedMs = System.currentTimeMillis() - startTimeMs; } while (elapsedMs / 1000.0 < waitTimeSeconds); throw new ResourceInUseException("Table did not become ACTIVE after "); This code appears to be waiting for a resource before it runs. You could use the waiters feature to help improve efficiency. Consider using TableExists, TableNotExists. For more information, see https://aws.amazon.com/blogs/developer/waiters-in-the-aws-sdk-for-java/ Recommendation Code We should use waiters instead - will help remove a lot of this code.Developer Feedback
  63. 63. CodeGuru Profiler: How It Works Input: Live application stack trace Application profile sampling Pattern matching Output: Method names, Recommendations and searchable visualizations Customer application runs in production CodeGuru Profiler continuously captures application stack trace information CodeGuru Profiler detects performance inefficiencies in the live application Customers see recommendations in their automated efficiency reports and visualizations Amazon Confidential
  64. 64. Employees spend 20% of their time looking for information. —McKinsey 20% 44%44% of the time, they cannot find the information they need to do their job. —IDC
  65. 65. Introducing Kendra Easy to find what you are looking for Fast search, and quick to set up Native connectors (S3, Sharepoint, file servers, HTTP, etc.) Natural language Queries NLU and ML core Simple API and console experiences Code samples Incremental learning through feedback Domain Expertise
  66. 66. Kendra connectors …and more coming in 2020
  67. 67. Getting started with Kendra Step 1 Create an index An index is the place where you add your data sources to make them searchable in Kendra. Step 2 Add data sources Add and sync your data from S3, Sharepoint, Box and other data sources, to your index. Step 3 Test & deploy After syncing your data, visit the Search console page to test search & deploy Kendra in your search application.
  68. 68. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
  69. 69. Introducing Amazon SageMaker Studio The first fully integrated development environment (IDE) for machine learning Organize, track, and compare thousands of experiments Easy experiment management Share scalable notebooks without tracking code dependencies Collaboration at scale Get accurate models for with full visibility & control without writing code Automatic model generation Automatically debug errors, monitor models, & maintain high quality Higher quality ML models Code, build, train, deploy, & monitor in a unified visual interface Increased productivity
  70. 70. Data science and collaboration needs to be easy Setup and manage resources Collaboration across multiple data scientists Different data science projects have different resource needs Managing notebooks and collaborating across multiple data scientists is highly complicated + + =
  71. 71. Introducing Amazon SageMaker Notebooks Access your notebooks in seconds with your corporate credentials Fast-start shareable notebooks Administrators manage access and permissions Share your notebooks as a URL with a single click Dial up or down compute resources Start your notebooks without spinning up compute resources
  72. 72. Data Processing and Model Evaluation involves a lot of operational overhead Building and scaling infrastructure for data processing workloads is complex Use of multiple tools or services implies learning and implementing new APIs All steps in the ML workflow need enhanced security, authentication and compliance Need to build and manage tooling to run large data processing and model evaluation workloads + + =
  73. 73. Introducing Amazon SageMaker Processing Analytics jobs for data processing and model evaluation Use SageMaker’s built-in containers or bring your own Bring your own script for feature engineering Custom processing Achieve distributed processing for clusters Your resources are created, configured, & terminated automatically Leverage SageMaker’s security & compliance features
  74. 74. Managing trials and experiments is cumbersome Hundreds of experiments Hundreds of parameters per experiment Compare and contrast Very cumbersome and error prone + + =
  75. 75. Introducing Amazon SageMaker Experiments Experiment tracking at scale Visualization for best results Flexibility with Python SDK & APIs Iterate quickly Track parameters & metrics across experiments & users Organize experiments Organize by teams, goals, & hypotheses Visualize & compare between experiments Log custom metrics & track models using APIs Iterate & develop high- quality models A system to organize, track, and evaluate training experiments
  76. 76. Debugging and profiling deep learning is painful Large neural networks with many layers Many connections Additional tooling for analysis and debug Extraordinarily difficult to inspect, debug, and profile the ‘black box’ + + =
  77. 77. Automatic data analysis Relevant data capture Automatic error detection Improved productivity with alerts Visual analysis and debug Introducing Amazon SageMaker Debugger Analyze and debug data with no code changes Data is automatically captured for analysis Errors are automatically detected based on rules Take corrective action based on alerts Visually analyze & debug from SageMaker Studio Analysis & debugging, explainability, and alert generation
  78. 78. Deploying a model is not the end, you need to continuously monitor it in production and iterate Concept drift due to divergence of data Model performance can change due to unknown factors Continuous monitoring of model performance and data involves a lot of effort and expense Model monitoring is cumbersome but critical + + =
  79. 79. Introducing Amazon SageMaker Model Monitor Automatic data collection Continuous Monitoring CloudWatch Integration Data is automatically collected from your endpoints Automate corrective actions based on Amazon CloudWatch alerts Continuous monitoring of models in production Visual Data analysis Define a monitoring schedule and detect changes in quality against a pre-defined baseline See monitoring results, data statistics, and violation reports in SageMaker Studio Flexibility with rules Use built-in rules to detect data drift or write your own rules for custom analysis
  80. 80. Successful ML requires complex, hard to discover combinations Largely explorative & iterative Requires broad and complete knowledge of ML domain Lack of visibility Time consuming, error prone process even for ML experts + + = of algorithms, data, parameters
  81. 81. Introducing Amazon SageMaker Autopilot Quick to start Provide your data in a tabular form & specify target prediction Automatic model creation Get ML models with feature engineering & automatic model tuning automatically done Visibility & control Get notebooks for your modelswith source code Automatic model creation with full visibility & control Recommendations & Optimization Get a leaderboard & continue to improve your model
  82. 82. Ground Truth Algorithms & Frameworks Collaborative notebooks ExperimentsDistributed Training & Debugger Deployment, Monitoring, & Hosting SageMaker AutoPilot Build, Train, Deploy Machine Learning Models Quickly at Scale Reinforcement Learning Tuning & Optimization SageMaker Studio Marketplace for ML Amazon SageMaker
  83. 83. AWS DeepRacer improvements • AWS DeepRacer Evo • Stereo camera • LIDAR sensor • New racing opportunities • Create your own races • Object Detection & Avoidance • Head-to-head racing
  84. 84. AWS DeepComposer • The world’s first machine learning-enabled musical keyboard • Compose music using Generative Adversarial Networks (GAN) • Use a pretrained model, or train your own
  85. 85. AWS DeepComposer • The world’s first machine learning-enabled musical keyboard • Compose music using Generative Adversarial Networks (GAN) • Use a pretrained model, or train your own
  86. 86. T h a n k y o u ! Marcia Villalba Developer Advocate, AWS @mavi888uy