Introduction to Amazon EC2
•
7 likes•9,427 views
by Sebastian Dreisch, Business Development Manager, AWS
Recommended
More Related Content
What's hot
What's hot (20)
Similar to Introduction to Amazon EC2
Similar to Introduction to Amazon EC2 (20)
More from Amazon Web Services
More from Amazon Web Services (20)
Recently uploaded
Recently uploaded (20)
Introduction to Amazon EC2
- 1. Amazon EC2 Foundations Sebastian Dreisch, AWS Business Development 24, 2017
- 2. What to expect from this short talk AWS concepts: AWS Regions, Availability Zones Understanding EC2 instance options and how to choose the right one/mix for your workload Understanding Storage options and how to choose the right one/mix for your workload The basics of VPC networking and setting up a load balancer Monitoring, Metrics & Logs Security and Access Control Deployment EC2 Cost Optimization
- 3. AWS global infrastructure 16 regions (a separate geographic area) Each region has multiple, isolated locations known as Availability Zones. Resources aren't replicated across regions unless you do so specifically. 42 Availability Zones *Throughout the next year, the AWS global infrastructure will expand with at least five new Availability Zones in new geographic regions: Ningxia in China, Paris in France.
- 4. AVAILABLILITY ZONES Distinct locations that are engineered to be insulated from failures in other Availability Zones Provide inexpensive, low latency network connectivity to other Availability Zones in the same region Regions contain between 2 & 5 EC2 Availability Zones
- 5. Amazon EC2
- 6. Amazon Elastic Compute Cloud (EC2) - Elastic virtual servers in the cloud Physical Servers in AWS Global Regions Host server Hypervisor Guest 1 Guest 2 Guest n
- 7. Amazon EC2 10+ years ago… • First generation, single instance family and size • m1.small (1 vCPU, 1.7 GiB RAM, 160 GB storage) • Linux only • On-Demand pricing only
- 8. EC2 instances today c4.large Instance family Instance generation Instance size
- 13. aws.amazon.com/ec2/faqs/ Extensive list of supported operating systems & software RedHat Linux, Windows Server, SuSE Linux, Ubuntu, Fedora, Debian, Cent OS, Gentoo Linux, Oracle Linux, and FreeBSD
- 14. STORAGE
- 15. File Amazon EFS Block Amazon EBS Amazon EC2 Instance Store Object Amazon S3 Amazon Glacier
- 17. NETWORKING
- 19. A virtual network in your own logically isolated area within the AWS cloud populated by infrastructure, platform, and application services that share common security and interconnection Amazon VPC aws.amazon.com/vpc/
- 20. ▶ Elastic network interface (ENI) ▶ Subnet ▶ Network access control list (ACL) ▶ Route table ▶ Internet gateway ▶ Virtual private gateway ▶ Route 53 private hosted zone VPC Networking
- 21. VPC Network Topology A VPC can span multiple AZs, but each subnet must reside entirely within one AZ Use at least 2 subnets in different AZs for each layer of your network
- 22. VPC Creation with the VPC Wizard
- 23. Availability Zone 1a Availability Zone 1b Internet 10.0.0.5 10.0.0.6 10.0.3.17 10.0.3.5 10.0.1.5 10.0.1.25 10.0.1.8 10.0.1.6 VPC Subnet VPC Subnet VPC Subnet Virtual Private Gateway Customer Gateway VPN Connection Internet Gateway Customer Data Center
- 24. Example: enterprise application architecture
- 26. ▶ Timeout Configuration ▶ Connection Draining ▶ Cross-zone Load Balancing aws.amazon.com/elasticloadbalancing/
- 27. Example: 3-tier web application architecture
- 28. MONITORING, METRICS & LOGS
- 29. A monitoring service for AWS cloud resources and the applications that you run on AWS. Use Amazon CloudWatch to collect and track metrics, collect and monitor log files, and set alarms. Amazon CloudWatch aws.amazon.com/cloudwatch/
- 31. CloudWatch Metrics in the EC2 Console
- 32. Monitoring Scripts for EC2 Instances docs.aws.amazon.com/AmazonCloudWatch/latest/DeveloperGuide/mon-scripts.html
- 33. Monitor applications and systems using log data Store in a highly durable storage and set retention Access your log files via Web, CLI, or SDK Amazon EC2 (Linux & Windows) AWS Lambda … Amazon CloudWatch Logs docs.aws.amazon.com/AmazonCloudWatch/latest/DeveloperGuide/WhatIsCloudWatchLogs.html
- 34. CloudWatch Metrics & Alarms AWS Resource Your Custom Data Metric Alarm Action CloudWatch
- 35. CloudWatch Logs + Filter AWS Resource Your Custom Data Metric Alarm Action CloudWatch FilterLogs
- 36. Alarm Actions Action Notification (SNS) Auto Scaling action EC2 action Recover Stop Terminate Amazon EC2 Auto Recovery Use this action together with status checks to automate instance recovery
- 37. SECURITY & ACCESS CONTROL
- 38. Consistent, regular, exhaustive 3rd party evaluations • Secured premises • Secured access • Built-in firewalls • Unique users • Multi-factor authentication • Private subnets • Encrypted data storage • Dedicated connection Architected for Enterprise Security
- 39. Access a deep set of cloud security tools Encryption Key Management Service CloudHSM Server-side Encryption Networking Virtual Private Cloud Web Application Firewall Compliance ConfigCloudTrailService Catalog Identity IAM Active Directory Integration SAML Federation
- 40. Access credentials Access key and secret key used to authenticate when accessing AWS APIs Key pairs Public key and private key used to authenticate when accessing an Amazon EC2 instance Security and Access Foundations
- 41. USE IAM ROLES TO PASS ACCESS CREDENTIALS TO AN INSTANCE
- 42. Amazon Lightsail
- 43. DEPLOYMENT
- 45. Amazon maintained Set of Linux and Windows images Kept up to date by Amazon in each region Community maintained Images published by other AWS users Managed and maintained by Marketplace partners Your machine images AMIs you have created from EC2 instances Can be kept private or shared with other accounts
- 46. Bake an AMI Start an instance Configure the instance Create an AMI from your instance Start new ones from the AMI
- 47. Bake an AMI Start an instance Configure the instance Create an AMI from your instance Start new ones from the AMI Configure dynamically Launch an instance Use metadata service and cloud-init to perform actions on instance when it launches
- 48. Bake an AMI Build your base images and set up custom initialization scripts Maintain your ‘golden’ base Configure dynamically Use bootstrapping to pass custom information in and perform post launch tasks like pulling code from SVN +
- 49. Time consuming configuration startup time Static configurations less change management Bake an AMI Configure dynamically
- 50. Continuous deployment latest code Environment specific dev-test-prod Bake an AMI Configure dynamically
- 51. AUTO SCALING
- 52. Maintain EC2 instance availability Detects impaired EC2 instances Replaces the instances automatically Automatically Scale Your Amazon EC2 Fleet Follow the demand curve for your applications Reduce the need to manually provision Amazon EC2 capacity Run at optimal utilisation
- 53. Reusable Instance Templates Provision instances based on a reusable template you define, called a launch configuration. Automated Provisioning Keep your Auto Scaling group healthy and balanced, whether you need one instance or 1,000. Adjustable Capacity Maintain a fixed group size or adjust dynamically based on Amazon CloudWatch metrics.
- 54. Launch Configuration Describes what Auto Scaling creates when adding Instances Only one active launch configuration at a time aws autoscaling create-launch-configuration --launch-configuration-name launch-config --image-id ami-54cf5c3d --instance-type m3.medium --key-name mykey --security-groups webservers Auto Scaling group Auto Scaling managed grouping of EC2 instances Automatically scale the number of instances by policy aws autoscaling create-auto-scaling-group --auto-scaling-group-name autoscaling-group --availability-zones eu-west-1a eu-west-1b --launch-configuration launch-config --load-balancer-names myELB --min-size 1 --max-size 5 Auto Scaling policy Parameters for performing an Auto Scaling action Scale in/out and by how much aws autoscaling put-scaling-policy --auto-scaling-group-name autoscaling-group --policy-name autoscaling-policy --min-adjustment-magnitude=2 --adjustment-type ChangeInCapacity --cooldown 300
- 55. 00:00 01:00 02:00 03:00 04:00 05:00 06:00 07:00 08:00 09:00 10:00 11:00 12:00 13:00 14:00 15:00 16:00 17:00 18:00 19:00 20:00 21:00 22:00 23:00 Utilisation & Auto Scaling Granularity
- 56. 00:00 01:00 02:00 03:00 04:00 05:00 06:00 07:00 08:00 09:00 10:00 11:00 12:00 13:00 14:00 15:00 16:00 17:00 18:00 19:00 20:00 21:00 22:00 23:00 Utilisation & Auto Scaling Granularity 41 Instance Hours m4.large @ $0.133/hr = $5.453/day
- 57. 00:00 01:00 02:00 03:00 04:00 05:00 06:00 07:00 08:00 09:00 10:00 11:00 12:00 13:00 14:00 15:00 16:00 17:00 18:00 19:00 20:00 21:00 22:00 23:00 Utilisation & Auto Scaling Granularity 70 Instance Hours t2.small @ $0.026/hr = $1.82/day
- 58. New Scaling Policies for More Responsive Scaling aws.amazon.com/blogs/aws/auto-scaling-update-new-scaling-policies-for-more-responsive-scaling
- 60. AWS CodeDeploy • Scale from 1 instance to thousands • Deploy without downtime • Centralize deployment control and monitoring • On-premises support Staging CodeDeployv1, v2, v3 Production Dev Coordinate automated deployments, just like Amazon Application Revisions Deployment Groups aws.amazon.com/codedeploy/
- 61. Amazon EC2 Container Service A highly scalable, high performance container management service aws.amazon.com/ecs/ Launch and terminate Docker containers Across a cluster of EC2 instances Mount persistent volumes at launch Private Docker repositories
- 63. On-Demand Pay for compute capacity by the hour with no long- term commitments For spiky workloads, or to define needs Reserved Make a low, one- time payment and receive a significant discount on the hourly charge For committed utilization Spot Bid for unused capacity, charged at a Spot Price which fluctuates based on supply and demand For time-insensitive or transient workloads Dedicated Launch instances within Amazon VPC that run on hardware dedicated to a single customer For BYOL and highly sensitive/regulated workloads Use a purchasing option (mix) that best fits your workload
- 64. Spot Instances Spot Instances are spare Amazon EC2 instances that you can bid on. The Spot price fluctuates in real-time based on supply and demand. When your bid exceeds the Spot Price and Spot capacity is available, your Spot instance is launched and will run until the Spot market price exceeds your bid (a Spot interruption – 2 minute warning!). aws.amazon.com/ec2/purchasing-options/spot-instances/
- 65. Getting Started with Amazon EC2: http://aws.amazon.com/ec2/getting-started/ Auto Scaling Getting Started Tutorial http://docs.aws.amazon.com/AutoScaling/latest/DeveloperGuide/GettingStartedTutorial.html Additional Resources and further Learning
- 66. Certification aws.amazon.com/certification Self-Paced Labs aws.amazon.com/training/ self-paced-labs Try products, gain new skills, and get hands-on practice working with AWS technologies aws.amazon.com/training Training Validate your proven skills and expertise with the AWS platform Build technical expertise to design and operate scalable, efficient applications on AWS AWS Training & Certification
- 67. Email me at dreischs@amazon.com with any questions! Thank you!
Editor's Notes
- We’ve also expanded globally Our data center footprint spans 5 continents with highly redundant clusters of data centers in each region. Our footprint is expanding continuously as we increase capacity, redundancy and add locations You can easily take your application global in minutes And each region has multiple, isolated availability zones, allowing you to place instances and data in multiple locations within the same region.
- Amazon Elastic Compute Cloud is a web service that makes it easy for you to obtain virtual servers, also known as instances quickly, inexpensively, and without making up front capital expenditures Guests/Instances comprise varying combinations of CPU, memory, storage Region, AZ, Instance Type, AMI, PV vs HVM, OS….
- Back when we started EC2, we had a fraction of the functionality we have today We only had a single instance size, the m1.small, which offers 1 vCPU, 1.7 Gibibyte of RAM, and 160 GB of storage We offered Linux operating systems, a single pricing model (On-Demand or by the hour), And missing were many features commonly used with EC2 today, such as Elastic Block Store, Autoscaling, Elastic load balancing, the AWS Management Console, and Elastic IP addresses: An Elastic IP address (EIP) is a static IP address designed for dynamic cloud computing. With an EIP, you can mask the failure of an instance or software by rapidly remapping the address to another instance in your account. Your EIP is associated with your AWS account, not a particular instance, and it remains associated with your account until you choose to explicitly release it.
- Here’s a visualization of the network components of a VPC, which can span availability zones Traffic can be routed from a subnet to the internet, or it can be kept private You can also route subnet traffic to a Virtual Private Gateway which connects via VPC to a customer data center
- Elastic Load Balancer does health checks. If Elastic Load Balancing finds an unhealthy instance, it stops sending traffic to the instance and reroutes traffic to healthy instances. At the same time, Auto Scaling periodically performs health checks on instances. When Auto Scaling determines that an instance is unhealthy, it terminates that instance and launches a new one. Using this functionality across multiple availability zones allows your architecture to fail over to either availability zone, enabling a highly available web architecture within a region You’ll notice as well that static content is delivered through CloudFront our Content Delivery Network
- Elastic Load Balancer does health checks. If Elastic Load Balancing finds an unhealthy instance, it stops sending traffic to the instance and reroutes traffic to healthy instances. At the same time, Auto Scaling periodically performs health checks on instances. When Auto Scaling determines that an instance is unhealthy, it terminates that instance and launches a new one. Using this functionality across multiple availability zones allows your architecture to fail over to either availability zone, enabling a highly available web architecture within a region You’ll notice as well that static content is delivered through CloudFront our Content Delivery Network
- Amazon CloudWatch is a monitoring service for AWS cloud resources and the applications you run on AWS. You can use Amazon CloudWatch to collect and track metrics, collect and monitor log files, and set alarms. Amazon CloudWatch can monitor AWS resources such as Amazon EC2 instances, Amazon DynamoDB tables, and Amazon RDS DB instances, as well as custom metrics generated by your applications and services, and any log files your applications generate. You can use Amazon CloudWatch to gain system-wide visibility into resource utilization, application performance, and operational health. You can use these insights to react and keep your application running smoothly.
- Amazon CloudWatch monitors your Amazon Web Services (AWS) resources and the applications you run on AWS in real-time. You can use CloudWatch to collect and track metrics, which are the variables you want to measure for your resources and applications. CloudWatch alarms send notifications or automatically make changes to the resources you are monitoring based on rules that you define. For example, you can monitor the CPU usage and disk reads and writes of your Amazon Elastic Compute Cloud (Amazon EC2) instances and then use this data to determine whether you should launch additional instances to handle increased load. You can also use this data to stop under-used instances to save money. In addition to monitoring the built-in metrics that come with AWS, you can monitor your own custom metrics. With CloudWatch, you gain system-wide visibility into resource utilization, application performance, and operational health.
- The Amazon CloudWatch Monitoring Scripts for Amazon Elastic Compute Cloud (Amazon EC2) Linux- and Windows-based instances demonstrate how to produce and consume Amazon CloudWatch custom metrics. These sample Perl scripts comprise a fully functional example that reports memory, swap, and disk space utilization metrics for a Linux instance. The scripts for Windows are sample PowerShell scripts that comprise a fully functional example that reports memory, page file, and disk space utilization metrics for a Windows instance. You can download the CloudWatch Monitoring Scripts for Linux and for Windows from the Amazon Web Services (AWS) sample code library and install them on your Linux- or Windows-based instances.
- Your applications and data protected by highly secure facilities and infrastructure, as well as extensive network and security monitoring systems. Additional security measures include: Secure API access –API endpoints allow secure HTTP access (HTTPS) so that you can establish secure communication sessions with your AWS services using SSL. Built-in firewalls – You can control how accessible your EC2 instances are by configuring firewall rules Unique users – The AWS Identity and Access Management (IAM) tool allows you to control the level of access your own users have to your AWS infrastructure services. Multi-factor authentication (MFA) Private Subnets – The AWS Virtual Private Cloud (VPC) service allows you to add another layer of network security to your instances by creating private subnets Encrypted data storage – Customers can have the data stored in Amazon EBS automatically encrypted using Advanced Encryption Standard (AES) 256 Dedicated connection option – The AWS Direct Connect service allows you to establish a dedicated network connection from your premise to AWS.
- To protect your application, AWS invests in a broad portfolio of security, identity, and management tools to help ensure your applications are secure and operate in a compliant manner. --NETWORKING-- Amazon VPC: Amazon Virtual Private Cloud (Amazon VPC) lets you provision a logically isolated section of the Amazon Web Services (AWS) cloud where you can launch AWS resources in a virtual network that you define. You have complete control over your virtual networking environment, including selection of your own IP address range, creation of subnets, and configuration of route tables and network gateways. With Amazon VPC, you can make the Amazon cloud a seamless extension of your existing on-premises resources. AWS WAF: AWS WAF is a web application firewall that helps protect your web applications from common web exploits that could affect application availability, compromise security, or consume excessive resources. AWS WAF gives you control over which traffic to allow or block to your web applications by defining customizable web security rules. You can use AWS WAF to create custom rules that block common attack patterns, such as SQL injection or cross-site scripting, and rules that are designed for your specific application. New rules can be deployed within minutes, letting you respond quickly to changing traffic patterns. Also, AWS WAF includes a full-featured API that you can use to automate the creation, deployment, and maintenance of web security rules. --ENCRYPTION— AWS KMS: AWS Key Management Service (KMS) is a managed service that makes it easy for you to create and control the encryption keys used to encrypt your data, and uses Hardware Security Modules (HSMs) to protect the security of your keys. AWS Key Management Service is integrated with several other AWS services to help you protect your data you store with these services. AWS Key Management Service is also integrated with AWS CloudTrail to provide you with logs of all key usage to help meet your regulatory and compliance needs. AWS CloudHSM: The AWS CloudHSM service helps you meet corporate, contractual and regulatory compliance requirements for data security by using dedicated Hardware Security Module (HSM) appliances within the AWS cloud. With CloudHSM, you control the encryption keys and cryptographic operations performed by the HSM. Server-side Encryption: AWS allows data to be encrypted with AWS service managed keys, AWS managed keys via AWS KMS, or customer managed keys. We also make the AWS Encyption SDK freely available to help developers correctly generate and use encryption keys, as well as protect the key after it has been used. --IDENTITY-- AWS IAM: AWS Identity and Access Management (IAM) enables you to securely control access to AWS services and resources for your users. Using IAM, you can create and manage AWS users and groups, and use permissions to allow and deny their access to AWS resources. AWS Directory Service: AWS Directory Service makes it easy to setup and run Microsoft Active Directory (AD) in the AWS cloud, or connect your AWS resources with an existing on-premises Microsoft Active Directory. Once your directory is created, you can use it to manage users and groups, provide single sign-on to applications and services, create and apply group policy, domain join Amazon EC2 instances, as well as simplify the deployment and management of cloud-based Linux and Microsoft Windows workloads. SAML Federation: AWS IAM supports SAML 2.0 to allow identity integration with most major identity management solutions. [http://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles_providers_saml_3rd-party.html] --COMPLIANCE— AWS Service Catalog: AWS Service Catalog allows organizations to create and manage catalogs of IT services that are approved for use on AWS. These IT services can include everything from virtual machine images, servers, software, and databases to complete multi-tier application architectures. AWS Service Catalog allows you to centrally manage commonly deployed IT services, and helps you achieve consistent governance and meet your compliance requirements, while enabling users to quickly deploy only the approved IT services they need. AWS CloudTrail: AWS CloudTrail is a web service that records AWS API calls for your account and delivers log files to you. The recorded information includes the identity of the API caller, the time of the API call, the source IP address of the API caller, the request parameters, and the response elements returned by the AWS service. With CloudTrail, you can get a history of AWS API calls for your account, including API calls made via the AWS Management Console, AWS SDKs, command line tools, and higher-level AWS services (such as AWS CloudFormation). The AWS API call history produced by CloudTrail enables security analysis, resource change tracking, and compliance auditing. AWS Config: AWS Config is a fully managed service that provides you with an AWS resource inventory, configuration history, and configuration change notifications to enable security and governance. Config Rules enables you to create rules that automatically check the configuration of AWS resources recorded by AWS Config. With AWS Config, you can discover existing and deleted AWS resources, determine your overall compliance against rules, and dive into configuration details of a resource at any point in time. These capabilities enable compliance auditing, security analysis, resource change tracking, and troubleshooting.
- Analytics Complex analytics such as log scanning or simulations, typically performed as batch jobs, can be completed cost-effectively with Spot Instances. Big Data Spot Instances can be used with tools like Amazon Elastic MapReduce to process massive amounts of data, from human genomes to the Twitter fire hose. Financial Modeling and Analysis Financial Services firms use Spot Instances to reduce the time and cost to perform complex analysis ranging from wealth management simulations to Counterparty Value Analytics. Geospatial Analysis Geographic information system (GIS) providers use Spot to speed up and reduce the cost of batch processing jobs such rendering and satellite image processing. Image and Media Encoding Media and Entertainment companies can cost-effectively render and encode media assets using Spot Instances, scaling their infrastructures based on demand. Scientific Computing Scientific researchers and high performance computing customers use Spot to cost-effectively perform simulations ranging from drug discovery to genomics research. Testing Load, integration, canary, and security testing all benefit from the elasticity and price savings associated with Spot Instances. Web Crawling Web crawling processes can easily and cost-effectively scale-out on Spot Instances by leveraging Amazon Elastic MapReduce or other tools to get work done faster and typically cheaper.
- We announced on July 7th that we are making Auto Scaling even more flexible with the addition of new scaling policies with steps. Our goal is to allow you to create systems that can do an even better job of responding to rapid and dramatic changes in load. You can now define a scaling policy that will respond to the magnitude of the alarm breach in a proportionate and appropriate way. For example, if you try to keep your average CPU utilization below 50% you can have a standard response for a modest breach (50% to 60%), two more for somewhat bigger breaches (60% to 70% and 70% to 80%), and a super-aggressive one for utilization that exceeds 80%. Step policies continuously evaluate the alarms during a scaling activity and while unhealthy instances are being replaced with new ones. This allows for faster response to changes in demand. Let’s say the CPU load increases and the first step in the policy is activated. During the specified warm up period (300 seconds in this example), the load might continue to increase and a more aggressive response might be appropriate. Fortunately, Auto Scaling is in violent agreement with this sentiment and will switch in to high gear (and use one of the higher steps) automatically. If you create multiple step scaling policies for the same resource (perhaps based on CPU utilization and inbound network traffic) and both of them fire at approximately the same time, Auto Scaling will look at both policies and choose the one that results in the change of the highest magnitude.
- Analytics Complex analytics such as log scanning or simulations, typically performed as batch jobs, can be completed cost-effectively with Spot Instances. Big Data Spot Instances can be used with tools like Amazon Elastic MapReduce to process massive amounts of data, from human genomes to the Twitter fire hose. Financial Modeling and Analysis Financial Services firms use Spot Instances to reduce the time and cost to perform complex analysis ranging from wealth management simulations to Counterparty Value Analytics. Geospatial Analysis Geographic information system (GIS) providers use Spot to speed up and reduce the cost of batch processing jobs such rendering and satellite image processing. Image and Media Encoding Media and Entertainment companies can cost-effectively render and encode media assets using Spot Instances, scaling their infrastructures based on demand. Scientific Computing Scientific researchers and high performance computing customers use Spot to cost-effectively perform simulations ranging from drug discovery to genomics research. Testing Load, integration, canary, and security testing all benefit from the elasticity and price savings associated with Spot Instances. Web Crawling Web crawling processes can easily and cost-effectively scale-out on Spot Instances by leveraging Amazon Elastic MapReduce or other tools to get work done faster and typically cheaper.
- The video from re:Invent 2014 includes further detail on event notifications and AWS Lambda