5. Boot from Amazon EBS» Amazon CloudFront Streaming » Amazon VPC enters Unlimited Beta » AWS Region in Northern California » International Support for AWS Import/Export » AWS Multi-Factor Authentication » Virtual Private Cloud » Lower Reserved Instance Pricing » Reserved Instances in EU Region » Elastic MapReduce » SQS in EU Region » Amazon RDS » High-Memory Instances » Lower EC2 Pricing » New SimpleDB Features » FPS General Availability » Amazon SNS » AWS Security Center 2009 Jan 2010 Jan Jul Sep Oct Dec Aug Nov Feb Mar Apr Jun May Feb Mar » Amazon EC2 with Windows » Amazon EC2 in EU Region » AWS Toolkit for Eclipse » Amazon EC2 Reserved Instances » Amazon CloudFront Private Content » SAS70 Type II Audit » AWS SDK for .NET » Amazon Elastic MapReduce in Europe » Amazon EC2 Reserved Instances with Windows, Extra Large High Memory Instances » Amazon S3 Versioning Feature » Consolidated Billing for AWS » Lower pricing for Outbound Data Transfer » AWS Import/Export » New CloudFront Feature » Monitoring, Auto Scaling & Elastic Load Balancing » EBS Shared Snapshots » SimpleDB in EU Region » Monitoring, Auto Scaling & Elastic Load Balancing in EU » Lower pricing tiers for Amazon CloudFront » AWS Management Console The pace of innovation in 2009
6. » Free Monitoring EC2 » Amazon Route 53 » PCI DSS Level 1 Certification » Mobile SDKs (Android, iPhone) » Large Object S3 Support » Florida POP » Import/Export APAC » Amazon SNS » Combined AWS Data Transfer Savings » Amazon EMR Bootstrap Actions » Amazon ELB Session Stickiness » Amazon RDS in EU » New Singapore Region » RDS Reserved » CloudFront Default Root » Startup Challenge 2010 » CloudFront Invalidation » AWS Elastic Beanstalk » Amazon Simple Email Service » Improved AWS Support “Bronze” » Amazon CloudWatch Console » CloudFront HTTPS » NYC Edge Location » Lowers Pricing HTTP » AWS Import Export GA » Amazon SNS » Amazon S3 Console » Amazon EBS CloudWatch » Amazon RDS Read Replicas » Suse EC2 Linux » Amazon SNS Console » Amazon ELB HTTPS » AWS Free Tier » EMR Resizing Cluster » EMR JobFlow Debugging » Simple DB Consistent Reads » Simple DB Conditional Puts » VM Connector » Tokyo Region » AWS Support JP 2010 Jan 2011 Jan Jul Sep Oct Dec Aug Nov Feb Mar Apr Jun May Feb Mar » New VPC » Dedicated Instances » Windows 2008 R2 » Amazon S3 Lowered Pricing » CloudFront GA, SLA » S3 Multipart » GPGPU Instance Types » ISO27001/2 Certification » Amazon SQS Longer retention, Free Tier Amazon S3 Bucket Policies » Amazon VPC IP Address » Cluster Compute Instances » Amazon S3 RRS Notifications » AWS Java SDK » Windows BYOL » Singapore Pop » CloudFront Private Streaming » Lowered Pricing EC2 » AWS IAM » Amazon VPC Console » Micro Instances » Amazon Linux AMI » Amazon EC2 Tagging, Filtering, Idempotency, » Oracle Certified AWS » AWS PHP SDK » AWS CloudFormation » Amazon S3 Static Websites » AWS IAM Website Login » Paris Edge Location » Amazon EC2 Reserved Instances with Windows, Extra Large High Memory Instances » Amazon S3 Versioning Feature » Consolidated Billing for AWS » Lower pricing for Outbound Data Transfer » VPC in EU » Amazon RDS in US-west » Amazon CloudFront Access Logs » Amazon RDS Multi-AZ » Amazon S3 RRS » Amazon RDS Console And pace accelerates in 2010….
7. “Every day is a launch day” » On-demand Red Hat » Stockholm Edge Location » AWS Elastic Beanstalk new enhancements » New Data Transfer pricing » Free Inbound Data Transfer » Spot Integration with HPC instances » Amazon EMR in APAC » AWS Mobile SDKs » Live Streaming with CloudFront » AWS IAM GA » AWS IAM Web Console » AWS Import/Export for EBS » AWS CloudFormation new features » AWS SDK for Ruby » Attachment support for Amazon SES » AWS Startup Challenge goes global » AWS DirectConnect » Amazon VPC Everywhere » Mulit-AZ VPC » AWS IAM Identity Federation » AWS toolkit of eclipse 2.0 » AWS GovCloud US » Spot in Amazon EMR » Amazon ElastiCache » Amazon VM import Win2k3 » VM Connector » Tokyo Region » AWS Support JP » AWS IAM for CloudFront » VPC Virtual Networking » VPC Internet Access 2011 Jan 2012 Jan Jul Sep Oct Dec Aug Nov Feb Mar Apr Jun May Feb Mar » AWS CloudFormation » Amazon S3 Static Websites » AWS IAM Website Login » Paris Edge Location » Amazon Route53 » New VPC » Dedicated Instances in VPC » Windows 2008 R2 » New AZ in JP » AWS IAM GA » AWS IAM Web Console » AWS Beanstalk Tomcat 7 Support » Amazon CloudWatch Custom Metrics » Amazon CloudWatch lower pricing » AWS SAP Certification » Amazon RDS for Oracle » Amazon ELB ipv6 support, Zone Apex » Amazon ELB Security Group integration » Amazon Route53 GA, ELB integration » Amazon Route 53 Weighted RR » New pricing control for Spot » AWS CloudFormation new enhancements » AWS Mobile SDK GA » AWS Toolkit of Visual Studio » AWS DirectConnect USWest Location » AWS Elastic Beanstalk » Amazon Simple Email Service » Improved AWS Support “Bronze” » Amazon CloudWatch Console
8. Each day, AWS adds the equivalent server capacity to power Amazon when it was a global, $2.76B enterprise (circa 2000)
9. GovCloud-US US West (Northern California) US East (Northern Virginia) Europe West (Dublin) Asia Pacific Region (Singapore) Asia Pacific Region (Japan) Ashburn, Dallas, Los Angeles, Miami, Newark, Palo Alto, Seattle, St. Louis, Amsterdam, Dublin, Frankfurt, London, Hong Kong, Singapore, Tokyo, New York, Paris Amazon CloudFront Edge Locations
10. The AWS Cloud Your Application Tools to access services Libraries and SDKs .NET/Java etc. Web Interface Management Console Tools AWS Toolkit Eclipse, VS Command Line Interface Cross Service features Auth, Authorization, FederationAWS IAM, MFA Monitoring Amazon CloudWatch Deployment and Automation AWS Elastic BeanstalkAWS CloudFormation High-level building blocks Content Delivery Amazon CloudFront Email Amazon SES Payments Amazon DevPay Amazon FPS Parallel Processing Amazon Elastic MapReduce Messaging Amazon SNS Amazon SQS Workforce Amazon Mechanical Turk Low-level building blocks Compute Amazon EC2 Auto Scaling Network Amazon VPC, ELB, DirectConnect Amazon Route 53 Storage Amazon S3 Amazon EBS Database Amazon RDS Amazon SimpleDB Amazon ElastiCache Amazon Global Physical Infrastructure (Geographical Regions, Availability Zones, Edge Locations)
12. Corporate data center Availability Zone 1 DirectConnect Location 10G VPC Subnet Router VPN Gateway Customer Gateway Corporate Headquarters VPC Subnet Internet Gateway Amazon VPC Availability Zone 2 Branch Offices The New Cloud-Ready Enterprise IT Amazon S3 Amazon SES Amazon SimpleDB Amazon SQS AWS Region
13. The “Living” AWS Cloud Your Application Tools to access services Libraries and SDKs .NET/Java etc. Web Interface Management Console Tools AWS Toolkit Eclipse, VS Command Line Interface Cross Service features Auth, Authorization, FederationAWS IAM, MFA Monitoring Amazon CloudWatch Deployment and Automation AWS Elastic BeanstalkAWS CloudFormation High-level building blocks Content Delivery Amazon CloudFront Email Amazon SES Payments Amazon DevPay Amazon FPS Parallel Processing Amazon Elastic MapReduce Messaging Amazon SNS Amazon SQS Workforce Amazon Mechanical Turk Low-level building blocks Compute Amazon EC2 Auto Scaling Network Amazon VPC Elastic LB Amazon Route 53 Storage Amazon S3 Amazon EBS Database Amazon RDS Amazon SimpleDB Amazon ElastiCache Amazon Global Physical Infrastructure (Geographical Regions, Availability Zones, Edge Locations)
23. The Automation You Always Meant to Build Provision and attach 1TB of storage in 2 minutes (from the back of an auto-rickshaw in India). 10 new Linux servers in 2 minutes (while sitting by the pool on a nice day). Monitoring server resources from an iPhone (in a bar). Source: Autodesk
25. www.yourApp.com media.yourApp.com (Static data) Input Parameters Resources Outputs JSON Plain Text Perfect for Version Control Validate-able Mappings Custom Metadata Amazon CloudFront Amazon Route 53 Elastic Load Balancer Amazon CloudWatch JSON Template AWS CloudFormation Service Amazon S3 Bucket Amazon SNS Notifications Auto Scaling Group Atomically creates and destroys groups of AWS Cloud Resources Amazon SimpleDB App Tier Configures the resources Multi-Tier or Multi-AZ stacks Manages the ordering of provisioning Email ElastiCache Tier Amazon RDS AZ-1 Rolls back in case of failure Or issues AZ-1 Region
48. Custom Metadata "Resources": {"Ec2Instance": { "Type": "AWS::EC2::Instance", "Metadata": { "Comment": "This metadata is available via the cfn-describe-stack-resource command line tool or the DescribeStackResource API call", "MyAMI": { "Fn::FindInMap": [ "RegionMap", { "Ref": "AWS::Region“ }, "AMI"]}, "MyRegion": {"Ref": AWS::Region"}, "MyStack": {"Ref": "AWS::StackName"} }, "Properties": { "ImageId": {"Fn::FindInMap": ["RegionMap",{"Ref": "AWS::Region"}, AMI"]}, "UserData": { "Fn::Base64": "80"} } } },
49. Standardized Application Stacks Apache Apache IIS Apache Mongrel Tomcat ASP.NET Mongrel Web Server Rails Struts ASP.NET MVC Rails App Server Your Code Your Code Your Code Your Code MVC logger Log4J Log4Net logger Your Code RubyGems Spring Spring.NET RubyGems Libraries memcached Hibernate nHibernate memcached Packages Ruby Runtime JEE .NET Ruby Runtime DB Caching Centos Linux Windows Centos Framework OS Java Stack .NET Stack RoR stack
51. Implement Elasticity 1. Frozen Pizza Model IIS IIS IIS IIS Apache Apache IIS IIS IIS IIS Tomcat Tomcat ASP.NET MVC ASP.NET MVC ASP.NET MVC ASP.NET MVC Struts Struts Your Code Your Code Your Code Your Code Your Code Your Code Log4Net Log4Net Log4Net Log4Net Log4J Log4J Spring.NET Spring.NET Spring.NET Spring.NET Spring Spring nHibernate nHibernate nHibernate nHibernate Hibernate Hibernate .NET .NET .NET .NET JEE JEE Amazon EC2 Windows Windows Windows Windows Linux Linux Java AMI Java Stack
52. Build Job does the following: build the artifact, publish it to Artifactory, build the package, publish the package to the repo . Then there is a follow on job that mounts a base OS image, installs the packages and then creates the final AMI. Source: http://techblog.netflix.com/2011/08/building-with-legos.html
53. Implement Elasticity 2. Take N Bake Pizza Model Apache Your Code Amazon S3 Tomcat Struts Log4J Spring Fetch on boot time Apache Struts Tomcat Source Control Hibernate Your Code JEE Linux Log4J Spring IIS IIS IIS IIS Hibernate IIS IIS IIS IIS JEE .NET .NET .NET .NET Linux Amazon EC2 Windows Windows Windows Windows Golden AMI Java Stack
57. Implement Elasticity 3. Made to Order Pizza Model Amazon S3 Apache Apache Struts Tomcat Log4J Hibernate Your Code Spring Tomcat Struts Cookbooks Recipes Source Control Your Code PuppetMaster Log4J Spring Hibernate JEE PuppetClient Agent Linux Linux Windows Amazon EC2 AMI (JeOS) Java Stack
63. Best Practices Puppet is great for incremental implementation! All modules and manifests should be kept under version control. Manage users and groups from the outset. Puppet Environments are your friend Skinny classes, fat modules. Use 'notify' for logging. Make it easy to check logs. ‘The Trifecta‘- Use the Package, file, service.
64. Implement Elasticity 3 approaches to designing your AMIs Easier to Setup Inventory of fully baked AMIs (Frozen/Ready made) “Golden AMIs” with fetch on boot (Take N’ Bake) AMIs with JeOSand PuppetMaster (Made to Order) More Control Easier to maintain
65. More Tools: CloudFormer Create a template from the running resources in your account Select the resources that should be included Customize the logical names Define the template output section Creates a starting point template for your to edit Add parameters Abstract properties and flow properties One-click launch in your account CloudFormer is an appliance that runs in your account
67. Optimizing = Cost Savings Free Memory Free CPU Free HDD At 1-min intervals PUT 2 weeks Alarm Amazon CloudWatch Instance Custom Metrics “You could save a bunch of money by switching to a small instance, Click on CloudFormation Script to Save”
68.
69. Elasticity is the fundamental property of the cloud and implement elasticity
74. Let go of (physical) control but retain your ownership
75. Enterprise Security Features Amazon VPC AWS Identity And Access Management User management Policy-based granular access control Web login to individual users Identity Federation (New!) Multi-Factor Authentication Services Security features Amazon S3 ACL and Bucket policies Amazon EC2 Security Groups, iptables HTTPS API Endpoints
76. SAS 70 Type II Audit ISO 27001/2 Certification PCI DSS 2.0 Level 1-5 HIPAA/SOX Compliance FISMA A&A Low Encrypt data in transit Encrypt data at rest Protect your AWS Credentials Rotate your keys Secure your application Enforce IAM policies Use MFA, VPC, Leverage S3 bucket policies, EC2 Security groups, EFS in EC2 Etc.. In the Cloud, Security is a Shared Responsibility How we secure our infrastructure How can you secure your application and what is your responsibility? What security options and features are available to you?
This is highly relevant in the cloud world because innovation is happening at break neck speed…..
Lets put everything in a context of a web application
See the animation. DirectConnect
Autodesk story – Paul Cochrane…
The key advance was using our continuous build system to build not only the artifact from source code, but the complete software stack, all the way up to a deployable image in the form of an AMI (Amazon Machine Image for AWS EC2).
Cloud-init supports several different mechanisms for passing data to the instance including ways to pass larger, more structured data and a way to provide a script that is executed at instance launch time.
Till now people who wanted to evaluate MCollective had to go through a manual process of starting first the ActiveMQ instance, gathering some data and then start a number of other instances supplying user data for the ActiveMQ instance. This was by no means a painful solution but CloudFormation can make this much better.
Remember TVs without remote control. We had to walk up to the television set (hardware) to change the channel or the volume. Now we have remote controls, we sit back relax on our couch and control the hardware with our finger tips. Cloud APIs are remote control of the cloud hardware. You don’t have the need to walk up to your hardware, you don’t have to hug your servers anymore. You let go of your (physical) control but retain your ownership
Service OptimizationsArchitectural recommendationsHealth ChecksSecurity Audits