Black Belt AWS Tips on Compute, Storage, Deployment & Security
•
2 gefällt mir•964 views
AWS Summit 2014 Melbourne - Breakout 5 Technical deep dive in to 10 AWS Cloud best practices with in-depth look at the tips and tricks of architecting on the AWS platform. Presenter: Dean Samuels, Solutions Architect, Amazon Web Services
Empfohlen
Empfohlen
Weitere ähnliche Inhalte
Was ist angesagt?
Was ist angesagt? (20)
Andere mochten auch
Andere mochten auch (20)
Ähnlich wie Black Belt AWS Tips on Compute, Storage, Deployment & Security
Ähnlich wie Black Belt AWS Tips on Compute, Storage, Deployment & Security (20)
Mehr von Amazon Web Services
Mehr von Amazon Web Services (20)
Kürzlich hochgeladen
Kürzlich hochgeladen (20)
Black Belt AWS Tips on Compute, Storage, Deployment & Security
- 1. Black Belt Tips on AWS Dean Samuels, Solutions Architect, Amazon Web Services © 2014 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified, or distributed in whole or in part without the express consent of Amazon.com, Inc.
- 2. AWS Rapid Pace of Innovation! +48! E!lastic Load! Balancing! Auto Scaling! Amazon VPC! Amazon RDS! 2009! +61! Amazon SNS! !AWS Identity ! & Access ! ! Management! Amazon Route 53! 2010! +82! Amazon SES! !AWS Elastic ! Beanstalk! !AWS ! CloudFormation! !Amazon ! ElastiCache! !AWS Direct ! Connect! GovCloud! 2011! +280! !Amazon Elastic! Transcoder! AWS OpsWorks! !Amazon ! CloudHSM! !Amazon ! AppStream! !Amazon ! CloudTrail! !Amazon ! WorkSpaces! Amazon Kinesis! 2013! +159! AWS S!torage! Gateway! !Amazon ! Dynamo DB! !Amazon ! CloudSearch! Amazon SWF! Amazon Glacier! Amazon Redshift! AWS Data ! ! Pipeline! 2012! Since inception AWS has:! ! • Released 927 new services and features ! • Introduced over 35 major new services! • Announced 45 price reductions! ! ! +24! Amazon EBS! Amazon! ! CloudFront! 2008! +270! Amazon Cognito! !Amazon Mobile! Analytics! Amazon Zocalo! 2014! *as of July 31, 2014
- 3. Ninja Tips • Compute and Networking • Storage & Content Delivery • Deployment & Management • Security • Big Data & App Services……maybe!
- 4. • Black Belt Tip – Route53 & Elastic Load Balancing • Cross-Zone Load Balancing • Application Meet Steve Failover via DNS Challenges • Use of AWS is starting to grow • Focus on end user experience • Minimise blast radius in event of issues • Prefers compartmentalization • Hitting AWS account limits
- 5. • Route 53 DNS Failover ELB & Route53 • Cross-Zone Load Balancing
- 6. Meet Steve • Black Belt Tip – Route53 & Elastic Load Balancing • Cross-Zone Load Balancing • Application Failover via DNS • Ninja Tip – VPC Peering • Trust thy neighbour! – VPC peering within an account – VPC peering between accounts Challenges • Use of AWS is starting to grow • Focus on end user experience • Minimise blast radius in event of issues • Prefers compartmentalization • Hitting AWS account limits
- 7. VPC Peering Steve’s Shared Services VPC 10.1.0.0/16 Steve’s Workspaces VPC 192.168.0.0/20 Steve’s Enterprise Apps VPC 172.16.0.0/16 Steve’s Web Apps VPC 10.11.0.0/16 Steve’s Proxy VPC 10.20.10.0/24 Internet Dean’s WAF VPC 10.100.0.0/16 George’s Test/Dev VPC 10.10.0.0/16
- 8. • Black Belt Tip – Storage Gateway File Shares • S3 Backed NAS – Large volume file shares, no upfront cost – On-premise or in the AWS Cloud This is Gwen Challenges • Leverages multiple storage tiers on AWS • EBS for persistent block storage • S3 for backups and serving web & media • Glacier for archiving data • But storage is starting to become costly… even on AWS • Favours the pay for what you use model with S3 rather than what you provision • Requires high performance block storage
- 9. Next Generation Storage Corporate Data center File Servers AWS Cloud Internet or WAN SSL On-Premise AWS Storage Gateway Cache & Upload Buffer Storage Direct Attached or Storage Area Network Disks iSCSI Cached-Volumes Multi-Terabyte AWS Storage Gateway Service “Block” Volumes @ S3 Prices Encrypted & Compressed Volume Snapshots “Block” Volumes @ S3 Prices EC2 File Servers iSCSI Cached-Volumes Multi-Terabyte CIFS/ NFS Clients CIFS/ NFS EC2 Clients Third-Party options too: • Riverbed SteelStore • SoftNAS • Maginatics EC2 AWS Cached Storage Gateway Cache & Upload Buffer EBS PIOPS
- 10. • Black Belt Tip – Storage Gateway File Shares • S3 Backed NAS – Large volume file shares, no upfront cost – On-premise or in the AWS Cloud • Ninja Tip – Instance Storage • Normally ephemeral storage – Using replication = durable storage – EBS PIOPs, General Purpose SSDs and Enhanced Networking This is Gwen Challenges • Leverages multiple storage tiers on AWS • EBS for persistent block storage • S3 for backups and serving web & media • Glacier for archiving data • But storage is starting to become costly… even on AWS • Favours the pay for what you use model with S3 rather than what you provision • Requires high performance block storage
- 11. High Speed* & High Density* Instance storage for durable data Instance Storage with sync to EBS Instance Storage to Instance Storage to EBS General Network Traffic EBS Optimized MDADM RAID 0 array *I2 and C3 Instances: - Multiple 10s & 100’s GB SSD-based instance storage - Enhanced Networking = Higher PPS and lower jitter & latency DRBD protocol A (asynchronous) Up to 50,000 IOPs = 800MBs Data Store EC2 Instance EBS PIOPS or GP2 SSD Backed MDADM RAID 0 or 1+0 array HDD or SSD (100,000s IOPS) Enhanced Networking*
- 12. • Black Belt Tip – AWS = Programmable Resources • AWS Support is an API • Use Resource Tags for management • Centralised logging and notification Say Hi to Felix Challenges • Still very manual deployment and configuration processes of AWS resources • Lots of human interaction • Starting to get resource sprawl – harder to manage • Not everything is supported by CloudFormation
- 13. Everything is an API • Monitoring Your Service Limits – Via Service API • aws iam get-account-summary • aws autoscaling describe-account-limits • aws ec2 describe-account-attributes • aws ses get-send-quota – Via Trusted Advisor • aws support describe-trusted-advisor-check-result --check-id eW7HH0l7J9 --language en • Accessing Support via API – Integrate with your own management/monitoring systems – Automatically log tickets via CloudFormation
- 14. Resource Management with Tags #!/usr/bin/ruby require 'aws-sdk' AWS.regions.sort_by(&:name).each do |region| puts region.name region.ec2.instances.each do |instance| if instance.status == :stopped and instance.tags.to_h.has_key?('DevProjectA') instance.start puts "t#{instance.id} starting" end end end for region in $(aws ec2 describe-regions --query 'Regions[*].RegionName' --output text) do Ruby SDK AWS CLI echo ${region} aws ec2 describe-instances --query 'Reservations[*].Instances[*].[InstanceId]' --filters "Name=instance-state-name,Values=running" "Name=tag-key, Values=Uptime, Name=tag-value, Values=BusinessHoursOnly" --output text --region ${region} | xargs aws ec2 stop-instances -- instance-ids --region ${region} 2> /dev/null done
- 15. Centralised Log Collection • CloudTrail – Get log files of API calls made on your AWS account • CloudWatch Logs – Store and Monitor OS & Application Log Files with Amazon CloudWatch • Service Logs – RDS, ELB, S3, CloudFront, EMR • Detailed Billing Reports – Cost Allocation For Customer Bills All stored in S3
- 16. • Black Belt Tip – AWS = Programmable Resources • AWS Support is an API • Use Resource Tags for management • Centralised logging and notification • Ninja Tip – CloudFormation • Taking it to the next level! – Custom Resources Say Hi to Felix Challenges • Still very manual deployment and configuration processes of AWS resources • Lots of human interaction • Starting to get resource sprawl – harder to manage • Not everything is supported by CloudFormation
- 17. CloudFormation Custom Resources 2 3 Custom Resource Implementation Region SQS Queue AWS CloudFormation Custom Resource Topic Auto scaling Group • Add New Resources – Including AWS resources not currently supported by CFN • Interact with the CloudFormation Workflow • Inject dynamic data into a stack • Extend the capabilities of existing resources • Data management via CloudFormation • It’s really simple if you use aws-cfn-resource-bridge – Install or fork from https://github.com/aws/aws-cfn-resource-bridge Create Parameter1:Value1 Parameter2:Value2 …. Parametern:Valuen Data Export Data Import DynamoDB Datapipeline S3 1 4 5 6 Output Parameter1:Value1 Parameter2:Value2 …. Parametern:Valuen
- 18. CloudFormation Custom Resources 1 2 Custom Resource Implementation 4 Region SQS Queue AWS CloudFormation Custom Resource Topic Auto scaling Group • Add New Resources – Including AWS resources not currently supported by CFN • Interact with the CloudFormation Workflow • Inject dynamic data into a stack • Extend the capabilities of existing resources • Data management via CloudFormation • It’s really simple if you use aws-cfn-resource-bridge – Install or fork from https://github.com/aws/aws-cfn-resource-bridge Delete Parameter1:Value1 Parameter2:Value2 …. Parametern:Valuen Data Import Data Export 3 DynamoDB Datapipeline S3 5 Output Parameter1:Value1 Parameter2:Value2 …. Parametern:Valuen 6
- 19. What’s up Alex? • Black Belt Tip – IAM Roles with EC2 • Don’t leave home without it! Challenges • Admin users with no MFA • Users leaving credentials in software • Users not rotating their credentials • Users not using strong password policies • Finds it hard to keep track of individual IAM identifies for users
- 20. IAM Roles for EC2 Instances AWS Cloud Amazon S3 Amazon DynamoDB Your Application AWS IAM Your Application Your Application Your Application Auto Scaling Your Application Auto Scaling Role: RW access to objects, items and instances • Eliminates use of long-term credentials • Automatic credential rotation • Less coding – AWS SDK does all the work • Easier and more Secure! Amazon EC2
- 21. What’s up Alex? • Black Belt Tip – IAM Roles with EC2 • Don’t leave home without it! • Ninja Tip – Limit number of IAM Users • Use IAM Roles instead – Cross-Account IAM Access – Identity Federation Challenges • Admin users with no MFA • Users leaving credentials in software • Users not rotating their credentials • Users not using strong password policies • Finds it hard to keep track of individual IAM identifies for users
- 22. Cross-account API access dsamuel@amazon.com Acct ID: 111122223333 ec2-role { "Statement": [ { "Action": [ "ec2:StartInstances", "ec2:StopInstances" ], "Effect": "Allow", "Resource": "*" } ] } squigg@amazon.com Acct ID: 123456789012 Authenticate with squigg access keys Optionally also with MFA Get temporary security credentials for ec2-role Call AWS APIs using temporary security credentials of ec2-role { "Statement": [ { IAM user: squigg "Effect": "Allow", "Action": "sts:AssumeRole", "Resource": "arn:aws:iam::111122223333:role/ec2-‐role" } ] } { "Statement": [ { "Effect":"Allow", "Principal":{"AWS":"123456789012"}, "Action":"sts:AssumeRole" } ] } ec2-role trusts IAM users from the AWS account squigg@amazon.com (123456789012) Permissions assigned to squigg granting him permission to assume ec2-role in dsamuel@amazon.com account Permissions assigned to ec2-role STS Amazon EC2
- 23. How to Keep Up to Date • AWS Podcast – http://aws.amazon.com/podcasts/aws-podcast/ • Amazon Web Services Blog – http://aws.amazon.com/blogs/aws • What’s New from AWS – http://aws.amazon.com/new • Social Media – @awscloud, /amazonwebservices, /amazonwebservices • Your Friendly Solution Architect Team – Speak to the team today at the SA booth
- 24. Expand your skills with AWS Certification Exams Validate your proven technical expertise with the AWS platform aws.amazon.com/certification On-Demand Resources Videos & Labs Get hands-on practice working with AWS technologies in a live environment aws.amazon.com/training/ self-paced-labs Instructor-Led Courses Training Classes Expand your technical expertise to design, deploy, and operate scalable, efficient applications on AWS aws.amazon.com/training
- 25. © 2014 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified, or distributed in whole or in part without the express consent of Amazon.com, Inc.