Diese Präsentation wurde erfolgreich gemeldet.
Wir verwenden Ihre LinkedIn Profilangaben und Informationen zu Ihren Aktivitäten, um Anzeigen zu personalisieren und Ihnen relevantere Inhalte anzuzeigen. Sie können Ihre Anzeigeneinstellungen jederzeit ändern.

AWS Summit Berlin 2013 - Optimizing your AWS applications and usage to reduce costs

1.801 Aufrufe

Veröffentlicht am

Many customers choose AWS because they need a highly reliable, scalable, and low-cost platform on which to run their applications. Low “pay only for what you use” pricing and frequent price decreases are just the beginning of how AWS can help you optimize your usage and achieve lower costs. In this session, you will learn about a few simple tools for monitoring and managing your AWS resource usage that you can start using right away, as well as some innovative features that can help you operate at lower costs programmatically. Cost allocation reporting, detailed usage reports, billing alerts, EC2 Auto Scaling, Spot and Reserved Instances, and idle resource detection are just a few of the tools and features we will cover.

Veröffentlicht in: Technologie, Business
  • Als Erste(r) kommentieren

AWS Summit Berlin 2013 - Optimizing your AWS applications and usage to reduce costs

  1. 1. Optimizing Your AWS Applications and Usageto Reduce CostsSteffen KrauseTechnical Evangelist@AWS_Aktuellskrause@amazon.de
  2. 2. Agenda• Objective– Options to save money on AWS• Fit the cloud to your product and business model– Use Only What You Need• pay only for what you use– Measure and Manage– Scale Opportunistically
  3. 3. Use Only What You NeedAnd pay only for what you use!
  4. 4. Scale on demandRigid On-Premise ResourcesWasteCustomerDissatisfactionActual demandPredicted DemandCapacityTimeElastic Cloud ResourcesActual demandResources scaled to demandCapacityTimeVS.
  5. 5. Use only what you needAWS cost savings opportunities• Right-size– Select appropriate resources– Scale up and down as appropriate– Turn off unused resources• Payment models– Flexibility vs. predictability– Mixing payment models• Measure and manage– Monitor for saving options
  6. 6. Right-size your EC2 instances• An instance type forevery purpose• Assess your memory& CPU requirements– Fit yourapplication to theresource– Fit the resource toyour application• Only use a largerinstance whenneeded
  7. 7. StandardHigh-CPUHigh-MemoryMicroCluster ComputeCluster GPUHigh I/OHigh StorageHigh Cluster MemoryMost Apps, Low-cost,App Server / WebServerDatabases, DatabasesDatabases…Compute + NetworkThroughputScale-out Compute,Batch ProcessingFor Starters, Lowthroughout, WebsitesParallel ProcessingOLAP, Hadoop,File SystemsNoSQL, Best forRandom IOPSIn-memory Appsand DBs. Best$/RAMEC2 Instance Family use cases
  8. 8. Optimize your storage choice too: S3 & Glacier• S3 and Glacier are both:– Secure– Flexible– Low-cost– Scalable: over 1.3 trillion customer objects– Durable: 99.999999999% (11 “9”s)AmazonGlacier
  9. 9. Choosing between S3 and Glacier• Amazon Simple Storage Service (S3)– Designed to serve static content• high volumes, low latency, frequent access– From 5.5¢/GB/Month: 11 9’s Durability– From 3.7¢/GB/Month: 4 9’s Durability (reducedredundancy)• Amazon Glacier– Designed for long-term cold storage/archiving• infrequent access, long retrieval times (3-5 hrs)– From 1¢ /GB/Month• But retrieving data is slower and more expensive than on S3
  10. 10. S3 and Glacier tips• Optimize access– Reduce payload size– # of accesses (e.g., consolidated logs)• Monitor for unexpected access/growth patterns– Misconfigured log archiving• Set Lifecycle Policies– Object expiration dates– Auto-move S3 files to GlacierIllumina, the leading provider of DNAsequencing instruments, uses Glacier to storelarge blocks of genomic data all over the world
  11. 11. Use only what you needAWS cost savings opportunities• Right-size– Select appropriate resources– Scale up and down as appropriate– Turn off unused resources• Payment models– Flexibility vs. predictability– Mixing payment models• Measure and manage– Monitor for saving options
  12. 12. Fit your payment model to your business modelEC2 pricing plansOn-DemandInstancesReservedInstancesSpotInstancesPay as you go for computingpowerFlat hourly rate, no up-frontcommitmentsPay an up-front fee for acapacity reservation and alower hourly rate (up to 72%savings)1-year or 3-year termsRI Marketplace: sell RIs you nolonger need; buy RIs at adiscountPay what you want for spareEC2 capacity: your instances runif your bid exceeds the SpotpricePotential for large scale at lowcost: When they’re available,take advantage of 1,000s ofSpot Instances at up to 90%savings10:0010:0510:1010:15
  13. 13. Use a spectrum of payment modelsExample:FrontendOn-Demand/Reserved Instances+BackendSpot Instances* e.g., batch video transcoding
  14. 14. Measure and Manage“If you cannot measure it,you cannot improve it.”- Lord Kelvin
  15. 15. AWS Monitoring and Management Services• Detailed cloud monitoring and management– Consolidated Billing (in “Account Activity”)– CloudWatch (in AWS ManagementConsole)– Billing Alerts (in “Account Activity”)– Trusted Advisor (in “Support Center”)– Other APIs: tags, programmatic access, etc.• + Third-party services
  16. 16. Consolidated BillingSingle payer for a group of accounts• One Bill for multiple accounts• Easy Tracking of accountcharges (e.g., download CSV ofcost data)• Group Activities by PayingAccount (e.g., Dev, Stage, Test,Prod)• Volume Discounts can bereached faster with combinedusage• Reserved Instances are sharedacross accounts (including RDSReserved DBs)• AWS Credits are combined tominimize your bill
  17. 17. Consolidated Billing Demo (1/3)• Get an overall summary total for all your users and accounts
  18. 18. Consolidated Billing Demo (2/3)• From your payment account login, view details of each linked account
  19. 19. Consolidated Billing Demo (3/3)• Drill down into detail’s of each account• Download a CSV file for line item details –> Excel
  20. 20. Amazon CloudWatch• Monitoring for AWS cloud resources and applications– EC2, RDS, EBS, ELB, SQS, SNS, DynamoDB, EMR,Auto Scaling, …– Custom metrics from your application (use Put API call)• Insight, Alarms, Notifications• Start easily, auto-scale with your application• Sophisticated Automation– Use CloudWatch metrics with Auto Scaling todynamically scale EC2 instances
  21. 21. Use CloudWatch to monitor & manage resource usage• Monitor resource utilization– Are you using the right instance type?– Have you left instances idle?– Is your instance usage level or bursty?• Manage resource utilization– Move bursty workloads to otherinstances– Rebalance your worker nodes– Scale nodes automatically with AutoScaling
  22. 22. Use CloudWatch to create Billing Alerts• Alert when estimated charges reach threshold• Track an individual developer, or your whole business• Set up your billing alarm and actions
  23. 23. Trusted AdvisorEnterprise Strength Monitoring/Optimization• Monitors and recommendsoptimizations for:– Cost– Security– Fault Tolerance– Performance• Available to customerswith Business andEnterprise-level supporthttp://aws.amazon.com/premiumsupport/trustedadvisor/
  24. 24. Trusted Advisor: Cost Optimization Tips
  25. 25. Trusted Advisor: Performance Tips
  26. 26. Third-party services to optimize your AWS usage
  27. 27. Scale OpportunisticallyOpportunity favors the preparedapplication
  28. 28. Time-to-Result Case 1: Value of result quickly diminishesExample:EngineeringsimulationDelay  Loss ofproductivity,project slips
  29. 29. Time-to-Result Case 2: Result is valuable…until it’s notExample:Weekendregression testsDelay  Minimalimpact until8:00AM Monday
  30. 30. Consider Spot Instances for greater savings and scale• Spot instances in a nutshell– Run when Your Bid ≥ Spot Price– Spot instances = Spare EC2 instances– Might be interrupted at any time• Benefits– Savings: Up to 90% off On-Demand– Scale: Access up to 1,000s of EC2 instances• To use Spot– Decide on a bid price– Launch via Console, API, Auto Scaling– Monitor Bid Statuses via Console/API
  31. 31. What applications work on Spot?• Good Spot applications are:– Delayable: to balance SLA/cost– Scalable: “embarrassingly parallel”– Fault-tolerant: can be terminated without losing all work– Portable across regions, AZs, instance types• Examples:– MapReduce (Hadoop, Amazon EMR)– Scientific Computing (Monte Carlo simulations)– Batch Processing (video transcoding)– Financial Computing (high-frequency trading algorithm backtesting)– and many others…Lucky Oyster crawled 3.4B Web Pages,building a 400M entry index in around14 hours for $100 (>85% savings)!
  32. 32. Use Auto Scaling to dynamically scale your app• Auto Scaling auto-sizes your fleet– based on preset triggers and schedules• Integrates with CloudWatch metrics• Use Auto Scaling to– Improve customer experience, application performance– Maximize CPU/IO/Memory utilization– Optimize other metricsScale with Real-Time Demand
  33. 33. Auto-Scaling Example: Netflix
  34. 34. Follow the Money vs. Follow the Customer• Optimize utilization– Auto Scale on utilization metrics:CPU, memory, requests, connections, …• Optimize price paid– Scale with Spot instances when Spot prices are low– e.g., Run batch processes off-peak (nights, weekends) whenSpot prices are lower
  35. 35. Follow the Money vs. Follow the Customer• Optimize customer experience with Auto Scaling• Example 1: Scale resources to meet customer demand– Video service Auto Scales instances to respond to customer web servicerequests• Example 2: Scale resources to ensure fresh results– A scientific paper search engine Auto Scales on queue depth (# of newdocs to crawl)– 10 instances steady state and up to 5,000+ to ensure minimumthroughput time• Example 3: Scale resources preemptively before large demand– A TV show marketing site scales up before the show and back downafter
  36. 36. Utility HPC / BigData Orchestration Software• Scale clusters from50–50,000 cores• Error-handling,reliability• Data scheduling:internal  cloud• Automated Security,Encryption Framework• Audit, compliance• Reporting, chargeback Automate spot bidding =>
  37. 37. Shared FSScale from 50–50,000+ coresCycleCloud Deploys Secured, Auto-scaled HPC ClustersUserCheck job loadCalculate ideal HPC clusterLegacyInternalHPCLoad-based Spot biddingProperly price the bidsManage Spot Instance lossSpot Instance Execute Nodes(auto-started & auto-stopped)FS /S3HPC ClusterOn-Demand Execute NodesData Scheduling to place dataHPC Orchestration toHandle Spot Instance Bid & Loss
  38. 38. Conclusion (Part I):Fit the cloud to your product and business model• Use Only What You Need (and pay only for what you use!)• Measure and Manage• Scale Opportunistically
  39. 39. An example putting it all together: Saving on BatchProcessinghttp://aws.amazon.com/architecture/3. ScaleOpportunistically:Auto Scale workernodes based onsize of input queue1. Pay Onlyfor What YouUse: Right-size yourcloudresources2. Monitor andManage your systemwith CloudWatch,Billing Alerts,Trusted Advisor
  40. 40. Conclusion (Part II):Use the cloud to create new products & business modelsOn-Premises• Failure isexpensive• Experimentinfrequently• Less InnovationOptimized Cloud• Failure isinexpensive• Experiment earlyand often• More Innovation
  41. 41. THANK YOUSteffen Krause@AWS_Aktuellskrause@amazon.de