AWS Webcast - Journey through the Cloud - Cost Optimization
•
1 gefällt mir•2,160 views
From turning systems off at night to implementing bidding strategies on the spot market, there are many ways in which you can manage costs in AWS. This presentation outlines strategies to help you save money in the AWS Cloud.
Empfohlen
Empfohlen
Weitere ähnliche Inhalte
Was ist angesagt?
Was ist angesagt? (20)
Andere mochten auch
Andere mochten auch (10)
Ähnlich wie AWS Webcast - Journey through the Cloud - Cost Optimization
Ähnlich wie AWS Webcast - Journey through the Cloud - Cost Optimization (20)
Mehr von Amazon Web Services
Mehr von Amazon Web Services (20)
Kürzlich hochgeladen
Kürzlich hochgeladen (20)
AWS Webcast - Journey through the Cloud - Cost Optimization
- 1. Journey through the Cloud: Cost optimization Jan Borch – Solutions Architect
- 2. Common use cases & stepping stones into the AWS cloud Learning from customer journeys Best practices to bootstrap your projects Journey through the cloud
- 3. A key step in the cloud journey Realize cost aware architectures Use elasticity to real and measurable benefit Do more, use less Cost Optimization
- 4. Fundamentals of AWS cost optimization Cost optimization in 5 steps Where to go next Agenda
- 5. Fundamentals of cost optimization
- 7. Why optimize? Utility Compute and Storage are a utility so ‘turning off’ should be natural
- 8. Why optimize? Utility Compute and Storage are a utility so ‘turning off’ should be natural Efficiency Efficiency allows more to be done within a given budget
- 9. Why optimize? Utility Compute and Storage are a utility so ‘turning off’ should be natural Efficiency Efficiency allows more to be done within a given budget Architecture Cost awareness drives adoption of 21st century architectures
- 10. Turn off the lights When you stop EC2 resources you stop paying for them
- 11. Be elastic Support workloads with the right amount of horsepower to get the job done
- 12. Continually optimize Drive recurring and improving savings through cost aware architectures
- 13. Elastic capacity Instance types Reserved instances Spot instances Complementary services 5 Steps for cost optimization
- 14. Elastic capacity Instance types Reserved instances Spot instances Complementary services 5 Steps for cost optimization
- 15. 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 ServerLoad Hour of day
- 16. 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 ServerLoad Hour of day Capacity of 1 Server
- 17. 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 ServerLoad Hour of day Capacity of 1 Server Traditional capacity required
- 18. 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 ServerLoad Hour of day Capacity of 1 Server Traditional capacity required 1 Server for 8 hours
- 19. 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 ServerLoad Hour of day Capacity of 1 Server Traditional capacity required 1 Server for 8 hours 1 Server for 8 hours
- 20. 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 ServerLoad Hour of day Capacity of 1 Server Traditional capacity required 1 Server for 8 hours 1 Server for 8 hours 1 Server for 8 hours
- 21. 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 ServerLoad Hour of day Capacity of 1 Server Traditional capacity required 1 Server for 8 hours 1 Server for 8 hours 1 Server for 8 hours 1 Server for 8 hours
- 22. 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 ServerLoad Hour of day Capacity of 1 Server Traditional capacity required 1/3rd Saving
- 28. Time: +00h <10 cores Elastic Capacity
- 29. Time: +24h >1500 cores Elastic Capacity
- 30. Time: +72h <10 cores Elastic Capacity
- 31. Time: +120h >600 cores Elastic Capacity
- 32. Manually Send an API call or use CLI to launch/terminate instances – Only need to specify capacity change (+/-) By Schedule Scale up/down based on date and time By Policy Scale in response to changing conditions, based on user configured real-time monitoring and alerts Auto-Rebalance Instances are automatically launched/terminated to ensure the application is balanced across multiple Azs Auto-scaling policies
- 33. Auto-scaling policies Scaling base on Policy Scale up and down base on metrics Scaling Up policy - Double the group size if avg cpu > 80% Scaling Down policy - Decrement by 10% if avg cpu < 30% Scaling by Schedule Scheduled Actions to meet known demand Scheduled up to 31 days into the future Recurring scheduled scaling activities
- 34. 0 1 2 3 4 5 6 0 1 2 3 4 5 6 7 8 9 101112131415161718192021222324252627282930 InstanceCount Day of Month
- 35. 0 1 2 3 4 5 6 0 1 2 3 4 5 6 7 8 9 101112131415161718192021222324252627282930 InstanceCount Day of Month Monthly predictable peak processing
- 36. 0 1 2 3 4 5 6 0 1 2 3 4 5 6 7 8 9 101112131415161718192021222324252627282930 InstanceCount Day of Month Traditional capacity required
- 37. 0 1 2 3 4 5 6 0 1 2 3 4 5 6 7 8 9 101112131415161718192021222324252627282930 InstanceCount Day of Month Elastic Capacity Traditional capacity required
- 38. 0 1 2 3 4 5 6 0 1 2 3 4 5 6 7 8 9 101112131415161718192021222324252627282930 InstanceCount Day of Month 75 % Savings Traditional capacity required Elastic Capacity
- 39. Elastic capacity Instance types Reserved instances Spot instances Complementary services 5 Steps for cost optimization
- 40. Elastic capacity Instance types Reserved instances Spot instances Complementary services 5 Steps for cost optimization
- 41. Instance types
- 42. Start Choose instance that meets your basic requirements best Match memory & virtual cores Instance types
- 43. Start Choose instance that meets your basic requirements best Match memory & virtual cores Tune Change instance size up or down based upon monitoring Use trusted advisor to assess Instance types
- 44. Start Choose instance that meets your basic requirements best Match memory & virtual cores Tune Change instance size up or down based upon monitoring Use trusted advisor to assess Spread Run instances across multiple availability zones Smaller sizes equals greater granularity Instance types
- 45. Instance Amazon CloudWatch Alarm Free Memory Free CPU Free HDD … Custom Metrics … At 1-min intervals PUT 2 weeks Know your usage
- 46. Choose your metric optimize for the metric
- 47. Choose your metric optimize for the metric Cost per unit of work per instance(size) Workload A Optimal on 4x m1.xlarge Workload B Optimal on 10x m1.medium Workload C Optimal on 2x m3.xxlarge
- 48. Choose your metric optimize for the metric Cost per unit of work per instance(size) 100 concurrent jobs on 10 x m1.large @ $0.26 / hr = $ 0.026 / job 300 concurrent jobs on 10 x m3.xlarge @ $0.58 / hr = $ 0.019 / job vs
- 49. Choose your metric optimize for the metric Think workload density Don’t focus on instance hourly rate per se
- 51. Division B admin@divisionB.com User2 Dev2 Admin2 IAM Master Account aws.invoices@mycompany.com consolidated billing information
- 52. Division B admin@divisionB.com User2 Dev2 Admin2 IAM Tags: Own=Div Proj=P Tags: Own=Div Proj=Q Tags: Own=Div Proj=R Master Account aws.invoices@mycompany.com consolidated billing information Tags: (key- value) e.g Own=Div Proj=R
- 53. Operating Co. A admin@opcoa.com User1 Dev1 Admin1 IAM Tags: Own=OpCo Proj=A Tags: Own=OpCo Proj=B Tags: Own=OpCo Proj=C Division B admin@divisionB.com User2 Dev2 Admin2 IAM Tags: Own=Div Proj=P Tags: Own=Div Proj=Q Tags: Own=Div Proj=R Business Unit C admin@busUnitC.com User3 Dev3 Admin3 IAM Tags: Own=BusC Proj=X Tags: Own=BusC Proj=Y Tags: Own=BusC Proj=Z Master Account aws.invoices@mycompany.com consolidated billing information
- 54. Operating Co. A admin@opcoa.com User1 Dev1 Admin1 IAM Tags: Own=OpCo Proj=A Tags: Own=OpCo Proj=B Tags: Own=OpCo Proj=C Division B admin@divisionB.com User2 Dev2 Admin2 IAM Tags: Own=Div Proj=P Tags: Own=Div Proj=Q Tags: Own=Div Proj=R Business Unit C admin@busUnitC.com User3 Dev3 Admin3 IAM Tags: Own=BusC Proj=X Tags: Own=BusC Proj=Y Tags: Own=BusC Proj=Z Master Account aws.invoices@mycompany.com consolidated billing information
- 55. Operating Co. A admin@opcoa.com User1 Dev1 Admin1 IAM Tags: Own=OpCo Proj=A Tags: Own=OpCo Proj=B Tags: Own=OpCo Proj=C Division B admin@divisionB.com User2 Dev2 Admin2 IAM Tags: Own=Div Proj=P Tags: Own=Div Proj=Q Tags: Own=Div Proj=R Business Unit C admin@busUnitC.com User3 Dev3 Admin3 IAM Tags: Own=BusC Proj=X Tags: Own=BusC Proj=Y Tags: Own=BusC Proj=Z Master Account aws.invoices@mycompany.com consolidated billing information Programmatic billing access S3 CSV
- 56. Operating Co. A admin@opcoa.com User1 Dev1 Admin1 IAM Tags: Own=OpCo Proj=A Tags: Own=OpCo Proj=B Tags: Own=OpCo Proj=C Division B admin@divisionB.com User2 Dev2 Admin2 IAM Tags: Own=Div Proj=P Tags: Own=Div Proj=Q Tags: Own=Div Proj=R Business Unit C admin@busUnitC.com User3 Dev3 Admin3 IAM Tags: Own=BusC Proj=X Tags: Own=BusC Proj=Y Tags: Own=BusC Proj=Z Master Account aws.invoices@mycompany.com consolidated billing information Programmatic billing access S3 CSV
- 57. Business Basic Developer Enterprise Offering 24x7x365 ✓ Forum Access ✓ Documentation ✓ Access to support Phone, Chat, Email Named Contacts 5 Fastest Response Time 1 Hour Architecture Support Use Case Guidance Best Practice ✓ Diagnostics Tools ✓ Direct Routing ✓ 3rd Party Software ✓ Trusted Advisor ✓
- 60. Elastic capacity Instance types Reserved instances Spot instances Complementary services 5 Steps for cost optimization
- 61. Elastic capacity Instance types Reserved instances Spot instances Complementary services 5 Steps for cost optimization
- 62. Unix/Linux instances start at $0.02/hour Pay as you go for compute power Low cost and flexibility Pay only for what you use, no up-front commitments or long-term contracts Use Cases: Applications with short term, spiky, or unpredictable workloads; Application development or testing On-demand instances Reserved instances
- 63. Unix/Linux instances start at $0.02/hour Pay as you go for compute power Low cost and flexibility Pay only for what you use, no up-front commitments or long-term contracts Use Cases: Applications with short term, spiky, or unpredictable workloads; Application development or testing On-demand instances 1- or 3-year terms Pay low up-front fee, receive significant hourly discount Low Cost / Predictability Helps ensure compute capacity is available when needed Use Cases: Applications with steady state or predictable usage Applications that require reserved capacity, including disaster recovery Reserved instances Reserved instances
- 64. Unix/Linux instances start at $0.02/hour Pay as you go for compute power Low cost and flexibility Pay only for what you use, no up-front commitments or long-term contracts Use Cases: Applications with short term, spiky, or unpredictable workloads; Application development or testing On-demand instances 1- or 3-year terms Pay low up-front fee, receive significant hourly discount Low Cost / Predictability Helps ensure compute capacity is available when needed Use Cases: Applications with steady state or predictable usage Applications that require reserved capacity, including disaster recovery Reserved instances Reserved instances > 80% utilization Lower costs up to 58% Use Cases: Databases, Large Scale HPC, Always-on infrastructure, Baseline Heavy utilization RI
- 65. Unix/Linux instances start at $0.02/hour Pay as you go for compute power Low cost and flexibility Pay only for what you use, no up-front commitments or long-term contracts Use Cases: Applications with short term, spiky, or unpredictable workloads; Application development or testing On-demand instances 1- or 3-year terms Pay low up-front fee, receive significant hourly discount Low Cost / Predictability Helps ensure compute capacity is available when needed Use Cases: Applications with steady state or predictable usage Applications that require reserved capacity, including disaster recovery Reserved instances Reserved instances > 80% utilization Lower costs up to 58% Use Cases: Databases, Large Scale HPC, Always-on infrastructure, Baseline Heavy utilization RI 41-79% utilization Lower costs up to 49% Use Cases: Web applications, many heavy processing tasks, running much of the time Medium utilization RI
- 66. Unix/Linux instances start at $0.02/hour Pay as you go for compute power Low cost and flexibility Pay only for what you use, no up-front commitments or long-term contracts Use Cases: Applications with short term, spiky, or unpredictable workloads; Application development or testing On-demand instances 1- or 3-year terms Pay low up-front fee, receive significant hourly discount Low Cost / Predictability Helps ensure compute capacity is available when needed Use Cases: Applications with steady state or predictable usage Applications that require reserved capacity, including disaster recovery Reserved instances Reserved instances > 80% utilization Lower costs up to 58% Use Cases: Databases, Large Scale HPC, Always-on infrastructure, Baseline Heavy utilization RI 41-79% utilization Lower costs up to 49% Use Cases: Web applications, many heavy processing tasks, running much of the time Medium utilization RI 15-40% utilization Lower costs up to 34% Use Cases: Disaster Recovery, Weekly / Monthly reporting, Elastic Map Reduce Light utilization RI
- 67. Best RI for Utilisation $- $2,000 $4,000 $6,000 $8,000 $10,000 $12,000 $14,000 $16,000 $18,000 Heavy Medium Light O-Demand
- 68. Best RI for Utilisation $- $2,000 $4,000 $6,000 $8,000 $10,000 $12,000 $14,000 $16,000 $18,000 Heavy Medium Light O-Demand
- 69. 0 2 4 6 8 10 12 14 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 On Demand Light Utilization RI Medium Utilization RI Heavy utilization RI Optimizing costs with RIs
- 70. Elastic capacity Instance types Reserved instances Spot instances Complementary services 5 Steps for cost optimization
- 71. Elastic capacity Instance types Reserved instances Spot instances Complementary services 5 Steps for cost optimization
- 72. Unix/Linux instances start at $0.02/hour Pay as you go for compute power Low cost and flexibility Pay only for what you use, no up-front commitments or long-term contracts Use Cases: Applications with short term, spiky, or unpredictable workloads; Application development or testing On-demand instances 1- or 3-year terms Pay low up-front fee, receive significant hourly discount Low Cost / Predictability Helps ensure compute capacity is available when needed Use Cases: Applications with steady state or predictable usage Applications that require reserved capacity, including disaster recovery Reserved instances Spot instances
- 73. Unix/Linux instances start at $0.02/hour Pay as you go for compute power Low cost and flexibility Pay only for what you use, no up-front commitments or long-term contracts Use Cases: Applications with short term, spiky, or unpredictable workloads; Application development or testing On-demand instances 1- or 3-year terms Pay low up-front fee, receive significant hourly discount Low Cost / Predictability Helps ensure compute capacity is available when needed Use Cases: Applications with steady state or predictable usage Applications that require reserved capacity, including disaster recovery Reserved instances Bid on unused EC2 capacity Spot Price based on supply/demand, determined automatically Cost / Large Scale, dynamic workload handling Use Cases: Applications with flexible start and end times Applications only feasible at very low compute prices Spot instances Spot instances
- 74. 100% Achieving economies of scale Time
- 75. Reserved capacity 100% Achieving economies of scale Time
- 77. On Reserved capacity 100% On-demand Time Achieving economies of scale Spot
- 80. If your bid > spot price You get an instance
- 81. If your bid < spot price Your instance is terminated
- 82. Architecting for spot instances Decouple components Separate interactive & backend processing Use frameworks such as Elastic MapReduce Design for interruption Use SQS, SWF Place data in a durable store such as S3, SimpleDB or DynamoDB Save progress regularly
- 83. Job Flow 14 Hours Duration: Scenario #1 EMR with spot instances #1: Cost without Spot 4 instances *14 hrs * $0.50 = $28
- 84. Job Flow 14 Hours Duration: Scenario #1 Duration: Job Flow 7 Hours Scenario #2 EMR with spot instances #1: Cost without Spot 4 instances *14 hrs * $0.50 = $28
- 85. Job Flow 14 Hours Duration: Scenario #1 Duration: Job Flow 7 Hours Scenario #2 EMR with spot instances #1: Cost without Spot 4 instances *14 hrs * $0.50 = $28 #2: Cost with Spot 4 instances *7 hrs * $0.50 = $14 + 5 instances * 7 hrs * $0.25 = $8.75 Total = $22.75
- 86. Job Flow 14 Hours Duration: Scenario #1 Duration: Job Flow 7 Hours Scenario #2 EMR with spot instances #1: Cost without Spot 4 instances *14 hrs * $0.50 = $28 #2: Cost with Spot 4 instances *7 hrs * $0.50 = $14 + 5 instances * 7 hrs * $0.25 = $8.75 Total = $22.75 Time Savings: 50% Cost Savings: ~22%
- 87. Spot customers
- 88. Elastic capacity Instance types Reserved instances Spot instances Complementary services 5 Steps for cost optimization
- 89. Elastic capacity Instance types Reserved instances Spot instances Complementary services 5 Steps for cost optimization
- 90. $0.028 per hour Web Servers Availability Zone Elastic Load Balancer DNS
- 91. Web Servers $0.065 per hour (small instance) Availability Zone $0.028 per hour Web Servers Availability Zone EC2 instance + software LB Elastic Load Balancer DNS DNS VS
- 92. SQS queue Consumers Producer $0.005 per 10,000 Requests ($0.0000005 per Request)
- 93. Producer SQS queue Consumers Consumers Producer EC2 instance + software queue $0.005 per 10,000 Requests ($0.0000005 per Request) $0.065 per hour (small instance) VS
- 94. Software vs services Software on EC2 Pros: Use custom features Cons: Requires an instance SPOF Limited to one AZ DIY administration AWS Services ELB, SNS, SQS, SES, SWF, DynamoDB etc Pros: Pay as you go Scalability Availability High performance
- 95. Summary
- 96. Elastic capacity Instance types Reserved instances Spot instances Complementary services 5 Steps for cost optimization
- 97. Where to go next
- 101. aws.amazon.com