7. how much capacity to deploy?
Actual
Usage
Capacity
Predicted
Usage
Waste
Time
8. benefits of
No Up-Front
Capital Expense cloud Improve Agility &
Time-to-Market
computing
deploy Pay Only for
Low Cost
What You Use
Self-Service Easily Scale Up/Down
16. Push-Button Scaling
scale up or down to the
desired instance class
scale up to an 8-core
server with 64GB of RAM
17. Multi-AZ
Deployment
one-click DR
Provisioned
IOPS
consistent throughput
to 10,000 IOPS
18. Case study
“in the past, we have invested hundreds of days in time-
consuming and costly code-based performance tuning”
“with Amazon RDS Provisioned IOPS we were able to exceed
those performance gains in a single day”
Adam Kierce - Director, Shine Technologies
19. scaling
READS
scale-out with master-slave
one or more read architecture
servers
Use Cases
Reporting and ETL
Discrete read/write transactions (browsers vs buyers)
20. scaling
READS
Tech tips
• Optimize master for OLTP and
read slaves for table scans
• Resize slaves as needed to boost
reporting performance
• Use short-term slaves to save cost
during monthly reporting
21. scaling
READS Amazon ElastiCache
hot items
small, frequently-accessed items are
candidates for read caching Application
Server
• Reduce server-side latency to microseconds
• Eliminate “hot spot” performance barriers
• Offload heavy read activity from database
22. Case study
• 6 months to deploy a real-time, socially relevant magazine
Challenge
• Constantly changing user interests
• Ability to change all hardware and software elastically
Architecture
• Frequently changing system requirements
• Complex queries on user and relevancy data
App needs
• Milliseconds count. So, does uptime
Solution RDS
MySQL + ElastiCache
23. scaling
WRITES
Database
shared-nothing
sharding complex to
architecture plan, implement
and maintain
splits large partitionable
tables across multiple,
smaller database servers
24. scaling application is
WRITES shard-aware
Application
Server
additional challenges shards may require
• key management periodic rebalancing
• query aggregation A-G H-P Q-Z
• cross-shard queries
multiple, small servers are fast,
inexpensive, and easy to manage
25. status check…
Low Administration
High Performance
Databases in the Cloud
27. Dynamod
NoSQL Database Service consistent, low latency
SSD = single-digit ms
Fast & Predictable Performance
Seamless Scalability
from hundreds to hundreds
Zero Administration of thousands of operations/sec
28. new app
Super Bowl promotion new data store
Millions of interactions over a
relatively short period of time
Built the app in 3 days, from
design to production-ready
Fruit Ninja Frenzy facebook game
Moved game data into DynamoDB
Grew from 1 million to 8 million
active monthly users in two weeks
29. Scaling Relational DB’s with DynamoDB
move simple, high-scale workloads
Application
Server
candidates for workload migration
tables with high transaction volumes (esp writes)
primary key-only is preferred (one non-pk index ok)
no dependencies (fk’s, triggers, procedures)
30. Summary
• Scaling database performance requires EFFORT
• Differentiated EFFORT is a good investment
• Scale up with more hardware
• Scale reads and writes across multiple servers
• Amazon RDS makes it easy to set-up, operate and scale
relational databases
• DynamoDB provides an option for existing and new apps
to achieve extreme levels of performance
31. Get started
today
AWS Free Usage Tier
http://aws.amazon.com/free
aws.amazon.com/dynamodb aws.amazon.com/rds aws.amazon.com/elasticache
Welcome to today’s webinar on high performance databases in the cloudWe will start by looking at the type of effort needed to manage and scale a databaseThen we’ll cover the options for scaling databases to achieve high performance, and the challenges associated with scaling a databaseWe’ll take a look at RDS and other Amazon Database Services to understand how customer deploy them for high performanceFinally, we’ll take a look at some examples
Anyone who has interacted with a database understands that it takes varying degrees of effort to manage a databaseIt’s also important to assess the quality of the effort spent managing a databaseIs this differentiated effort, or undifferentiated?
Differentiated effort increases the uniqueness of an applicationIt is the work you put in that enhances the value of the businessThe most obvious examples of application differentiation are compelling new features and an optimal user experienceWhen building high-scale databases, low-level tasks can often reduce the time available to enhance the experience
When we look at the time spent by a database administrator, it’s easy to see what effort help the application differentiate itselfScripting and coding enable the application to provide new features and capabilities – yet it generally it done only 5% of the timeUnderstanding the quality of the effort help us prioritize our optionsLet’s move to specific options for scaling databases
There are two types and three basic approaches to scalingScaling up adds hardware to improve performanceScaling out is applied to reads and writes independently, and the nature of the workload determines which approach is adopted firstRelational databases generally experience more reads than writes, often up to 90% of transaction volume is generated by read activityLet’s examine each of these approaches in more detail…
Scaling the infrastructure means adding new hardware – sometimes lots of itKnowing where to add infrastructure is importantIncreasing server capacity addscpu and memory to the database – this is compute powerIncreasing throughput requires faster networks and more storage volumesThe data files are striped across these volumes to increase parallel I/O to the database, producing higher throughputThis is primarily suited to OLTP workloads, in which reads and writes are randomly distributed across the full data setScaling infrastructure requires time and money for on-premise database servers –and that’s just the first challenge…
The great unknown that exists at the beginning of the process (to request, purchase, and provision the hardware) is how much capacity is needed, and for how long?Traditional infrastructure generally requires that you predictthe amount of computing resources your application will useover a period of several years. If you under-estimate, your applications will not have the horsepower to handle unexpected traffic, potentially resulting in customer dissatisfaction. If you over-estimate, you’re wasting money withsuperfluous resources. The on-demand and elastic nature of the cloud enables the infrastructure to be closely aligned (as it expands and contracts) with the actual demand, thereby increasing overall utilization and reducing cost.
Cloud computing is a better way to run your business. The cloud helps companies of all sizesbecome moreagile. Instead of running your applications yourself you can run them on the cloud where IT infrastructure is offered as a service like a utility. With the cloud, your company saves money: there are no up-front capital expenses as you don’t have to buy hardware for your projects. The massive scale and fast pace of innovation of the cloud drive the costs down for you. In the cloud, you pay only for what you use just like electricity.The cloud can also help your company save time and improve agility – it’s faster to get started: you can build new environments in minutes as you don’t need to wait for new servers to arrive. The elastic nature of the cloud makes it easy to scale up and down as needed. At the end of the day you have more resources left for innovation which allows you to focus on projects that can really impact your businesses like building and deploying more applications. “With the high growth nature of our business, we were looking for a cloud solution to enable us to scale fast. Think twice before buying your next server. Cloud computing is the way forward.” - Sami Lababidi, CTO, Playfish
Extending these benefits of cloud computing is the concept of the managed serviceA managed database service helps the database team focus on doing the tasks that differentiate their appplication
Amazon RDS is a fully managed serviceRDS was designed to be simple to deploy – a database can be provisioned in a few minutes, with very little effortRDS is easy to scale, both up and down, ensuring that the database can adapt quickly to the demands of the applicationRDS is reliable, with automated functions to constantly protect the data under managementAnd RDS is cost-effective compared with traditional databases – starting from less than cents an hourLet’s look at three aspects of RDS… productivity, performance and data protection
In addition to the price advantages of a managed cloud-based database service, RDS provides specific benefits in terms of productivity, performance and protection
By automating standard database management processes, RDS frees the DBA from low-level, undifferentiated effortWith RDS, tasks like deploying or scaling-up a new instance require only a few clicks effortBackground tasks like backups and patching are automatically enabled and constantly monitored with RDSAnd with automation, the potential for human error is reduced, providing a highly reliable database environmentRDS allows administrators to spend more time on important, differentiating activities like performance optimization and building new features
This validation has been very important for RDS, as the challenge of consistently high I/O is a significant one for a virtualized databaseCustomers deploying provisioned IOPS are also saving money from using resources more efficiently
Scaling reads is enabled by provisioning read replicas, then diverting read-only workloads to those instancesPerfect for both ad hoc and structured reporting, or ETL operationsThe read replica isolates the reporting workload away from the OLTP transactionsBy removing this burden, a master database has significantly more resources available, often enabling it to be scaled-down
Read replicas are independent database instances, connected by the replication streamRead replicas can be scaled-up like any other database – which is great for heavy reporting demandsRead replicas can be optimized to accelerate reporting – add indexes, tune memory and storage
There are several key challenges and advantages to database shardingFirstly, the application needs to be aware of the shards that exist and direct workload accordinglyIf a shard experiences excessive workload, it may need to be resharded – which may become a periodic taskThere are other challenges to sharding a database which a good plan will cover. These factors are managed by the application. For example:Keys need to be coordinated across different databases to ensure they are allocated consistentlyDistributed queries are fast, but if the results need to be aggregated, this is a task for the applicationCross-shard queries will be slow and must be avoided as much as possibleWhile challenging, sharding provides several important advantages to performance, cost and manageabilityFor the right type of application (esp those with heavy write workloads), sharding can provide very high scalability
DynamoDB achieves this performance without compromising data durability by staging writes in memoryData is written to disk in multiple data centers before writes are acknowledgedReads can be consistent or eventually consistent
DynamoDB is often deployed as the database underneath a new applicationBut increasingly, customers are moving high-scale workloads into DynamoDB, maintaining a hybrid database environment