by Mahesh Pakala, Solutions Architect, AWS Database Week at the AWS Loft is an opportunity to learn about Amazon’s broad and deep family of managed database services. These services provide easy, scalable, reliable, and cost-effective ways to manage your data in the cloud. We explain the fundamentals and take a technical deep dive into Amazon RDS and Amazon Aurora relational databases, Amazon DynamoDB non-relational databases, Amazon Neptune graph databases, and Amazon ElastiCache managed Redis, along with options for database migration, caching, search and more. You'll will learn how to get started, how to support applications, and how to scale.

1. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved
Pop-up Loft
Amazon Aurora
Mahesh Pakala
pakalam@amazon.com
Solutions Architect

2. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved
What is Amazon Aurora ……
D a t a b a s e r e im a g in e d f o r t h e c l o u d
R Speed and availability of high-end commercial databases
R Simplicity and cost-effectiveness of open source databases
R Drop-in compatibility with MySQL and PostgreSQL
R Simple pay as you go pricing
Delivered as a managed service

3. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved
Re-imagining relational database
Automate administrative tasks – fully managed service
1
2
3
Scale-out and distributed design
Service-oriented architecture leveraging AWS services

4. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved
Scale-out, distributed architecture
Master Replica Replica Replica
Availability
Zone 1
Shared storage volume
Availability
Zone 2
Availability
Zone 3
Storage nodes with SSDs
§ Purpose-built log-structured distributed
storage system designed for databases
§ Storage volume is striped across
hundreds of storage nodes distributed
over 3 different availability zones
§ Six copies of data, two copies in each
availability zone to protect against
AZ+1 failures
§ Plan to apply same principles to other
layers of the stack
SQL
Transactions
Caching
SQL
Transactions
Caching
SQL
Transactions
Caching

5. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved
Leveraging cloud ecosystem
Lambda
S3
IAM
CloudWatch
Invoke Lambda events from stored procedures/triggers.
Load data from S3, store snapshots and backups in S3.
Use IAM roles to manage database access control.
Upload systems metrics and audit logs to CloudWatch.

6. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved
Automate administrative tasks
• Schema design
• Query construction
• Query optimization
Automatic fail-over
Backup & recovery
Isolation & security
Industry compliance
Push-button scaling
Automated patching
Advanced monitoring
Routine maintenance
Takes care of your time-consuming database management tasks, freeing
you to focus on your applications and business
You
AWS

7. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved
Aurora is used by ¾ of the top
100 AWS customers
Aurora customer adoption
Fastest growing service in AWS history

8. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved
Who is moving to Aurora and why?
Customers using
commercial engines
Customers using MySQL
and PostgreSQL engines
§ Higher performance – up to 5x
§ Better availability and durability
§ Reduces cost – up to 60%
§ Easy migration; no application change
§ One tenth of the cost; no licenses
§ Integration with cloud ecosystem
§ Comparable performance and availability
§ Migration tooling and services

9. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved
Amazon Aurora is fast …
5x more throughput than MySQL
3x more throughput than PostgreSQL

10. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved
WRITE PERFORMANCE READ PERFORMANCE
MySQL SysBench results
R3.8XL: 32 cores / 244 GB RAM
5X faster than RDS MySQL 5.6 & 5.7
Five times higher throughput than stock MySQL
based on industry standard benchmarks.
0
25,000
50,000
75,000
100,000
125,000
150,000
0
100,000
200,000
300,000
400,000
500,000
600,000
700,000
Aurora MySQL 5.6 MySQL 5.7

11. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved
Do fewer I/Os
Minimize network packets
Cache prior results
Offload the database engine
DO LESS WORK
Process asynchronously
Reduce latency path
Use lock-free data structures
Batch operations together
BE MORE EFFICIENT
How did we achieve this?
DATABASES ARE ALL ABOUT I/O
NETWORK-ATTACHED STORAGE IS ALL ABOUT PACKETS/SECOND
HIGH-THROUGHPUT PROCESSING IS ALL ABOUT CONTEXT SWITCHES

12. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved
BINLOG DATA DOUBLE-WRITELOG FRM FILES
TYPE OF WRITE
MYSQL WITH REPLICA
EBS mirrorEBS mirror
AZ 1 AZ 2
Amazon S3
EBS
Amazon Elastic
Block Store (EBS)
Primary
Instance
Replica
Instance
1
2
3
4
5
AZ 1 AZ 3
Primary
Instance
Amazon S3
AZ 2
Replica
Instance
ASYNC
4/6 QUORUM
DISTRIBUTED
WRITES
Replica
Instance
AMAZON AURORA
780K transactions
7,388K I/Os per million txns (excludes mirroring, standby)
Average 7.4 I/Os per transaction
MySQL I/O profile for 30 min Sysbench run
27,378K transactions 35X MORE
0.95 I/Os per transaction (6X amplification) 7.7X LESS
Aurora IO profile for 30 min Sysbench run
Aurora I/O profile

13. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved
Scan
Delete
Aurora lock management
Scan
Delete
Insert
Scan Scan
Insert
Delete
Scan
Insert
Insert
MySQL lock manager Aurora lock manager
§ Same locking semantics as MySQL
§ Concurrent access to lock chains
Needed to support many concurrent sessions, high update throughput
§ Multiple scanners in individual lock chains
§ Lock-free deadlock detection

14. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved
New performance enhancements
► Smart selector
► Logical read ahead
► Read views
► Hash joins *coming soon*
► Parallel query *coming soon*
Read performance
Write performance
Meta-data access
► NUMA aware scheduler
► Latch-free lock manager
► Instant schema update
► B-Tree concurrency
► Catalog concurrency
► Faster index build

15. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved
What about availability?
“Performance only matters if your database is up”

16. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved
• Six copies across three availability zones
• 4 out 6 write quorum; 3 out of 6 read quorum
• Peer-to-peer replication for repairs
• Volume striped across hundreds of storage nodes
SQL
Transaction
AZ 1 AZ 2 AZ 3
Caching
SQL
Transaction
AZ 1 AZ 2 AZ 3
Caching
Read and write availabilityRead availability
6-way replicated storage
S u r v iv e s c a t a s t r o p h ic f a il u r e s

17. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Amazon Aurora Recovers Up to 97%Faster
3 GiB Redo
Recovered in 19 seconds
10 GiB Redo
Recovered in 50 seconds
30 GiB Redo
Recovered in 123 seconds
0
20
40
60
80
100
120
140
160
0 20,000 40,000 60,000 80,000 100,000 120,000 140,000
RecoveryTimeinSeconds(lessisbetter)
Writes / Second (more is better)
RECOVERY TIME FROM CRASH UNDER LOAD
Bubble size represents redo log, which must be recovered
As PostgreSQL
throughput goes up, so
does log size and crash
recovery time
Amazon Aurora has no redo.
Recovered in 3 seconds while
maintaining significantly greater
throughput.

18. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved
Up to 15 promotable read replicas
Master
Read
Replica
Read
Replica
Read
Replica
Shared distributed storage volume
Reader end-point
► Up to 15 promotable read replicas across multiple availability zones
► Re-do log based replication leads to low replica lag – typically < 10ms
► Reader end-point with load balancing and auto-scaling

19. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved
Cross-region read replicas
Fa st e r d isa st e r r e c o v e r y a n d e n h a n c e d d a t a l o c a l it y
• Promote read-replica to a
master for faster recovery in
the event of disaster
• Bring data close to your
customer’s applications in
different regions
• Promote to a master for easy
migration

20. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved
Availability is about more than HW failures
• You also incur availability disruptions when you
1. Patch your database software – Zero Down Time Patch
2. Perform large scale database reorganizations – Fast Cloning
3. DBA errors requiring database restores – Online Point-in-time Restore

21. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved
Database backtrack
• Backtrack brings the database to a point in time without requiring restore from backups
• Backtracking from an unintentional DML or DDL operation
• Backtrack is not destructive. You can backtrack multiple times to find the right point in time
t0 t1 t2
t0 t1
t2
t3 t4
t3
t4
Rewind to t1
Rewind to t3
Invisible Invisible

22. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved
Performance Insights
• Dashboard showing
• Load on Database
– Easy
– Powerful
• Identifies source of
bottlenecks
– Top SQL
• Adjustable time frame
– Hour, day, week , month
– Up to 35 days of data
Max CPU

23. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved
Fast database cloning
• Create a copy of a database without
duplicate storage costs
• Creation of a clone is nearly instantaneous –
we don’t copy data
• Data copy happens only on write – when
original and cloned volume data differ
Typical use cases:
• Clone a production DB to run tests
• Reorganize a database
• Save a point in time snapshot for analysis
without impacting production system.
Production database
Clone Clone
Clone
Dev/test
applications
Benchmarks
Production
applications
Production
applications

24. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved
DevTest environments
Master Replica
Shared storage volume
Clone
Shared storage volume
Master Replica
Production Account Development Account
Cross-account snapshot
Test on prod data
• Create a clone of multi-terabyte DB clusters in minutes. Use for diagnosis and pre-prod
testing
• Restore a production DB cluster in a development accounts for development, staging, or
partnering with other teams

25. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved
Hybrid operations and migration
Amazon Aurora
Corporate Data Center
Replication
Analytics
Prod apps
Provider of an online marketplace for short-term rentals replicates on-premise
production data into Aurora for read-only analytics workloads.
§ Seamless replicate to Aurora: Full compatibility with MySQL and PostgreSQL
§ Data replicated in minutes – run analytics on near real-time data
§ Provides a clean waypoint towards full write/read migration
MySQL

26. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved
Higher performance, lower Cost
§ Fewer instances needed
§ Smaller instances can be used
Safe.com lowered their bill by 40% by switching from sharded
MySQL to a single Aurora instance.
Double Down Interactive (gaming) lowered their bill by 67%
while also achieving better latencies (most queries ran faster)
and lower CPU utilization.
§ No need to pre-provision storage
§ No additional storage for read replicas

27. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved
Higher Performance, Lower Cost
Our application usage had grown exponentially over the last year. We were looking for
horizontal scaling of our database to address the increased load. Amazon Aurora’s
relatively low replication lag has helped us handle current load and positions us well for
future growth.
“
7x 10x
Database connections CPU utilization
2x
Response time
“

28. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved
Pop-up Loft
aws.amazon.com/activate
Everything and Anything Startups
Need to Get Started on AWS