More Related Content Similar to DAT317_Migrating Databases and Data Warehouses to the Cloud (20) More from Amazon Web Services (20) DAT317_Migrating Databases and Data Warehouses to the Cloud1. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
AWS re:INVENT
Migrating Databases and Data
Warehouses to the Cloud
G e t t i n g S t a r t e d w i t h A W S D M S a n d A W S S C T
J o h n W i n f o r d — S e n i o r T e c h n i c a l P r o d u c t M a n a g e r
N o v e m b e r 2 8 , 2 0 1 7
2. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Agenda
• How does the cloud help?
• How do I get there?
• When should I use it?
• How does it work?
• Show me!
• What have others done?
• Resources and details
3. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Cloud benefits
4. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
• Multi-engine support: Amazob Aurora, MySQL,
MariaDB, PostgreSQL, Oracle, SQL Server
• Automated provisioning, patching, scaling,
backup/restore, failover
• High availability with RDS Multi-AZ
– 99.95% SLA for Multi-AZ deployments
Amazon Relational Database Service
(Amazon RDS)
5. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
• Lower TCO because we manage the muck
• Get more leverage from your teams
• Focus on the things that differentiate you
• Built-in high availability and cross-region
replication across multiple data centers
• Available on all engines, including
base/standard editions, not just for enterprise
editions
• Now even a small startup can leverage multiple
data centers to design highly available apps with
over 99.95% availability
The cloud makes things cheaper, easier, better
6. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
The journey
7. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
How can I get to the cloud?
How will my on-premises data migrate to the cloud?
How can I make it transparent to my users?
Afterwards, how will on-premises and cloud data interact?
How can I integrate my data assets within AWS?
Can I get help moving off of commercial databases?
8. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Migration used to be cost + complexity + time
Commercial data migration and replication software
Complex to set up and manage
Application downtime
Database-engine-specific application code
9. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
What are DMS and SCT?
AWS Database Migration Service (DMS) easily and securely
migrate and/or replicate your databases and data warehouses
to AWS
AWS Schema Conversion Tool (SCT) convert your commercial
database and data warehouse schemas to open-source engines or
AWS-native services, such as Amazon Aurora and Amazon Redshift
We have migrated over 45,000 unique databases. And counting…
10. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Migration options
If you’re not switching engines and can take downtime:
- SQL Server: bak file import
- MySQL: read replicas
- Oracle SQL Developer, Data Pump, Export/Import
- PostgreSQL: pg_dump
- SAP ASE: bcp
11. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
When should I use it?
12. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
When to use DMS and SCT?
Modernize Migrate Replicate
Modernize your database tier—
• Commercial to open-source
• Commercial to Amazon Aurora
Modernize your Data Warehouse—
• Commercial to Amazon Redshift
• Migrate business-critical applications
• Migrate from Classic to Amazon
Virtual Private Cloud (Amazon VPC)
• Migrate data warehouse to Redshift
• Upgrade to a minor version
• Consolidate shards into Aurora
• Create cross-regions Read Replicas
• Run your analytics in the cloud
• Keep your dev/test and production
environment sync
13. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
When to use DMS and SCT? Modernize
Modernize your database tier
• Commercial to open-source
• Commercial to Amazon Aurora
Modernize your warehouse
• Commercial to Amazon Redshift
Amazon
Aurora
Amazon
Redshift
Amazon S3
Amazon
DynamoDB
14. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Why use DMS and SCT?
Secure
Cost Effective
Remove Barriers
to Entry
Allow DB
Freedom
Keep a Leg in
the Cloud
Easy to Use, but
Sophisticated…
Near-Zero
Downtime
15. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
How does it work?
16. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Database migration process
Source
Oracle Database on premises
in EC2 or RDS
Source
Oracle Database on premises
in EC2 or RDS
AWS Schema Conversion Tool
Target
Amazon Aurora PostgreSQL
Target
Amazon Aurora PostgreSQL
Step 1
Step 2
AWS Database Migration Service
17. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
AWS Schema Conversion Tool (AWS SCT)
Features
Converting database schema
Converting data warehouse schema
Converting application SQL
Code browser that highlights places where
manual edits are required
Secure connections to your databases with SSL
The AWS Schema Conversion Tool helps automate many
database schema and code conversion tasks when
migrating from source to target database engines
18. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Database migration assessment
Connect SCT to source
and target databases
Run assessment report
Read executive
summary
Follow detailed
instructions
19. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Components of the SCT console
1. Source Schema
2. Action items
3. Target Schema
4. Schema element details
5. Edit window
20. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Customer
premises
Application users
AWS
Internet
VPN
Keep your apps running during the migration
Start a replication instance
Connect to source and target
databases
Select tables, schemas, or
databases
Let AWS Database Migration
Service (AWS DMS) create
tables, load data, and keep
them in sync
Switch applications over to
the target at your convenience
AWS
DMS
21. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Load is table by table
Replication instance
Source Target
22. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Change data capture (CDC) and apply
Replication instance
Target
Transactions
Change apply after bulk load
t1
t2
Source
t1 t2
Update
23. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Source
Target
What else can I do?
Source
Source
Replication instance
24. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
What else can I do? Target
Source Target
Target
Replication instance
25. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Source
Take it all—or not
Target
Replication instance
26. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Homogenous or heterogeneous
SQL Server MySQL
Oracle Oracle
Aurora
Amazon S3
27. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Demo
28. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
What have others done?
29. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Our customers…
30. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
The fine print
31. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Other database migration use cases
Migration of business-critical applications
Migration from Classic to VPC
Cheap Read Replicas for Oracle
Read Replicas for other engines
Cross-region Read Replicas for Oracle and SQL Server
Analytics in the cloud
Dev/test and production environment sync
Ongoing replication for BI
Minor version upgrade
32. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
On-premises and Amazon EC2 instance databases:
• Oracle Database 10g–12c
• Microsoft SQL Server 2005–2016
• MySQL 5.5–5.7
• MariaDB (MySQL-compatible data source)
• PostgreSQL 9.4–9.5
• SAP ASE 15.7+
• Azure SQL Database (No CDC)
• MongoDB 2.6.x and 3.x+
• S3
RDS instance databases:
• Oracle Database 11g–12c
• Microsoft SQL Server 2008R2–2016. (No CDC)
• MySQL versions 5.5–5.7
• MariaDB (MySQL-compatible data source)
• PostgreSQL 9.4–9.5
• Amazon Aurora (MySQL and PostgreSQL compatible data sources)
Supported sources for AWS DMS
33. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
On-premises and EC2 instance databases:
• Oracle Database 10g–12c
• Microsoft SQL Server 2005–2016
• MySQL 5.5–5.7
• MariaDB (MySQL-compatible data source)
• PostgreSQL 9.3–9.5
• SAP ASE 15.7+
• S3
RDS instance databases:
• Oracle Database 11g–12c
• Microsoft SQL Server 2008 R2 - 2014
• MySQL 5.5–5.7
• MariaDB (MySQL-compatible data source)
• PostgreSQL 9.3–9.5
• Amazon Aurora
• Amazon Redshift
• Amazon S3
• Amazon DynamoDB
Supported targets for AWS DMS
34. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
AWS Database Migration Service pricing
T2 for developing and periodic data migration tasks
C4 for large databases and minimizing time
T2 pricing starts at $0.018 per hour for T2.micro
C4 pricing starts at $0.154 per hour for C4.large
50 GB GP2 storage included with T2 instances
100 GB GP2 storage included with C4 instances
Data transfer inbound and within AZ is free
Data transfer across AZs starts at $0.01 per GB
35. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Resources available to customers—DMS
Getting Started Guide: review technical
documentation
Features and benefits: highlights DMS
features
Pricing: prices for replication instances,
storage, and data transfer
Support: post your questions to our
support forum
Java SDK: Java-based API for creating and
managing data migration tasks
AWS Command Line Interface: start and
stop replication tasks with simple
commands.
36. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Resources available to customers—SCT
User Guide: review technical docs at
aws.amazon.com/documentation/SchemaConversionTool/
or choose the Download button
Download area: get installation files for the
Schema Conversion Tool
Support forums: ask questions and review how-
to guides
https://forums.aws.amazon.com/forum.jspa?forumID=208
37. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
AWS database migration partners
38. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Thank you!
a w s . a m a z o n . c o m / d m s