Database migration doesn’t need to be difficult or time-consuming. Learn about the AWS Database Migration Service, which helps you migrate databases with minimal downtime from on-premises and cloud environments to Amazon RDS, Amazon Redshift, Amazon Aurora, Amazon DynamoDB, and Amazon EC2. We discuss homogeneous (same database engine) and heterogeneous migrations, as well as migrations from data warehouse platforms. We’ll also talk about the AWS Schema Conversion Tool, which saves you development time when migrating your Oracle, SQL Server, and data warehouse schemas and procedural code and exporting your data to the cloud. You'll hear from GumGum, an artificial intelligence company with deep expertise in computer vision that uses DMS to replicate its dimension data from different sources into a cohesive data warehouse.

1. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
John Winford, Senior Technical Program Manager, AWS
Vaibhav Puranik, VP of Engineering, GumGum
July 27, 2017
Deep Dive on AWS Database
Migration Service

2. Agenda
• How does the cloud help?
• How do I get there?
• When should I use it?
• How does it work?
• What else can I do?
• See it in action
• What have others done? Customer story: GumGum

3. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Cloud benefits

4. • Multi-engine support: Amazon Aurora, MySQL,
MariaDB, PostgreSQL, Oracle, SQL Server
• Automated provisioning, patching, scaling,
backup/restore, failover
• High availability with Amazon RDS Multi-AZ
– 99.95% SLA for Multi-AZ deployments
Amazon RDS

5. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
The journey

6. How can I get to the cloud?
How will my on-premises data migrate to the cloud?
How can I make it transparent to my users?
Afterwards, how will on-premises and cloud data interact?
How can I integrate my data assets within AWS?
Can I get help moving off of commercial databases?

7. Migration used to be cost + complexity + time
Commercial data migration and replication software
Complex to set up and manage
Application downtime
Database-engine-specific application code

8. What are AWS DMS and AWS SCT?
AWS Database Migration Service (AWS DMS) easily and
securely migrates and/or replicates your databases and data
warehouses to AWS.
AWS Schema Conversion Tool (AWS SCT) converts your
commercial database and data warehouse schemas to open-source
engines, Amazon Aurora and Amazon Redshift. Converts and loads
data warehouse data into Amazon Redshift.
We have migrated over 30,000 unique databases. And counting…

9. Migration options
If you’re not switching engines and can take downtime:
- SQL Server: bak file import
- MySQL: read replicas
- Oracle SQL Developer, Data Pump, Export/Import
- PostgreSQL: pg_dump
- SAP ASE: bcp

10. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
When should I use it?

11. When to use AWS DMS and AWS SCT?
Modernise Migrate Replicate
Modernise your database tier –
• Commercial to open-source
• Commercial to Amazon Aurora
Modernise your data warehouse –
• Commercial to Amazon Redshift
• Migrate business-critical
applications
• Migrate from Classic to VPC
• Migrate data warehouse to
Amazon Redshift
• Upgrade to a minor version
• Consolidate shards into Aurora
• Create cross-regions read replicas
• Run your analytics in the cloud
• Keep your dev/test and production
environment sync

12. When to use AWS SCT?
Modernise your database tier
• Commercial to open-source
• Commercial to Amazon Aurora
• Amazon S3 target
Modernise your warehouse
• Commercial to Amazon Redshift
Amazon Redshift
Amazon Aurora

13. When to use AWS DMS*?
Migrate
• Migrate business-critical
applications
• Migrate from Classic to VPC
• Migrate data warehouse to Amazon
Redshift
• Upgrade to a minor version
• Consolidate shards into Aurora
• Migrate from NoSQL to SQL, SQL
to NoSQL or NoSQL to NoSQL
Sources:
Targets:
Amazon
Dynamo DB
Amazon Redshift
Amazon S3
Amazon Aurora
*AWS DMS is a HIPAA certified service

14. Why use AWS DMS and AWS SCT?
Secure
Cost Effective
Remove Barriers
to Entry
Allow DB
Freedom
Keep a Leg in
the Cloud
Easy to Use, but
Sophisticated…
Near-Zero
Downtime

15. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
How does it work?

16. Database migration process
Step 1: Convert or copy your schema
Source DB or DW
AWS SCT
Native Tool
Destination DB or DW
Step 2: Move your data
Source DB or DW
AWS SCT
Destination DB or DW
AWS DMS

17. AWS Schema Conversion Tool

18. Customer
premises
Application users
AWS
Internet
VPN
Start a replication instance
Connect to source and target
databases
Select tables, schemas, or
databases
 Let AWS DMS create tables,
load data, and keep them in
sync
 Switch applications over to
the target at your convenience
Keep your apps running during the migration
AWS
DMS

19. Load is table by table
Replication instance
Source Target

20. Change data capture (CDC) and apply
Replication instanceSource Target
Update
t1 t2
t1
t2
Transactions Change
apply
after bulk
load

21. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
What else can I do?

22. Source
Target
What else can I do?
Source
Source
Replication instance

23. What else can I do? Target
Source Target
Target
Replication instance

24. Homogenous or heterogeneous
SQL Server
Amazon
Aurora
Oracle Oracle
Oracle
Amazon
S3

25. AWS SCT data extractors
Extract data from your data warehouse and migrate to Amazon Redshift
• Extracts through local migration agents
• Data is optimized for Amazon Redshift
and saved in local files
• Files are loaded to an Amazon S3 bucket
(through network or AWS Snowball) and
then to Amazon Redshift
Amazon
Redshift
AWS SCT
Amazon S3
Bucket

26. NoSQL support with AWS DMS
Migrate to AWS
• Move from MongoDB to Amazon DynamoDB
• Move from MongoDB to relational DBs
Move between NoSQL and SQL
• Change technologies
Amazon Aurora
DynamoDB
DynamoDB
Amazon RDS

27. Other database migration use cases
Migration of business-critical applications
Migration from Classic to Amazon VPC
Cheap read replicas for Oracle
Read replicas for other engines
Cross-region read replicas for Oracle and SQL Server
Analytics in the cloud
Dev/test and production environment sync
Ongoing replication for BI
Minor version upgrade

28. AWS database migration partners

29. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Demo – SQL to NoSQL

30. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
What have others done?

31. Our customers…

32. Who is saying What about AWS DMS and AWS SCT?
"We migrated hundreds of our clients from our in-house data-center to Amazon
RDS Oracle 12c using the AWS Data Migration Service (DMS). Due to this
service, we could live-replicate the databases between our data-center and RDS
before the migration. That kept the migration down-time to the very minimum.
We are very happy with DMS and are planning to use it for Oracle to MySQL
migration next.”
”The SCT Assessment Report was the key enabler to allow us to understand the
scope of effort required to complete an Oracle to PostgreSQL migration. What
was originally thought to be a largely manual task that no one was particularly
excited about having to do became a very straight-forward quick and easy
process."
“We are in the process of migrating some databases to Amazon Aurora. The ease
by which we can do this using the AWS Database Migration Service has
simplified this process for us and enabled us to accelerate our migration
efforts. The ability to closely monitor the process, the detailed logging feature,
and the support we received from AWS have given us a great deal of confidence
in a successful migration.”

33. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Vaibhav Puranik, VP of Engineering
AWS DMS Best Practices
At GumGum

34. GumGum
Applied Computer Vision Company
9 years old, 225 employees
Based in Santa Monica, CA
Offices in London, Sydney, New York, Chicago
Thousands of publishers and advertisers
Billions of impressions per day

35. Advertising

36. Sports

37. Replication use case
Amazon RDS
(MySQL)
Amazon
Redshift

38. Old replication solution
Groovy script that runs periodically
Issues
• Not real-time
• Hack
• Need to maintain

39. Why AWS DMS?
• No approval required!
• Security
• APIs

40. Implementation
• Proof of Concept
• Implementation in Secondary
• Implementation in Primary

41. Issues faced
• Amazon Redshift primary keys
• MySQL Enum fields
• Amazon RDS Auto Minor Upgrade
• Source database reboot
• Amazon S3 outage

42. Best practices
• One task per one table replication
• Script to add new task
• Create target tables manually
• Script to compare tables in source and target database
• Amazon Cloudwatch monitoring
• Experiment with instance types and number of tasks per
instance

43. When replication breaks…

44. Questions?
aws.amazon.com/dms