Database Migration – Simple, Cross-Engine and Cross-Platform Migration
•
3 gefällt mir•3,532 views
Learn about the new AWS Database Migration Service, which helps you migrate databases with minimal downtime from on-premises and Amazon EC2 environments to Amazon RDS, Amazon Redshift, Amazon Aurora and EC2 databases.
Empfohlen
Empfohlen
Weitere ähnliche Inhalte
Was ist angesagt?
Was ist angesagt? (20)
Andere mochten auch
Andere mochten auch (20)
Ähnlich wie Database Migration – Simple, Cross-Engine and Cross-Platform Migration
Ähnlich wie Database Migration – Simple, Cross-Engine and Cross-Platform Migration (20)
Mehr von Amazon Web Services
Mehr von Amazon Web Services (20)
Kürzlich hochgeladen
Kürzlich hochgeladen (20)
Database Migration – Simple, Cross-Engine and Cross-Platform Migration
- 1. Database Migration: Simple, Cross-Engine and Cross-Platform Migrations with Minimal Downtime Prahlad Rao, Solutions Architect
- 2. Agenda • How does the cloud help? • How do I get there? • How does it work? • What have others done? • Are there any tricks? Where is the magic? • Show me!
- 3. Relational databases Fully managed Simple and fast to scale Fast, predictable performance Low cost, pay for what you use Amazon RDS
- 4. How does the cloud help? • Provision a database in 6 minutes • Provision an MAZ database with a few mouse clicks • Scale a database up/down with 60–90 seconds of downtime • Apply patches with 60–90 seconds of downtime • Add Read Replicas with a few mouse clicks • Protect your backups and logs with 11 9s of durability • Recover to any point in time from nightly backups + logs • Detailed metrics, down to 1-second intervals • Secure your data with single-click encryption at rest Amazon RDS
- 5. How can I get to the cloud? • How will my on-premises data migrate to the cloud? • How can I make it transparent to my users? • Afterwards, how will on-premises and cloud data interact? • How can I integrate my data assets within AWS? • Can I get help moving off of commercial databases?
- 6. Migration used to be cost + complexity + time • Commercial data migration and replication software • Complex to set up and manage • Application downtime • Database-engine-specific application code
- 7. Start your first migration in 10 minutes or less Keep your apps running during the migration Replicate within, to, or from Amazon EC2 or RDS Move data to the same or a different database engine AWS Database Migration Service (AWS DMS)
- 8. Customer premises Application users AWS Internet VPN • Start a replication instance • Connect to source and target databases • Select tables, schemas, or databases Let AWS DMS create tables, load data, and keep them in sync Switch applications over to the target at your convenience Keep your apps running during the migration AWS DMS
- 9. Load is table by table Replication instance Source Target
- 10. Change data capture (CDC) and apply Replication instance Source Target Update t1 t2 t1 t2 Transactions Change apply after bulk load
- 11. Multi-AZ option for high availability Customer premises or AWS AWS Internet VPN AWS DMS AWS DMS
- 12. Replication instance Source Target What else can I do? Source Source
- 14. Replication instance Source Target What else can I do? Target Target
- 15. AWS Schema Conversion Tool Features • Oracle and Microsoft SQL Server schema conversion to MySQL, Amazon Aurora, MariaDB, and PostgreSQL • Or convert your schema between PostgreSQL and any MySQL engine • Database Migration Assessment report for choosing the best target engine • Code browser that highlights places where manual edits are required • Secure connections to your databases with SSL • Cloud native code optimization The AWS Schema Conversion Tool helps automate many database schema and code conversion tasks when migrating between database engines or data warehouse engines
- 16. AWS Schema Conversion Tool • Converts relational databases • Converts warehouses
- 17. SCT helps with converting tables, views, and code Sequences User-defined types Synonyms Packages Stored procedures Functions Triggers Schemas Tables Indexes Views Sort and distribution keys
- 18. Split out demo with AWS Schema Conversion Tool
- 19. Take it all—or not Source Target Replication instance instance
- 20. Homogenous or heterogeneous Replication instance SQL Server MySQL Replication instance Oracle Oracle Replication instance Oracle Aurora
- 21. Other database migration use cases Migration of business-critical applications Migration from Classic to VPC Cheap Read Replicas for Oracle Read Replicas for other engines Cross-region Read Replicas for Oracle and SQL Server Analytics in the cloud Dev/test and production environment sync Ongoing replication for BI
- 22. Sources for AWS Database Migration Service Customers can use the following databases as a source for data migration using AWS DMS: On-premises and Amazon EC2 instance databases: • Oracle Database 10g–12c • Microsoft SQL Server 2005–2014 • MySQL 5.5–5.7 • MariaDB (MySQL-compatible data source) • PostgreSQL 9.4–9.5 • SAP ASE 15.7+ RDS instance databases: • Oracle Database 11g–12c • Microsoft SQL Server 2008R2–2014. CDC operations are not supported yet. • MySQL versions 5.5–5.7 • MariaDB (MySQL-compatible data source) • PostgreSQL 9.4–9.5. CDC operations are not supported yet. • Amazon Aurora (MySQL-compatible data source)
- 23. Targets for AWS Database Migration Service Customers can use the following databases as a target for data replication using AWS DMS: On-premises and EC2 instance databases: • Oracle Database 10g–12c • Microsoft SQL Server 2005–2016 • MySQL 5.5–5.7 • MariaDB (MySQL-compatible data source) • PostgreSQL 9.3–9.5 • SAP ASE 15.7+ RDS instance databases: • Oracle Database 11g–12c • Microsoft SQL Server 2008 R2 - 2014 • MySQL 5.5–5.7 • MariaDB (MySQL-compatible data source) • PostgreSQL 9.3–9.5 • Amazon Aurora (MySQL-compatible data source) Amazon Redshift
- 24. AWS Database Migration service pricing • T2 for developing and periodic data migration tasks • C4 for large databases and minimizing time • T2 pricing starts at $0.018 per hour for T2.micro • C4 pricing starts at $0.154 per hour for C4.large • 50 GB GP2 storage included with T2 instances • 100 GB GP2 storage included with C4 instances • • Data transfer inbound and within AZ is free • Data transfer across AZs starts at $0.01 per GB
- 26. Expedia: Online travel marketplace Migrating some databases to Amazon Aurora Kuldeep Chowhan, Principal Engineer, Expedia, Inc.: “The ease by which we can do this using the AWS Database Migration Service has simplified this process for us and enabled us to accelerate our migration efforts. The ability to closely monitor the process, the detailed logging feature, and the support we received from AWS have given us a great deal of confidence in a successful migration.” World’s leading online travel company, with a portfolio that includes 150+ travel sites in 70 countries.
- 27. Thomas Publishing: Digital-friendly business Needed to grow database footprint but using Oracle would require significant up front investment in both infrastructure and license expense Wanted to migrate to Amazon Aurora AWS DMS automated most of the work and dramatically reduced the manual effort involved in the code migration Hans Wald, Chief Technology Officer, Thomas Publishing: "The AWS Database Migration Service will be a key enabler for our plans to migrate more databases to Amazon Aurora in 2016.” Connecting buyers and suppliers across all industrial sectors, evolving from an industrial trade print publisher into industry’s most respected group of digital-friendly businesses.
- 28. Resources available to customers—DMS • Getting Started Guide: Review technical documentation. • Features and benefits: Highlights DMS features. • Pricing: Prices for replication instances, storage, and data transfer. • Support: Post your questions to our Support forum. • Java SDK: Java-based API for creating and managing data migration tasks. • AWS Command Line Interface: Start and stop replication tasks with simple commands.
- 29. Resources available to customers—AWS Schema Conversion Tool • User Guide: Review technical docs at • aws.amazon.com/documentation/SchemaConversion Tool/ • or choose the Download button. • Download area: Get installation files for the Schema Conversion Tool. • • Support forums: Ask questions and review how-to guides. • https://forums.aws.amazon.com/forum.jspa?forumID= 208.
- 30. AWS database migration partners
- 31. aws.amazon.com/activate Everything and Anything Startups Need to Get Started on AWS
Hinweis der Redaktion
- Start with some audience interaction… How many of you have been to the AWS Database Migration website? Has anyone tried using DMS yet?
- When migrating your database to the cloud you do more than just change location. You leverage the power of the cloud. Customers appreciate the flexibility that platforms like EC2 offer, and many do run database engines there, but the RDS engines take things to the next level.
- Other great features our customers rave about include the ability to get up an running quickly, deploy a redundant system easily, scale as required, have guaranteed backups, and secure information with a click.
- - Responsive to customers - Migrations easier and less intrusive - Migrate will low downtime to any RDS engine type - Post migration, customers asked for ongoing sync - Sync between on prem and also between engines - This mobility addresses one of the big requests: The ability to migrate off of commercial, license-intensive database engines, onto cloud-native, non-commercial engines.
- - Pre DMS, costly, license-intensive replication software to move data - Were not simple to setup or use required expert resources - Long downtimes the other option - Customers who wanted to switch engines had to deal with differences in datatypes, table syntax, sp’s, functions, triggers
- So it was with all these factors in mind that we developed the database migration service. We designed it to be simple - you can get started in less than ten minutes. We designed it to enable near-zero-downtime migration. And we designed it to be a kind of replication Swiss army knife. To replicate data between on-premises systems, RDS, EC2, and across database engine type
- Let’s explore how it is that DMS can enable near zero downtime migration. The customer starts by launching a DMS instance in their AWS account. (click) Then the customer provides database connection information to connect out to their on-premises database, and in to their AWS database. (click) Next the customer will select which tables, schemas or databases they want to migrate, (click) and DMS will load the data, and keep it in sync on an ongoing basis. (click) Finally, at the time of the customer’s choosing, they can simply change the application to point to the new AWS database, instead of their old on-premises database. (click)
- Creates tables at target database Sets up metadata required at target Populates data from source Each process loads one entire table Can use multiple processes Can be paused When restarted will continue from where it was stopped Will reload any tables that were currently in progress
- Oracle supplemental logging MySQL row-level bin logging SQL Server bulk logged/full recovery Postgres WAL No agent Uses recovery log Native change data capture API
- When you either migrate or replicate, you can choose the Multi-AZ option. This option provides DMS in a high availability mode–in case the primary DMS server fails, the secondary one kicks in.
- So by now it should be pretty obvious that data can move from A to B, and many have used DMS to do just that. Some customers have been pretty ingenious with DMS and have fanned in, or consolidated systems. DMS will put data where you ask it to, so if you want to merge data from a number of legacy systems into a single repository, DMS can help with this. And remember, DMS can just as easily move data between like databases and unlike databases, so you can take information from a number of different platforms and consolidate it in a single, cloud database.
- In addition to “fanning-in” or consolidating, we’ve had a customer who has taken their database and “fanned-out” different portions to different targets. For example, from their ERP system they replicate they customer list out to the CRM system and also take some of the general ledger transactions and funnel them over to a BI system. We’ve also had a customer who, for legacy reasons, used a single database for three different applications. With DMS you can split this information out to different databases. One could be PostgreSQL, one SQL Server, and one Aurora
- So what happens if you are converting from one database engine to another? Where does the magic happen? This is where the AWS Schema Conversion Tool steps in.
- - Embedded SQL conversion - Cloud native code optimization
- Some organisations have many years of historical data, and for those that are willing to make the leap, filtering can be done during a migration so that only newer information is kept. Filtering can be done on any field in the source database–not just dates.
- Just in case it wasn’t abundantly clear–DMS can move your data from pretty much anywhere to pretty much anywhere. Sort of like a courier company for the cloud
- Mention why no older engines as sources
- Mention submitted slides show only SQL 2014
- MySQL to Aurora. They are moving to Aurora because they have to store more data for compliance. Since they operate in every country there are lots of compliacnes they hae to adhere to. Currently 120 TB of data for 1.5 years. They need to start holding for 2 years. 120 TB. Reduce cost and improve performance. Lodging events and changes are stored in the database.
- Thomas Publishing is a family-owned company headquartered in New York City. They began in 1898 by publishing the Thomas Register of American Manufacturers, a 34-volume buying guide offering sourcing information and product specifications for industrial products and services. Thomas Publishing’s homegrown applications ran on a single, monolithic, Oracle database that could not support their growing user base with dynamic usage. Their application performance declined. Thomas Publishing came to Apps Associates looking for an alternative to the high cost associated with adding Oracle licenses, and the inflexible IT infrastructure necessary to maintain acceptable application performance. Solution Amazon Aurora AWS Database Migration Service S3 EC2 VPC Using the AWS Database Migration Service and Schema Conversion Tool, the migration was completed in less than 4 weeks and all the data was successfully imported into Amazon Aurora.
- Where can you learn more? (CLICK) The User Guide and download option are available from the DMS section of the AWS Console (CLICK) Yeah, there’s the Download button. Click it and you’ll see something like this. (CLICK) Information on installation and use (CLICK) The Schema Conversion Tool has its own forum to ask questions and get help