AWS Community Day CPH - Three problems of Terraform
Trivadis TechEvent 2017 Tools and Methods for DB Migrations by Kim Berg Hansen
1. BASEL BERN BRUGG DÜSSELDORF FRANKFURT A.M. FREIBURG I.BR. GENF
HAMBURG KOPENHAGEN LAUSANNE MÜNCHEN STUTTGART WIEN ZÜRICH
Tools and Methods for DB Migrations
DISCUSSION
Kim Berg Hansen
Senior Consultant
2. About me
Discussion: Tools and Methods for Migrating between Databases2 30-Oct-17
• Danish geek
• SQL & PL/SQL developer since 2000
• Developer at Trivadis AG since 2016
http://www.trivadis.dk
• Oracle Certified Expert in SQL
• Oracle ACE
• Blogger at http://www.kibeha.dk
• SQL quizmaster at
http://devgym.oracle.com
• Likes to cook
• Reads sci-fi
• Member of Danish Beer Enthusiasts
3. Agenda
Discussion: Tools and Methods for Migrating between Databases3 30-Oct-17
Brief intro of 3 migration cases with issues
Discussion
4. Case 1
Discussion: Tools and Methods for Migrating between Databases4 30-Oct-17
Old danish ERP system "Navigator" with own legacy database
Task: Copy all data from all tables into a new schema in Oracle database
Old ODBC driver for the legacy database available, but didn't have newer metadata
methods - had to use TABLE_INFORMATION, GetSchema(), GetSchemaTable()
Tried SQL Developer Migration tool - couldn't find tables with old driver
Tried various freeware or trialware tools - either same problem or they failed some
datatypes
Solution: C# console program written to:
– Generate CREATE TABLE scripts
– Copy all data with ODP.NET OracleBulkCopy Class
5. Case 2
Discussion: Tools and Methods for Migrating between Databases5 30-Oct-17
Application at Energy sector Trivadis customer
Task: Copy all data from app schema in Oracle to Microsoft SQL Server Database
Target schema was pre-created by app installer - a few small diffs in columns in a
few tables had to be mapped
Microsoft SQL Server Migration Assistant - Problem schema precreated datatypes
not quite matching -> Manual map every column!
+ Issues with BLOBs, Identity columns, and performance
Solution: PowerShell scripts:
– Source Oracle OLE DB Driver, target SQL Server .NET SqlBulkCopy class
– Mapping scripted based on column names
6. Case 3
Discussion: Tools and Methods for Migrating between Databases6 30-Oct-17
Trivadis very large German customer
Migrate data from Oracle to MySQL ( >1000 databases )
An existing tool didn't have quite correct datatype mapping
BulkCopy classes does not exist to MySQL
– (One attempt at such class exists, but not maintained for years)
Solution: Creation of new TVD-Espresso tool
– Classifies the many databases according to ease of migration
– Migrates data via JDBC and verifies correctness of migration
– Can do Flashback queries for point-in-time consistency
7. Discussion
Discussion: Tools and Methods for Migrating between Databases7 30-Oct-17
Themes:
Migrating data from one database brand to another
Tools available for such migrations
What to do when the tools aren't quite up to the task
Goal:
Sharing experiences
Gather tips to write a small paper to help for future migration projects