Database migration is the process of transferring data between different database systems or upgrades. It involves analyzing and mapping data from the source to the target system, transforming the data, validating data quality, and maintaining the migrated data. For example, Capital One migrated from Oracle to Teradata databases as their data volume grew too large for Oracle to efficiently handle. The migration process includes pre-migration planning, extraction, transformation, data loading, validation, and post-migration maintenance.
“Oh GOSH! Reflecting on Hackteria's Collaborative Practices in a Global Do-It...
Database Migration Process and Phases
1.
2. Database migration is the transferring of data between storage
types, formats, or computer systems. Database migration is
usually performed programmatically to achieve an automated
migration, freeing up human resources from tedious tasks. It is
required when organizations or individuals change computer
systems or upgrade to new systems.
For example:- Organisation want to move from one
database to other because of some business requirement, then
they can go for database migration. Capital One is one of the US
largest credit card industry, initially they used oracle as their
database, but when their volume grow Oracle not able to
support this huge volume of data, then they move to another
database called as Teradata which can store terabyte of
information and improve the data processing speed.
3.
4. Pre Migration :- Analysing, Mapping, Design
Migration : - Transform, Normalize and
backup
Post Migration : Quality Control, Clean-up,
Maintenance.
6. Analysis:- The analysis phase of data migration should be scheduled to
occur concurrently with the analysis phase of the core project. The aim of
the analysis phase in data migration projects is to identify the data
sources that must be transported into the new system. For example what
is my source database and what is my target database.
Mapping:- Mapped each source field to target field. Eg. In my source
table I have emp_id, salary as column in target database what should be
my column name and its datatype.
Design :- After you have decided upon the legacy data sources and have
conducted thorough data analysis, you must begin the roster selection.
This involves going through the list of data elements from each and every
source data structure, and deciding whether to migrate each one.
7. Transform:- After the design phase Transform
phase start. In this phase data transform
happen from source to target system. For this
organization can use different ETL(Extract,
Transform, Load) process.
ETL Process
Source Target
8. Normalize:- After moving the data to Target system, data normalization
required. Such as remove inconsistence data, remove duplicate data,
storing data based on Normal form. Also moving data from stage layer to
Target layer.
Normalization Stage
Stage Process
System System
9. Back Up:- This is one of the important phase of
the migration. After transforming data from
source to target state organization need to
create backup process. This will help during
database maintenance and recovery process.
10. Quality Control:-This is the import phase in Post migration process.
Here testing team need to check target database table structure, column
type and its value. Also need to map this value with source system value.
If any error occur then raise an defect and assign to the concern team.
Also ask Business to verify their system whether the output is as expected
or not. Also need to create reports, wellness check document and
published these document in the organization portal.
Clean-Up:- During migration process data transformation happened
through different channels, such as file system, Temporary tables etc.
After the migration process complete need to clean these temporary
system.
Maintenance:- The maintain phase is where all of the mappings are
validated and successfully implemented in a series of scripts that have
been thoroughly tested. In all organization they have separate team
which take care of this process, they maintain the Database, do
housekeeping. Organization spend huge amount in the maintenance
phase.
11. The solution which is proposed here works on the
concept of ETL methodology where in the data is
extracted from the source database transformed
within staging database and then commenced into
target database.
Whether to use staging database or not depends on
the fact that what is the amount of cleansing and
data transformation is required.
Minimal amount of data cleansing or
transformation can be managed within the data
migration tool using a query builder tool but as a
best practice major cleansing has to be done using
a staging database.