1. Establishing a Robust Data
Readiness Methodology
Prepared by:
James Chi

2. Our Recommended Solution
Summary
The data readiness strategy and methodology
The planning, execution, verification, and
described below is the result of an evolutionary
documentation of the migration of application data
process developed over many SAP
from legacy or source systems to SAP are critical
implementations with multiple clients in various
components to any successful SAP project
industry verticals. This methodology is intended to
implementation. SAP requires and expects master
not only deliver repeatable, predictable, and
and transactional data of high quality for the
demonstrate results; but also bring visibility to
intended process integration benefits to be
data quality issues early enough in the project to
realized.
mitigate them.
Data Readiness is, however, one of the most
overlooked aspects of an implementation project.
Data Readiness Components
This is partly because so much emphasis is placed Let us first introduce the distinct components that
on re-engineering the business processes that the make up a data migration landscape. As
quality and accuracy of data often takes a lesser illustrated in Figure 1, our recommended
priority. However, based on our experience, we methodology follows the traditional Extract,
would suggest that many SAP implementation Transform, and Load (ETL) data migration
projects simply lack the tools and methodologies component model.
to systematically identify and perform data
readiness and conversion activities and resolve
data quality issues.
Data Conversion Component Overview
Data Export
Data Input Source Data Staging
Destination
Source
Applications LSMW
Manual Data Collection BDC BDC Direct
/
iMac
via data construction
application
CATT
Manual Data Collection
-
via Excel and Flat File
Custom ABAP
Central Data Staging
And Transformation Tool
Manual Data Collection
via SAP Manual Input
SAP Systems
Extract Transform Load
Figure 1
Confidential GROM Associates, Inc. -2

3. Data Input Sources These data are subsequently provided to the
central Data Staging & Transformation tool
Data for the project implementation come from
sources as identified in the functional Manual Data Collection in Excel and Flat File – In
specifications. The data for loading into SAP either some cases the need to collect data manually that
already exists in an electronic format or are does not exist in the source system(s) is served by
manually captured in an approved electronic MS-Excel Spreadsheets or Flat Text File. Based on
format. Import programs need to be kept as the complexity of the data that is needed, the
simple as possible for faster implementation and project team develops and distributes an Excel
easier traceability. Import data can come from the spreadsheet application to help facilitate the
following sources: manual data collection process. The data is
subsequently uploaded to the central Data Staging
Source Application Data – Data from source & Transformation tool.
systems are either exported into a comma
delimited text file or copied tables when ODBC Manual Data Collection in SAP – In certain
database connections are available. Data are functional areas, the project can manually collect
extracted out of source applications following the data for SAP where data do not exist in source
principle of “all data and records” without data systems directly in the SAP system. It is
filtering, filtering, translation, or formatting. sometimes advantageous to build SAP data directly
in the SAP environment and take advantage of
Manual Data Collection – Data may be manually existing pre-defined data value tables and
collected in situations where source data does not validation logic. The data is subsequently
exist. Based on the complexity and referential
extracted from SAP and provided to the central
dependency of the collected data, a data Data Staging & Transformation tool.
construction application can be developed to help
facilitate the manual data collection and validation
process.
Data Readiness Process Overview
Extract Transform Load
1
1 7
2 4
SOURCE Staged s
Proces
Target TARGET
SYSTEMS Process Source Data SYSTEMS
Data
DATA STAGING Uploaded
Up
Up
APPLICATION Target Data
rt
rt
da
po
dat
po
5
te
Re
Re
e
8
Target Data
Source Data Kickouts
3 Kickouts
Configuration
Team
Team
Data Owner Referential
& Target Data
Supplemential Kickouts
Data
Data Owner
6
Figure 2
Confidential GROM Associates, Inc. -3

4. Data Staging Comprehensive Data Readiness Process
All master and transactional data loaded into the Let us now describe the steps involved in a robust
SAP system should be staged in a central Data and comprehensive data readiness process. The
Staging & Transformation tool. This repository overall process is illustrated in Figure 2.
receives source data and outputs transformed
In order to ensure ongoing execution,
target data. It contains source data in its
troubleshooting, and problem resolution
originally supplied form, all the rules to convert,
throughout the data conversion test cycles
translate, supplement and format this data into the
described in the next section “Data Conversion
destination format, and intermediate tables
Approach and Methodology”, the Systematic Data
required for data readiness processing. The output
Readiness Process is followed for each data test
from the central Data Staging & Transformation
run. Following is a high-level overview of the
tool is used as the source of data loads into SAP.
process.
Commercial ETL tools are designed for the purpose
of extracting, transforming, and loading data.
These tools should be leveraged on projects where
available. On projects where a commercial ETL
Step 1: Extraction of Source Data
tool is not available, native database tools such as The conversion starts with the extraction of source
Microsoft’s DTS or Oracle’s Warehouse Builder can data. This extraction, depending upon its source
be used as well. may be a direct ODBC connection, a spreadsheet
or flat file created programmatically, or a manually
Once staged in their original or approved collection
loaded spreadsheet. Original spreadsheets and
format, all data is filtered, translated, and
flat files must be secured in a centralized location
formatted in a traceable and reportable fashion via
for audit and validation purposes. In all cases, the
execution of individual data rules in the central
extract of source data must be accompanied by a
Data Staging & Transformation Tool. Exceptions to
report that details the contents. A Source Data
this rule should only be permitted for manually
Reconciliation Report should be produced for each
entered data objects.
extract and must indicate the total number of
records contained in the source. Other metrics
Data Export Destination Programs should be supplied for key data fields such as
sums, totals, or hash totals of data columns
Data is exported from the central Data Staging &
contained in the source. This information will be
Transformation tool into SAP via standard SAP
very important in demonstrating that the source
data conversion methods and tools. Data
data has been completely and accurately imported
programs must be kept as simple as possible to
into the central Data Staging & Transformation
ensure quick development and better traceability
tool.
for troubleshooting and reconciliation purposes.
These conversion methods and tools are:
LSMW – Legacy System Migration Step 2–3: Upload, Process, and
Workbench Verification of Extracted Data & Data
BDC Programs – Binary Direct Connection Quality Checkpoint One
CATT – Computer Aided Test Tool
Post Load Custom ABAP The next step in the process begins the upload of
Post Load Manual Input data from source applications and manual
collection repositories in their native format into
the central Data Staging & Transformation tool. It
is critical for all data to be imported into the
staging tool in an “as-is” format. All source
Confidential GROM Associates, Inc. -4

5. application tables and/or spreadsheet rows and source data from its original record format to a
columns are imported into the staging tool without format that can be read by the SAP data upload
any filtering and manipulation. This ensures that programs for loading into SAP. These data staging
all data record filtering, translation, harmonization, rules, define the main transformation of the
and formatting operations are performed in the filtered source data into data that is coded and
staging tool in an approved, auditable, traceable, formatted for SAP upload purposes. All data
and reportable fashion via execution of business formatting, filtering, and translation rules are
rules at individual source level. based on criteria documented in the functional
specifications. Data reconciliation activities are
Once the data has been successfully extracted into
performed to verify that all required business rules
the central Data Staging & Transformation tool,
defined in the functional specifications have been
the source data is modified according to data
completely and accurately applied.
filtering rules. Data filtering refers to reducing the
dataset based upon rules documented in the Step 5: Data Quality Checkpoint Two
functional specifications and business relevancy
parameters. This filtering is performed in order to Once the data has been successfully filtered,
ensure that only active and relevant data are translated, and formatted, the resulting dataset
loaded into SAP. Additionally, source data can can be subject to another set of quality and
now be subject to a variety of quality and integrity integrity checks aimed at identifying target data
checks to identify source data issues that can integrity and completeness issues. These issues
either be resolved in the staging tool as a can be resolved in the staging tool as a
transformation rule, resolved in SAP, resolved in
transformation rule or be resolved back in the
source system. Data records that do not pass key the data construction application, or resolved back
quality or integrity checks should be flagged as in the source system. Data records which do not
such and omitted from subsequent transformation pass key quality or integrity checks should be
and loading steps, and directed to Data Owners for flagged as such and omitted from subsequent
correction or clarification. loading steps, and directed to data owners and
configuration team for correction or clarification.
Data reconciliation activities are also performed.
All results are gathered and compared to the Data reconciliation activities are also performed
Source Data Reconciliation Report. Results and from the target SAP environment perspective. All
Kickouts are provided to Data Owners for review, results are gathered and compared to verify that
approval and correction. all required business rules defined in the functional
specifications have been completely and accurately
Step 4: Transformation of Staged Data applied. Results and kickouts reports are provided
to Data Owners and Configuration Team for review
Once the source data has been filtered, all source
and correction.
data are combined into a single staged target SAP
data for translation, supplementation and Step 6: Data Supplementation
formatting rules specifically designed for the target
environment per Design Specifications. Data Following review of target data results and
translation refers to replacing source system kickouts reports, data owners have the opportunity
coding, groupings, and other source system to inject additional data into the transformation
application data characteristics to corresponding process of staged data. Additional data refers to
SAP coding, groupings, and data characteristics. missing data component that is required according
to functional or SAP system specifications and
Supplementation refers to supplying additional
referential or required data according to Design cross reference data that mapping legacy data into
Specifications that are not available from source new SAP data per Design Specifications.
data. Data formatting refers to converting the Configuration team has the opportunity to verify,
validate and correct data value needed in target
Confidential GROM Associates, Inc. -5

6. SAP system in order to load approved staged Project Preparation – This phase is to provide
target data without errors. initial preparation and planning for the SAP
implementation project, the important data
Step 7-8: Loading of Target Data into SAP readiness issues addressed during the project
& Final Verification preparation phase are:
Subsequent to the successful completion of data Finalization of data migration scope and data
quality checks, translated and formatted data will readiness strategy
be loaded into SAP via any of the mechanism On-boarding of data team
described under the “Data Export Destination Installation of ETL toolset
Programs” section of this document and verified Initiation of legacy system connection and
for accuracy and completeness. This verification extraction
will involve a combination of visual inspection and
technical checks including record counts, sums, Business Blueprint – Define the business
and or hash totals of data columns contained in processes to be supported by the SAP system and
the export files and SAP tables. the functional requirements, data conversion and
readiness activities begins with the identification of
Data Readiness Approach and data objects which require conversion from the
Methodology source application to the SAP system. During this
phase, all data and records will be extracted and
profiled from source systems, business and SAP
Now that we have introduced both data readiness readiness requirements will be defined, and
landscape components and process, we can finally Mapping Documentation completed in order for
position how this all fits in the lifecycle of an SAP data quality report development. The quality and
implementation project. integrity of the source data will assessed
What follows is a description of the various data repeatedly during this period.
readiness activities as they are executed Realization (Build) – Build the system based
throughout the Grom’s Best Practice Data upon the requirements described in the functional
Readiness Approach. Grom’s Data Readiness specifications, included in this phase are several
Approach is an enhanced, refined and data readiness process development and individual
complementary to ASAP methodology that SAP data object testing cycles. During the early part of
implementation project is typically followed. realization, functional specifications are developed
for the data conversion objects identified during
requirements gathering. These design
Project Definition – The purpose of this phase is specifications serve as the basis for determining
to understand and define data quality baseline and which conversion mechanisms are used and
a path forward with respect to data readiness for provide additional functional conversion program
SAP implementation. Once data quality baseline development and testing details for a given data
has been defined and understood, data migration object. The project team develops all required data
and readiness scope can be derived and estimated conversion rules and programs. These conversion
in alignment with business objectives of SAP rules and programs are tested repeatedly in the
implementation. Toolset selection can be Q/A or Unit Test environments as illustrated in
accomplished based on scope of the conversion. Figure 3.
Finally, the effort and cost of the conversion can
be estimated for approval.
Confidential GROM Associates, Inc. -6

7. Continual Improvement Iterative Process
Business
Blueprint
Unit Test
Environments
Integration Test
Test
Environments
ing E
Pull On Demand
entsv
Sources
Data Staging
Application Go-Live
Cutover
Rehearsal
Environments
Results
Resolutions User Reports
SAP
Production
Figure 3
Realization (Test) – The purpose of this phase is Level prior to Go-Live as illustrated below in figure
dedicated for testing and refinement of conversion 4. By the end of this realization test phase, the
rules and programs of the central Data Staging central Data Staging & Transformation tool will be
and Transformation tool. As source data evolves tested with full data conversions in 2 to 3 rounds
in the course of normal business operation over of Unit Testing and 2 to 3 rounds of Integration
the project timeline, new data issues may surface Testing.
and conversion rules may need to be updated or
refined through the Continual Improvement Data Quality with Continual Improvement Process
Interactive Process. As the target SAP system in Transactionable Data Quality Level
High
each environments continue to mature into “To-
Data
Go-Live
Be” production system, data readiness will be Qua
lity
measured and reported against environment to
confirm alignment of design and functional
Business Blueprint
Install/Run/Support
specifications. Through this iterative testing and
Final Preparation
Da Act
repeatable process, data quality with respect to
ta ivit
Re ies
readiness will elevate closer toward
ad
Transactionable Data Quality
ine
ss
Realization Realization
(Build) (Test)
Low
Project Time Line
Figure 4
Confidential GROM Associates, Inc. -7

8. Final Preparation – Development of the central About the Author
Data Staging & Transformation tool is completed
and cutover activities will be rehearsed 2 to 3 James Chi is the Director of the GROM’s Business
rounds during this phase. As part of final Consulting Group Enterprise Solutions Practice and
production cutover, final source data extractions has overall delivery responsibilities for all GROM-
and preparations will be performed and all master led projects. James joined GROM after spending
and transactional data will be loaded into the the last seventeen years delivering SAP solutions
production environment. Production data in the pharmaceutical, medical device, and
reconciliation and validation reports will be consumer products industries. James’ strong
prepared to ensure all records are accounted for. functional background in Supply Chain Planning
Any additional manual data conversion activities and Manufacturing Execution has blended to create
and manual configuration steps in SAP will a well-rounded business expert with more than
executed according to conversion plan. Finally, fifteen years of Project Management experience.
data owners sign-off the production load and James has a BE in Electrical Engineering from
validation reports as required by the SAP Stevens Institute of Technology.
implementation project.
Install / Run / Support – As the purpose of this
phase is the transition from the pre-production
environment to live production operation, this
phase is used to closely monitor system
transactions, and to optimize system performance.
From a data conversion perspective, any post go-
live issues related to data should be investigated,
resolved, and closed.
Confidential GROM Associates, Inc. -8