Authored by Bloor Research, this paper is about using information management software from SAP BusinessObjects for SAP data migration projects, either for upgrades from one version of SAP to a newer one, or from other environments to SAP.

1. SAP Data Migration Using
Business Objects Information
Management Software
A White Paper by Bloor Research
Author: Philip Howard
Review Date: 17 July 2008

2. Executive summary
This paper is about using information management software from Business Objects, an SAP
company, for SAP data migration projects, either for upgrades from one version of SAP to a
newer one, or from other environments to SAP. In practice, many of the considerations that
apply to SAP data migrations are no different from those that pertain generally to non-SAP
environments. However, there are some particular requirements that apply to SAP
implementations, which we will discuss.
A 2007 survey conducted by Kognitio found that 72% of UK financial services organizations
believe data migration projects to be too risky and too costly, and the report found similar
levels of trepidation across the retail, manufacturing, and transport sectors. This is true
regardless of the reason for the migration – whether the migration is based on a change of
application (such as moving from Oracle Applications to SAP), a change of database
provider, a consolidation of multiple applications of the same type, or even when upgrading
from one version of an application to another.
Organisations fear migrations because experience has taught them that migrations often
overrun their budgets, are delivered late, do not meet user expectations, and, in some cases,
fail completely. Moreover, it is not simply a question of a project running over time or over
budget – there are also significant business risks associated with overruns in terms of
customer relations and user morale, as well as application-specific risks, such as potential
delays in introducing new products.
In order to understand why these overruns occur, it is important to understand the various
elements that make up a migration from, say, one version of SAP R/3 to another. The first
part consists of the two versions of R/3 themselves. Since SAP R/3 software is well known
and understood, there is little application development to be done at this level other than
customisation. However, there is a significant amount of work to be done at the data level,
both in the movement and cleansing of the data that has to be migrated, and in the process
involved in the migration. It is necessary to understand the relationships that exist between
different data elements so that you can migrate business entities (such as a customer with his
orders) rather than treating these separately, which runs the risk that these become
disconnected.
Of course, if you are developing a new application or re-developing an existing one, then
there is considerable risk associated with this work at the application level. However, very
many migrations do not involve much more than customisation of the new application and yet
still companies continue to experience overruns, delays, and failures. The conclusion has to
be that the migration of the data is the issue.
Research bears this out. In 2006/2007, Bloor Research conducted a survey with Global 2000
companies and found that more 80% of data migration projects ran over time and/or over
budget. What is more, The Standish Group conducted a similar study in 1999 and got almost
identical results. In other words, the ability of the industry to properly conduct data migrations
has not improved in seven years!
However, this does not mean that it is impossible to undertake data migration projects that are
successful and are delivered on time and on budget. Rather, successful data migration
requires the use of appropriate tools and methodologies, best practices, and expertise, and it
is on these factors that we focus in this white paper. While there are some specific
requirements for SAP environments that go beyond those that might be necessary for other
data migration scenarios, the details and discussion points in this paper otherwise apply to
any data migration project.
© Bloor Research July 17, 2008 Page 2

3. In practice, many migration projects are delayed longer due to the fear of project failure. As a
result, this only makes them complex and time-consuming. It is our view that with the
appropriate processes, tools, and people in place (perhaps through a migration competency
centre), data migration should become routine, so that migrations can be planned when they
are most useful for the business and without the concern over potential failures that
characterise them today.
© Bloor Research July 17, 2008 Page 3

4. Before you begin
In our research, we found that around 20% of companies had no separate budget or
timescale for data migration as opposed to the application element of their projects – a recipe
for disaster if ever there was one.
So, the first thing is that there must be a separate budget and timeline for data migration.
Moreover, an appropriate person with data management expertise should be on the overall
project steering committee from the outset. If data management issues are left to users and/or
application developers, it is all too easy for the project committee to make unfounded and
unwarranted assumptions about the data elements of the project, which can lead to precisely
the difficulties you want to avoid.
There remains the question of how you estimate the timescales and resources that will be
required to achieve a successful migration. There are three things that you need to know:
1. Where is the data coming from to populate your new application? The answer to this
may be simple, in that all the data is coming from an existing application, or it may be
more complex in that the data is coming from multiple existing applications. In the latter
case, if some of the same data (for example, customer data) currently exists in multiple
applications, then you have to decide which version of the customer data you are going to
use or how you are going to combine data from those sources. In the worst case
scenario, you may have data required for the new application that does not exist in current
applications but does exist in spreadsheets, Access databases, and other end-user
resources spread around the organisation that are not known about by the IT department.
In this case, you will need to find out about these resources by talking to relevant end
users. Finally, the new application may be able to exploit data that you do not currently
have at all and you will need to understand how to use this new data. If the data is not
mandatory then it should not affect initial implementation, but otherwise you will need to
know whether the data can be created manually or sourced from a third party.
2. What sort of state is this data in? Once you know where the data is coming from you
need to understand it both in terms of the structure of the data and its quality. The former
is important because the number and complexity of the mappings that you will construct to
take the data from their existing sources to the intended target systems will depend, in
part, on the structure of that data. The latter is important both in business terms (you do
not want to be working with missing, invalid, duplicated, or incorrect data) and for
technical reasons. It is important to understand this last reason. With a different data
model in the source and target systems, there may be constraints upon the data in the
new system that did not apply in the original application. For example, you might have a
customer ID field that requires a numeric entry; if the previous application did not have the
same constraint, then any non-numeric customer ID will either not load into the new
application or, if you can load it, it will not run with the new application – it may even crash
the new application.
3. How will the data be structured in the new environment? This is the inverse of
knowing the structure of the existing system and, combined with that information, can be
used to derive the mappings and transformations that will be required.
Once you have this information, you need someone who can interpret it. That is, you need
someone who can estimate how long it will take (and what resources will be required) to
create the relevant mappings from the source to the target system, and how long it will take to
clean up the data from the source systems so they are in a suitable format for loading into the
© Bloor Research July 17, 2008 Page 4

5. new application. It is worth noting at this point that this is not a pipedream: some organizations
specializing in data migration will give you a fixed price quote and fixed delivery schedule,
provided that they have the information just outlined. Note that we are not necessarily
advocating such an approach; we are simply pointing out that once you know all about the
data, and given relevant expertise, then you know everything you need to know to about the
likely costs and resources that will be involved in the migration.
In practice, points 1) and 3) are relatively easy – particularly when you are migrating between
known environments, such as an SAP R/3 upgrade or a migration from a third-party
application to SAP R/3, because both sides of the equation are well-known and documented.
Where there is an issue is with point 2), because most companies do not know what state
their data is in – and this is complicated further if there are multiple source systems such as
when consolidating several applications of the same type. The process of understanding your
data is known as profiling.
Tools
As we have intimated in the previous section, managing a successful migration project is not
simply about the tools that you use but also about the skills and expertise of the people
involved. Our research indicates that the average company in the Global 2000 undertakes 4.5
migration projects each year – in which case it might be worth considering the establishment
of a migration competency centre or, at the least, requiring that data migration specialists get
recognition and rewards commensurate with the importance of accomplishing successful and
timely migrations. The skills possessed by such individuals will mitigate the risk involved in
migrations and reduce any worries associated with them. This should mean that businesses
can choose to migrate when it best serves the business rather than being constrained by fear
of failure.
Nevertheless, while expertise is important, so too are tools and the facilities they provide. The
appropriate use of tools will not only assist in project estimation and scoping but also in
cleansing and transforming the data, documenting the processes involved, and in ensuring
that data lineage is provided for compliance reasons.
The main tools required are data profiling, data cleansing and matching, and transformation
and data movement tools. These information management tools are provided by Business
Objects in the form of:
o BusinessObjects Data Insight, which provides facilities for profiling and analysing the
data in your source systems. To put it simply, BusinessObjects Data Insight allows you to
discover the extent of the problem you will have when migrating the data. Historically, and
even today in very many companies, profiling has been done manually (that is, by
visually inspecting the data and comparing it with what you expect to see). There are a
number of problems with this approach: it is easy to miss things, it is time-consuming, it is
labour-intensive, and it is boring. In particular, the fact that it is boring means that nobody
wants to do it, so it tends to be given to less experienced personnel who are less likely to
do a good job. In other words, you get a vicious circle. In particular, companies who use
manual profiling never do enough profiling – and when we say never, we mean never!
We have never, ever met a company that did an adequate job of manual profiling. Thus,
we consider the use of automated data profiling to be mandatory.
o BusinessObjects Data Quality Management, which provides data cleansing, matching,
de-duplication, and similar capabilities, as well as the ability to enrich the data from
outside sources such as Dun & Bradstreet. BusinessObjects Data Quality Management
© Bloor Research July 17, 2008 Page 5

6. includes metrics management and dashboard capabilities so that you can monitor the
quality of your data as you proceed through the data migration project.
o BusinessObjects Data Integrator, which is the company’s tool for accessing any data
source, defining appropriate transformations, and scheduling and running the actual
process of moving the data.
o BusinessObjects Data Services, which is a single product providing all the functionality
of BusinessObjects Data Integrator and BusinessObjects Data Quality Management
using a common development, runtime, and management environment.
In addition, Business Objects offers an extraction, transformation, and load (ETL) mapping
tool, BusinessObjects Composer, and the SAP NetWeaver Master Data Management
component. While we do not intend to spend time discussing these individually, since the
technologies involved are well-known, it is worth briefly commenting on the application of
master data management (MDM) to migration scenarios, in that the data consistency
provided by MDM should make future data migrations much easier. More broadly, the
implementation of data governance should mean that data sources are understood, and of
good quality, prior to the commencement of any migration project, thus significantly reducing
the efforts required in migration projects.
Business Objects is one of the leading vendors of information management software for data
profiling, data quality, and data integration products. Here we highlight the key features that
are especially appropriate in data migration environments in general, and in SAP
environments in particular:
o Connectivity – The ability to connect to a variety of sources and targets is important for
any data migration project. This includes databases, file systems, message queues, and
so on, supporting structured, semi-structured, and unstructured data. It is also useful to be
able to offer adapters that have a deep understanding of application environments, such
as SAP (see next bullet), Siebel, Oracle, and so forth. The reason is that connectivity
adapters facilitate the discovery of metadata during the data profiling stage and, indeed,
are useful during the movement phase also because they enable understanding of the
data model underlying the application.
o SAP Application Interfaces – It is important to shield the data migration developer from
having to know about specific SAP constructs such as ABAP, BAPI, and iDOCS, both
because this reduces the skill level required of the developer and also because it reduces
the complexity of the project and should, therefore, speed up the delivery of the project.
Business Objects provides appropriate application interfaces that have been certified by
SAP.
o Normalisation – SAP application environments are strictly normalised. That is, they
adhere closely to formal relational standards. However, many third-party applications do
not have that requirement and it is often the case that databases are de-normalised for a
variety of reasons. Thus, it is important when you are migrating to SAP that the software
you use has the ability to normalise the data. This is a facility within BusinessObjects Data
Integrator.
o Data Auditing and Validation – The ability to check if your data successfully loaded into
the target environment is necessary for any data migration job. With BusinessObjects
Data Integrator, an auditing feature allows you to check the sum and row count of data
leaving the source system and entering the target environment. Also, the ability to ensure
that unwanted data does not enter the target system is crucial to ensuring the success of
a data migration load. BusinessObjects Data Integrator also provides the ability to set up
© Bloor Research July 17, 2008 Page 6

7. rules that filter unwanted data. It also provides a dashboard for you view statistics on the
percentage of data which passed and didn’t pass your rules.
o Templates – What would be especially useful to support SAP migrations would be a
template of the target application, with the relevant data model built-in to the template.
This would significantly improve migration timescales. In practice, no vendor currently
supplies such a starter for migration, even though SAP has encouraged them to do
deliver such a capability. However, SAP and Business Objects are jointly developing such
a facility.
Managing the migration
A migration project may consist of multiple elements. For example, you may be migrating
across operating systems, hardware platforms, databases, and applications as well as
migrating the data. However, regardless of the complexity of the migration environment (and,
actually, it is a good idea to simplify it, migrating as few elements as possible within a single
project). The overall process can be likened to the design of a car: whatever the chassis,
bodywork, or interior trim, it is the engine that makes a car go; and it is the data that makes
applications go.
It is a good idea to bear this analogy in mind. For example, it is poor practice to reallocate staff
from the data migration team to other groups within a project just because the application part
of the migration is running behind schedule – this gives the impression that the application is
more important than the data and those that work on the latter are less important. This kind of
practice will be counterproductive if you wish to establish a data migration competency centre
or its equivalent.
More generally, you will need conventional project management tools for this purpose, either
from a third-party supplier or using built-in features, such as those provided by
BusinessObjects Composer. Note that just as data migration should have its own budget and
timescales, so it should be managed as its own subproject.
Conclusion
Our research shows that many IT professionals delay application migration and upgrade
decisions due to fear of failure, even when a delay may not be in the best interest of the
business. However, with the appropriate tools, methodology, and expertise, data migrations
can proceed smoothly and on time.
Of the above three requirements, all are important – but, perhaps most especially, expertise is
important. Companies need to build up a core of experienced data migration staff that are
recognised and rewarded as such, so that the business determines the pace of upgrades and
migrations, rather than worries about IT failure. In this context, it is worth noting that Business
Objects offers experienced data migration staff and consultants as well as an established
migration methodology, so the company can provide exactly the required expertise either on
an on-going basis or for knowledge transfer.
With respect to SAP in particular, a couple of technical requirements may mean that the
environment differs from some others. Clearly, as an SAP company, Business Objects is well-
placed to service these needs. Moreover, we can expect to see even closer links between the
data integration facilities provided by Business Objects and the application needs of SAP now
that they form a single company. In time, we expect Business Objects to offer data migration
© Bloor Research July 17, 2008 Page 7

8. facilities (such as templates) for SAP environments that are not available elsewhere, making
BusinessObjects Data Services software the de facto standard for SAP data migrations.
© Bloor Research July 17, 2008 Page 8