SlideShare ist ein Scribd-Unternehmen logo
1 von 28
Downloaden Sie, um offline zu lesen
IBM Global Technology Services
December 2009




                                 Implementing nondisruptive
                                 data migrations for UNIX using
                                 TDMF technology.
                                 A technical implementation guide
Implementing nondisruptive data migrations for UNIX using TDMF technology.
Page 2




                                           Introduction
                   Contents
                                           IBM Softek Transparent Data Migration Facility (TDMF®) technology for
                                           UNIX® is a host-based software that moves data at the block level between
        2    Introduction
                                           logical volumes (file system managed or raw volumes) without interrupting the
        2    TDMF technology for UNIX
             capabilities                  applications that read or write to those volumes. TDMF software works with
        6    TDMF architecture             any volume manager, file system or database raw volume.
        7    TDMF technology modes of
             operation                     This software is ideal for the following types of local data migrations:
        7    TDMF virtual migration
             device                        ●    Technology refresh, especially in heterogeneous or high-availability
        9    Migration phases for               environments
             dynamic activation mode       ●    Direct attached storage (DAS) to storage area network (SAN) migrations
       13    Migration phases for
                                           ●    Implementing tiered storage
             dynamic switchover mode
                                           ●    Consolidating storage locally
       17    Performance considerations
       21    Tips for conducting large
                                           ●    Dynamic data movement to fix performance problems (for example,
             migrations                         moving volumes from storage hot spots)
       23    Comparison with volume
             manager mirroring             This white paper presents an overview of Softek TDMF for UNIX capabilities,
       24    Other IBM Softek TDMF         its mode of operation and performance considerations.
             migration solutions
       25    Platform support
                                           TDMF technology for UNIX capabilities: Designed for simple installation
       25    Summary                       The distinct value of the TDMF for UNIX solution is its simplicity—three com-
                                           ponents designed to execute flawlessly together. It is designed to be simple to
                                           install, simple to use and simple to maintain.
Implementing nondisruptive data migrations for UNIX using TDMF technology.
Page 3




                                            Dynamic driver activation
                   Highlights               Dynamic driver activation (DDA) enables TDMF technology to be installed
                                            and configured without an application outage and allows a data migration to
                                            occur on demand.

                                            Patent-pending nondisruptive switchover
                                            A key differentiator for the TDMF software is its ability to nondisruptively
                                            switch the source and target volumes, which moves the applications dynami-
                                            cally onto new storage. This switchover feature, which can occur automatically
                                            or on command, results in redirection of an application’s read function (that
                                            is, from the original source to the target). The switchover is designed to occur
                                            with no disruption to the application.

                                            TDMF technology continues to synchronously write to source and target vol-
                                            umes, allowing switchback at any time. Switchback simply refers to the ability
                                            to repeatedly switch the source and target volumes. A switchback might be
                                            used, for example, to direct application reads back to the original source while
                                            tuning the configuration of the new storage volumes to improve performance.

                                            Volumes that are no longer in an applications’ read path can be taken offline
                                            without disruption.

      With TDMF technology, administra-     Flexible target configuration
      tors can proactively prepare a sys-   Once the TDMF software is installed and the source volumes are configured,
      tem for its future migration before   target volumes can be identified at any time. This capability allows an admin-
      the details are defined.               istrator to proactively prepare a system for a future migration event, even
                                            when the details of that event are still undefined (for example, new storage
                                            hardware has not yet been selected).
Implementing nondisruptive data migrations for UNIX using TDMF technology.
Page 4




                                           TDMF software supports the definition of up to eight target volumes for each
                   Highlights              source volume. This ability to create and synchronize multiple copies of data
                                           (known as “cloned data”) is useful when supporting multiple test environ-
                                           ments or a variety of backup scenarios.

                                           Finally, target volume characteristics can be different from source volume
                                           characteristics. For example, the source may be a simple volume while the tar-
                                           get may be a software redundant array of independent disks (RAID) or striped
                                           configuration.

      TDMF software allows configura-       When TDMF software is used, the number of volumes concurrently migrating
      tion of the number of volumes con-   data is configurable. It is also important to note that logical unit numbers
      currently migrating data.            (LUNs) and storage layout can be changed during the data migration, because
                                           logical volumes are being moved.

                                           Migration tuning parameters
                                           To adjust performance impact on applications during a migration, TDMF pro-
                                           vides the ability to control migration rate and processor impact through per-
                                           formance tuning parameters, including:

                                           ●    Block size—determines the amount of data moved for each single read and
                                                write during the copy phase.
                                           ●    Sleep interval—determines the pause interval between block movements.
                                           ●    Concurrent copies in progress—determines the number of threads concur-
                                                rently allocated to the copy phase of the migration. TDMF technology
                                                allows up to 50 concurrent volumes to be copied on a single copy
                                                command.
                                           ●    Concurrent copy processes allowed on a single volume—TDMF technology
                                                allows up to four copy processes to be copying data on a single volume.
                                           ●    Maximum speed allowable during a copy—sets the ceiling to the highest
                                                point that can be achieved during a copy by throttling the maximum
                                                speed.
Implementing nondisruptive data migrations for UNIX using TDMF technology.
Page 5




                                           Should business conditions require a change to these parameters, TDMF soft-
                     Highlights            ware supports the ability to pause the migration, reconfigure the parameters
                                           and then resume the migration from the point it left off.
      TDMF software can pause the
      migration, reconfigure the parame-    Migration groups
      ters and resume the migration from   TDMF supports the definition and concurrent migration of thousands of vol-
      where it left off.                   umes. Because migration projects may involve several hundred volumes sup-
                                           porting a range of application types, TDMF technology provides the ability to
                                           logically group volumes for efficient operational control. Each group’s migra-
                                           tion parameters are independently configured and controlled to best suit the
                                           business applications supported.

                                           Monitoring features
                                           TDMF technology provides the ability to monitor the migration process from
                                           start to finish. Statistics that may be monitored include details such as esti-
                                           mated time to completion, elapsed time, time remaining, copy rate, application
                                           read and write rate and percentage complete. This data is kept on disk for his-
                                           torical review.

      TDMF technology is designed to       Cluster interoperability
      work with any clustered system.      TDMF technology is designed to work with virtually any clustered system.
                                           Automation is available for HP Service Guard clusters, IBM HACMP™ clus-
                                           ters, VERITAS clusters and Sun clusters. This cluster integration automates
                                           the majority of the processes needed to successfully complete data migrations
                                           in highly available cluster environments. It can also reduce the complexity
                                           and risk in the most important production environments.
Implementing nondisruptive data migrations for UNIX using TDMF technology.
Page 6




                                            TDMF architecture
                   Highlights
                                            As stated earlier, one of the greatest distinctions of the TDMF for UNIX solu-
                                            tion is its ability to execute its components together smoothly. TDMF technol-
      The TDMF solution consists of
                                            ogy consists of three components:
      three components: CLI, filter driver
      and configuration files.                ●     Command line interface (CLI)—provides a user interface to issue com-
                                                  mands and control migrations.
                                            ●     Filter driver—captures application updates during the data migration.
                                            ●     Configuration files—contain system and migration configuration details ref-
                                                  erenced by the TDMF software.


                                            The TDMF software is transparent to the application(s) running on the appli-
                                            cation server. Because the TDMF driver is located between the file systems
                                            and the volume manager, there are no driver, microcode or multipathing
                                            dependencies.

                                                                          Application server (host)

                                                                                                          App

                                                                                  TDMF CLI
                                                                                                            File
                                                                                                          system
                                                                                       TDMF




                                                                                                      Softek TDMF
                                                                                                       filter driver


                                                                           TDMF config files

                                                                                                        Volume
                                                                                                        manager


                                            Figure 1: TDMF architecture
Implementing nondisruptive data migrations for UNIX using TDMF technology.
Page 7




                                           TDMF technology modes of operation
                   Highlights
                                           TDMF technology has two possible modes of operation: dynamic activation
                                           and dynamic switchover. The advantages of each mode are explained in
                                           greater detail in the following sections.

                                           Dynamic activation mode
                                           Dynamic activation mode allows TDMF for UNIX to be dynamically activated
                                           without any application outage. The TDMF driver is dynamically inserted for
                                           the volumes that are specified on the TDMF create command. This allows a
                                           data migration to begin on demand without advance setup or outages before
                                           the beginning of the data migration. This mode of operation requires a short
                                           application outage to cutover to the new storage at the end of the data
                                           migration.

                                           Dynamic switchover mode
                                           Dynamic switchover mode enables new storage to migrate without an applica-
                                           tion outage once TDMF for UNIX has been installed and configured. In this
                                           mode of operation, an initial application outage is required to configure the
                                           TDMF devices so that the dynamic switchover feature is enabled. After the
                                           TDMF devices are configured, data migrations can occur on demand with vir-
                                           tually no outages.

      TDMF for UNIX uses a virtual         TDMF virtual migration device
      migration device to support data     TDMF for UNIX uses a virtual migration device to support data integrity dur-
      integrity during dynamic swap and    ing dynamic swap and to capture the application updates during the migra-
      to capture the application updates   tion. This virtual migration device is created using the CLI. One virtual device
      during the migration.                is created for each logical volume that will be migrated.
Implementing nondisruptive data migrations for UNIX using TDMF technology.
Page 8




                                           The directory structure for the virtual migration device is /dev/mig. There are
                                           two entries created for each volume; a block device and a special character
                                           device. The name of the virtual device is chosen by the administrator of the
                                           migration, but the directory path must be /dev/mig. For example, if a migra-
                                           tion volume is created with the name of oracledb, then the following virtual
                                           devices are created:

                                           ●   /dev/mig/roracledb
                                           ●   /dev/mig/oracledb


                                           To activate the TDMF technology, the file system entries for the volumes to be
                                           migrated must be changed in /etc/fstab to point to the TDMF virtual devices.
                                           A script automates this process, saving the current fstab and then making the
                                           appropriate changes, to put the TDMF software in the data path. To activate
                                           these changes, the source file systems require an unmount and a remount,
                                           which normally requires a short application outage. This is only a requirement
                                           when using the “Dynamic Switchover” mode of operation.

                                           If the volumes to be migrated are raw volumes, it is recommended that the
                                           TDMF dynamic activation mode be used. This makes it possible to leave the
                                           application and any symbolic links completely untouched. The source vol-
                                           umes are monitored for input/output (IO) and mirrored to the target without
                                           any changes in the application.
Implementing nondisruptive data migrations for UNIX using TDMF technology.
Page 9




                                           Migration phases for dynamic activation mode


                                                         Installation                      Install TDMF


                                                                                   Create migration volumes

                                                          Volume                         Add new targets
                                                         migrations

                                                                                        Copy/synchronize


                                                                                         Stop application

                                                                                     Unmount file systems

                                                                                    Delete TDMF definitions
                                                         Cutover to
                                                        new storage
                                                                                    Rename volume groups

                                                                                        Start applications

                                                                                         Uninstall TDMF
                                           Figure 2: TDMF for UNIX phases for dynamic activation mode



                                           Phase 1: Install TDMF software
                                           The first phase of the migration is to install the software on the production
                                           application servers. This is done with the platform provided installation tool,
                                           for example, installp for IBM AIX®. The installation dynamically installs
                                           the TDMF software and loads the driver so it is ready for use. This normally
                                           takes about five minutes per server. It is recommended to put the TDMF dis-
                                           tribution on a network share or network file share (NFS) mount for ease of
                                           installation.
Implementing nondisruptive data migrations for UNIX using TDMF technology.
Page 10




                                           Phase 2: Migration volume creation
                                           The second phase of a migration is to define the source volumes to be
                                           migrated using the TDMF “create” subcommand. Once the migration volumes
                                           have been identified, they can be grouped at any time for more efficient oper-
                                           ational control.

                                           Note that installing TDMF software and creating or grouping the migration
                                           volumes can occur before target storage systems have even been decided
                                           upon. This lets an administrator proactively prepare a system for a future
                                           migration event. This phase requires no outage for activation when used with
                                           the dynamic driver mode of operation.

                                           Phase 3: Addition of target volumes
                                           Each migration volume requires at least one associated target volume of
                                           equal or greater size to start a migration copy process. Target volumes can be
                                           dynamically added to a migration configuration at any time, mapping a target
                                           volume to a migration volume (and its correlating source device).

                                           Once the target volume has been added, the TDMF driver begins to send all
                                           updates synchronously to both the source and the target volumes. This helps
                                           ensure that, during the complete data migration progress, all updates that
                                           occur by the application will be applied to both the source and target
                                           volumes.
Implementing nondisruptive data migrations for UNIX using TDMF technology.
Page 11




                                           The initial status of a newly added target volume is SCRATCH, which means
                   Highlights              that the target volume is not usable by an application until a copy
                                           is completed in the next phase, and both the target and the source are
                                           synchronized.

                                           Phase 4: Data copy
                                           The copy phase is started on command by an administrator. The copy occurs
                                           asynchronously in the background while the applications are up and running.
                                           During the copy process, the status of target volume is COPYING.

                                           The copy process can be paused and resumed during a migration. When
                                           paused and resumed, the copy operation begins from the point at which it was
                                           paused. It is also possible to start a migration from the beginning when
                                           paused if desired.

      A migration can be paused or         A migration can be paused or resumed at any time to modify parameters that
      resumed at any time to modify        increase or decrease the speed of the data copy process (called “throttling”).
      parameters that increase or          Throttling parameters allows an administrator to control subsystem I/O to
      decrease the speed of the data       adjust the impact that a migration has on an application’s performance,
      copy process.                        including:

                                           ●    Block size of each read and write request (the greatest effect on speed)
                                           ●    Sleep time used between each read and write copy request (10 ms
                                                default)
                                           ●    Concurrent copy threads (four default)
                                           ●    Concurrent copy processes used on a single volume (four default)
Implementing nondisruptive data migrations for UNIX using TDMF technology.
Page 12




                                           When the copy phase is completed, the migration volumes will automatically
                                           transition into the UPTODATE state. Once this transition occurs, then the
                                           migration volumes are eligible for the cutover phase. The TDMF provides the
                                           ability to view the status of a migration at any time.

                                           Phase 5: Cutover to new storage
                                           The cutover phase is the only phase in the dynamic driver mode of operation
                                           that requires a short application outage. The purpose of the cutover phase is
                                           to discontinue using the old storage and to rename the new storage to take on
                                           the exact identity of the old discontinued storage. It is very common for
                                           clients to use volume names and volume group names to build their backup
                                           infrastructure. Therefore, it is important not to change storage names during a
                                           data migration. Following are the steps required to perform the cutover phase:

                                           1.   Stop the application(s).
                                           2.   Unmount file systems.
                                           3.   Delete the TDMF definitions using the TDMF delete command.
                                           4.   Rename the volume groups by exporting the old volume groups and
                                                importing the new LUNs with the old volume group names.
                                           5.   Mount the file systems.
                                           6.   Start the application(s).
Implementing nondisruptive data migrations for UNIX using TDMF technology.
Page 13




                                            This marks the completion of a successful data migration using the dynamic
                                            driver mode. If the TDMF software is to be uninstalled, then that process can
                                            be performed at this time. After the TDMF devices have been deleted, no
                                            allocations exist and TDMF software can be uninstalled.

                                            Migration phases for dynamic switchover mode


                                                        Create migration devices
                             Installation
                                                           Re-mount file system


                                                                Add new targets

                                                              Copy/synchronize                             Online
                              Volume
                             migrations                                                                   migrations
                                                                    Switchover

                                                            Remove old volumes


                            Removal of                     Re-mount file systems
                               TDMF
                             (optional)
                                                                 Uninstall TDMF
                                            Figure 3: TDMF for UNIX phases for dynamic switchover mode
Implementing nondisruptive data migrations for UNIX using TDMF technology.
Page 14




                                           Phase 1: Install TDMF software
                   Highlights              As noted above, this is a typical software platform package installation.

      Once the migration volumes have      Phase 2: Migration volume creation
      been identified, they can be          The first phase of a migration is to create the virtual migration devices for
      grouped at any time for more effi-   each source volume and update /etc/fstab as described in the TDMF virtual
      cient operational control.           device section. Once the migration volumes have been created, they can be
                                           grouped at any time for more efficient operational control.

                                           Note that installing TDMF software and creating or grouping the migration
                                           volumes can occur before target storage systems have even been decided
                                           upon. This lets an administrator proactively prepare a system for a future
                                           migration event. Often, this preparation is conducted during a normally
                                           scheduled downtime window or during off-peak hours (because of the appli-
                                           cation bounce) and can take less than 15 minutes.

                                           Phase 3: Addition of target volumes
                                           Each migration volume requires at least one associated target volume of equal
                                           or greater size to start a migration copy process. Target volumes can be
                                           dynamically added to a migration configuration at any time, mapping a target
                                           volume to a migration volume (and its correlating source device).

                                           Once the target volume has been added, the TDMF driver begins to send all
                                           updates synchronously to both the source and the target volumes. This helps
                                           ensure that, during the complete data migration progress, all updates that
                                           occur by the application will be applied to both the source and target
                                           volumes.
Implementing nondisruptive data migrations for UNIX using TDMF technology.
Page 15




                                           The initial status of a newly added target volume is SCRATCH, which means
                                           that the target volume is not usable by an application until a copy is com-
                                           pleted and both the target and the source are synchronized.

                                           Phase 4: Data copy
                                           The copy phase is started on command by an administrator. The copy occurs
                                           asynchronously in the background while the applications are up and running.
                                           During the copy process, the status of target volume is COPYING.

                                           The copy process can be paused and resumed during a migration. When
                                           paused and resumed, the copy operation begins from the point at which it was
                                           paused. It is also possible to start a migration from the beginning when
                                           paused if desired.

                                           A migration can be paused or resumed at any time to modify parameters that
                                           increase or decrease the speed of the data copy process (called “throttling”).
                                           Throttling parameters allows an administrator to control subsystem I/O to
                                           adjust the impact that a migration has on an application’s performance,
                                           including:

                                           ●    Block size of each read and write request (the greatest effect on speed)
                                           ●    Sleep time used between each read and write copy request (10 minutes
                                                default)
                                           ●    Concurrent copy threads (four minutes default)
                                           ●    Concurrent copy processes used on a single volume (four minutes default).


                                           When the copy phase is completed, the migration volumes will automatically
                                           transition into the UPTODATE state. Once this transition occurs, then the
                                           migration volumes are eligible for the switchover phase. The TDMF software
                                           provides the ability to view the status of a migration at any time.
Implementing nondisruptive data migrations for UNIX using TDMF technology.
Page 16




                                           Phase 5: Switchover
                   Highlights              Switchover is a patent-pending feature of TDMF for UNIX software. This
                                           process is designed to help enable the applications to remain online and
      The switchover process is designed
                                           100 percent available. During the copy phase, all read requests are satisfied by
      to enable applications to remain
                                           the source disk device. When the switchover occurs, the source and target
      online and 100 percent available.
                                           relationship is reversed. Following the switchover, all read requests will be sat-
                                           isfied by the new volume (old target), while all updates (writes) are still
                                           applied to both the source and target volumes. The switchover is abstracted by
                                           the TDMF driver, which helps ensure full data integrity. All updates before
                                           and after the switchover are applied to both source and target volumes.

                                           A switchover can occur by command when migration and target volumes are
                                           in an UPTODATE state (fully synchronized). Switchover can also occur auto-
                                           matically following a copy operation by using the –x option, which is auto-
                                           matic switchover after the copy is complete.

                                           A switchover command can be repeated (called a “switchback”) in the event
                                           that a new storage volume is not performing as expected or configuration
                                           problems are encountered. Switchback is not possible after the old storage is
                                           removed.

                                           If a switchover command is attempted while any of the migration volumes are
                                           not in the UPTODATE status, a switchover for that logical volume will not
                                           occur. Instead, an error message will be generated so that the migration
                                           administrator will be aware of the situation.
Implementing nondisruptive data migrations for UNIX using TDMF technology.
Page 17




                                            Phase 6: Old volume removal
                   Highlights               When the switchover process is complete, the original source volumes can be
                                            removed from the migration pairs. When the old volumes are removed, all
                                            reads and writes are satisfied from the new storage device. By removing the
                                            original source volumes, the TDMF driver will no longer be writing to both
                                            the source and target volumes. The process of removing the original source
                                            volumes removes all TDMF allocation to these volumes. The volumes that
                                            have been removed from the migration can be varied offline and discontinued
                                            without an application outage. This marks the completion of a successful data
                                            migration. If the TDMF software is to be uninstalled at a future time, then the
                                            fstab will need to be restored or changed to point to the new volumes.

      After planning, the data copy phase   Performance considerations
      is normally the longest migration     Data copy speed
      phase.                                The data copy phase will normally be the longest elapsed time phase in the
                                            migration. Some recommendations for initial throttle settings include:

                                            ●    Leave the –s parameter (sleep) value at the default for normal migrations
                                                 (current default).
                                            ●    Set the –b parameter (block size) to 2048, which is a good general setting
                                                 for data migrations (current default).
                                            ●    Set the –t parameter (thread count) to 25, which gives the migration copy
                                                 25 concurrent copy threads.
                                            ●    Set the –n (concurrent copy processes) parameter to four for volumes that
                                                 are greater than 500 gigabytes in size (current default).
Implementing nondisruptive data migrations for UNIX using TDMF technology.
Page 18




                                           The limitation of concurrent copy threads is 50 per copy command. If this is
                   Highlights              not enough for the migration environment, multiple copy commands can be
                                           run concurrently, and there is no limitation in the number of concurrent
      Multiple copy commands can be
                                           copies that can be run. Of course, the storage subsystem eventually will be
      run concurrently.
                                           saturated, and no work other than the migration will occur.

                                           Below is a sample table of results showing the impact of modifying throttle
                                           parameters during a sample data migration. These measurements were taken
                                           on an IBM DS8000® system attached to an AIX 5.3 system.


                                           Sleep    Block size in KB       Speed in MB/s          Elapsed time to copy
                                                                                                  20 GB
                                                                                                  (h:m:s)
                                           0        32                     206.8                  0:1:39
                                           0        64                     235.4                  0:1:24
                                           0        128                    255                    0:1:20
                                           0        256                    256                    0:1:20
                                           0        512                    243.8                  0:1:24
                                           0        1024                   235.4                  0:1:27
                                           0        2048                   227.5                  0:1:30
                                           0        4096                   252.8                  0:1:21
                                           0        8192                   252.8                  0:1:21
                                           10       8192                   4.9                    1:09:06
                                           10       64                     9.8                    0:34:36
                                           10       128                    19.4                   0:17:36
                                           10       256                    37.7                   0:9:03
                                           10       512                    71.9                   0:4:45
                                           10       1024                   127.2                  0:2:41
                                           10       2048                   193.8                  0:01:43
                                           10       4096                   227.5                  0:01:30
                                           10       8192                   243.8                  0:01:24
Implementing nondisruptive data migrations for UNIX using TDMF technology.
Page 19




                                           Processor and memory consumption
                   Highlights              The impact on the application and storage systems from TDMF software is
                                           very low. The majority of the resources consumed are I/O resources. The
      TDMF software is designed to con-
                                           processor consumption will normally be less than 3 percent of a single proces-
      sume minimal processor and mem-
                                           sor for the software. The memory consumed by a single copy process can typi-
      ory resources.
                                           cally be less than 3 MB but can vary based on the adjustment of thread count
                                           and block size.1 High thread count values and higher block size can cause the
                                           memory and processor consumption to increase.

                                           Below is an example of the resource consumption used during a migration
                                           and the recorded data transfer speed.


                                            Environment setup              Migration test               Migration results
                                            AIX 5.3                        Copy using four threads on   Processor consumption –
                                                                           a single copy with block     3 percent of a single
                                                                           size of 2048K                processor

                                            TDMF technology for UNIX 3.1                                Memory consumption –
                                            software IBM multipath                                      3344K
                                            input/output software

                                            2 Fiber host bus adapters                                   Sustained I/O rate –
                                                                                                        61 MB/s

                                            Source: IBM DS8000

                                            Target: IBM DS8000



                                           TDMF driver overhead
                                           IBM System Services Software Development has measured the overhead of
                                           the TDMF driver when in synchronous mode (during the copy phase) with a
                                           100 percent write workload. The results indicate an additional throughput and
                                           transfer speed overhead of about 25 percent.1
Implementing nondisruptive data migrations for UNIX using TDMF technology.
Page 20




                                            To put these results into perspective: Most software mirroring drivers can add
                   Highlights               up to 100 percent overhead in the I/O latency. Given that a typical commer-
                                            cial workload has about a 20 percent write ratio, the overhead of the synchro-
                                            nous driver is actually greatly reduced in a live environment.

                                            Note that the described overhead is related to that period of time in which
                                            TDMF software is copying data to synchronize the source and target systems,
                                            which is a few days on average. At all other times, TDMF software adds
                                            extremely minimal overhead, especially when compared with traditional vol-
                                            ume manager mirrors.

      With TDMF software, migration vol-    Grouping
      umes can be added or deleted from     TDMF technology has the ability to group volumes together for operational
      groups without affecting the active   control. The migration volumes can be put into groups so that it is easier to
      migration.                            control migrations with a single command(s) applied to all volumes in the
                                            group. All migration volumes will always be in the group of “all” so that com-
                                            mands can be entered with the scope of all or –a.

                                            The grouping is decided upon by the migration administrator and can be
                                            changed dynamically. Migration volumes can be added or deleted from groups
                                            without affecting the active migration. It is recommended that the grouping
                                            follows some conventions such as by volume group name or by application or
                                            by array that is being migrated.
Implementing nondisruptive data migrations for UNIX using TDMF technology.
Page 21




                                           Tips for conducting large migrations
                   Highlights
                                           When multiple application requirements exist or when large servers are
                                           migrated, it is recommended that the migration be controlled through the use
                                           of groups and dynamic switchover features.


                                           When a migration uses the dynamic switchover feature, you have the ability to
                                           start redirecting I/O application by application over to new hardware systems.
                                           Combined, these capabilities can produce several favorable results:

      The synchronous driver overhead is   ●   The overhead of the synchronous driver can be minimized because it is
      minimized because the dynamic            working by group, rather than slowing down the entire system by moving
      switchover feature works by group,       all volumes simultaneously.
      rather than by moving all volumes    ●   As volumes are migrated from the older system and application I/O is
      simultaneously.                          redirected to the new system, active I/O to the volumes on the original
                                               source can be reduced, which can improve application performance
                                               through load balancing.
                                           ●   As the migration proceeds and I/O is redirected volume by volume, appli-
                                               cations can begin taking advantage of performance enhancements on the
                                               newer storage system for potential service level improvements.


                                           When a large migration uses the dynamic driver feature, the migration can be
                                           started with virtually no outages. This allows the migration to require less
                                           planning to schedule an initial outage to activate TDMF software and to acti-
                                           vate the migration much closer to the required cutover date. Highly active
                                           volumes should be migrated later in the migration to reduce mirroring
                                           overhead.
Implementing nondisruptive data migrations for UNIX using TDMF technology.
Page 22




                                           Migration involving multiple applications
                                           The following is an example of a migration where a storage hardware lease is
                                           expiring and data is being moved to a new higher-performing storage system.
                                           The storage system has 90 logical volumes defined, with 40 of them support-
                                           ing a mission-critical, highly available Oracle application and 50 volumes sup-
                                           porting a data warehouse with predictable access patterns. Variable access
                                           exists during the week, but no access is available on weekends.


                                           The administrator groups the 40 Oracle volumes into Source Group A and the
                                           50 warehouse volumes into Source Group B.

                                           ●    Source Group A: Migration is configured to occur nondisruptively during
                                                the week (that is, running in the background) with synchronization
                                                expected in three business days. After peak business hours, the administra-
                                                tor accelerates migration speed slightly before going home; the adjust-
                                                ments affect all volumes in the group. In the morning, the administrator
                                                throttles back the migration of all volumes by modifying a single group
                                                parameter. By Friday, the target is synchronized with the source and a
                                                switchover occurs automatically.
                                           ●    Source Group B: This group is configured to migrate as swiftly and rap-
                                                idly as possible during weekend hours when the warehouse is rarely
                                                accessed and when the Oracle application from Group A is scheduled to
                                                be offline for two four-hour maintenance windows. This configuration
                                                maximizes the speed and I/O usage of the machine while the Oracle appli-
                                                cation is offline. On Saturday, the administrator starts the copy operation
                                                for Group B volumes during the first four-hour downtime window. With a
                                                single command, the administrator pauses the migration when the Oracle
                                                application is put back online. The next day, the administrator resumes
                                                the copy operation for this group with a single command; migration is
                                                complete before the second four-hour downtime window expires and a
                                                switchover occurs to the new hardware.
Implementing nondisruptive data migrations for UNIX using TDMF technology.
Page 23




                                           With both groups fully migrated and both applications now reading from the
                   Highlights              target device, the original source device can be removed at any time.

      The grouping feature allows users    Without the grouping feature, the administrator would have had to issue copy,
      to independently control different   pause and throttle-change pause commands for each of the 90 volumes. The
      workloads during a data migration.   grouping feature allows for independent control of different workloads during
                                           a data migration.

                                           Larger-scale migrations
                                           A similar approach can be used for larger migration scenarios. For example,
                                           suppose the migration involves 2,000 volumes supporting five applications.
                                           One approach could be to group volumes by application and conduct migra-
                                           tions one application at time, using dynamic switchover to maintain a per-
                                           formance balance.

                                           If the migration involves 2,000 volumes supporting a single application,
                                           another approach might be to group the volumes in 10 groups of 200. As
                                           each group of 200 is moved, an automatic switchover lets the old source vol-
                                           umes be removed or repurposed right away. This also removes any overhead
                                           introduced by synchronously writing to both the source and target volumes.

      Several key factors make TDMF        Comparison with volume manager mirroring
      software a stronger migration        Some of the reasons why using the TDMF software can be a better migration
      solution than volume manager         solution than volume manager mirroring include the following:
      mirroring.
                                           ●    Volume manager mirroring copies a single volume from start to finish.
                                                TDMF software allows a volume to be copied concurrently with up to four
                                                copy processes.
                                           ●    Volume manager mirroring is not always included in the base volume
                                                manager and can add monetary cost and complexity to a migration
                                                project.
Implementing nondisruptive data migrations for UNIX using TDMF technology.
Page 24




                                           ●   Volume manager mirrors only mirror data within a single volume group.
                                               Often, this is not an option when the new LUNs do not fit into existing
                                               volume groups. TDMF technology allows the copying of data between vol-
                                               ume groups.
                                           ●   Volume manager mirroring normally only allows copying one volume at a
                                               time within a volume group, while TDMF software allows many volumes
                                               to be copied at a time within volume groups.
                                           ●   Volume manager mirroring does not allow the enterprise to change the
                                               volume or volume group characteristics during the migration.
                                           ●   There are volume group types such as RAID 5 that are not supported by
                                               volume mirroring. TDMF software can copy RAID 5 volume groups to
                                               other RAID 5 volume groups or to simple volumes.
                                           ●   Volume manager mirroring has no pacing or restart option built in. These
                                               limitations can hurt application performance during a migration and can
                                               extend the length of a migration project when problems occur.
                                           ●   Most enterprises have a mix of volume managers and operating environ-
                                               ments. TDMF software provides a single, standard method to perform a
                                               data migration in these environments.

                                           Other IBM Softek TDMF migration solutions
                                           IBM Softek TDMF software was designed and optimized specifically for local
                                           data migrations in which data will not be moved across a Transmission
                                           Control Protocol/Internet Protocol (TCP/IP) network. Softek provides TDMF
                                           (IP) for migrations involving movement over distances (known as “global
                                           migrations”). TDMF (IP) uses TCP/IP to deliver data over a local area net-
                                           work (LAN) or wide area network (WAN).
Implementing nondisruptive data migrations for UNIX using TDMF technology.
Page 25




                                              Both IBM Softek TDMF software and IBM Softek TDMF (IP) software can be
                   Highlights                 used in the same environment to address different migration project require-
                                              ments. TDMF software provides a very fast, simplified, optimized migration
      TDMF software is designed to pro-
                                              solution for local migrations; TDMF (IP) software provides a broader set of
      vide a fast, simplified, optimized
                                              features ideal for moving data asynchronously over distances.
      migration solution for local migra-
      tions, while TDMF (IP) software pro-    Platform support
      vides a broader set of features ideal   TDMF for UNIX is currently generally available on:
      for moving data over distances.         ●   HP-UX on the PA-RISC and Intel® Itanium® platforms
                                              ●   Solaris on SPARC platforms
                                              ●   IBM AIX on RISC platforms

                                              Summary
                                              TDMF technology is optimized for local data migrations involving heteroge-
                                              neous storage devices. It is designed to be lightweight, easy to install and easy
                                              to use. TDMF can scale as a solution for small- to large-scale data migration
                                              projects. This software can be controlled locally on each server via the CLI or
                                              centrally via the Data Mobility Console.

                                              TDMF technology can help ensure that applications maintain maximum avail-
                                              ability even as the underlying storage infrastructure is phased out and taken
                                              offline, enabling customers, storage vendors and integrators to successfully
                                              and quickly move to new storage technology.
Notes
Notes
For more information                                                  © Copyright IBM Corporation 2009
                                                                           IBM Global Services
To learn more about implementing TDMF for UNIX, please contact your
                                                                           Route 100
IBM sales representative, or visit the following Web site:                 Somers, NY 10589
                                                                           U.S.A.
ibm.com/services/datamobility                                              Produced in the United States of America
                                                                           December 2009
                                                                           All Rights Reserved
                                                                           IBM, the IBM logo, ibm.com and TDMF are
                                                                           trademarks or registered trademarks of
                                                                           International Business Machines Corporation in
                                                                           the United States, other countries, or both. If
                                                                           these and other IBM trademarked terms are
                                                                           marked on their first occurrence in this
                                                                           information with a trademark symbol (® or ™),
                                                                           these symbols indicate U.S. registered or
                                                                           common law trademarks owned by IBM at the
                                                                           time this information was published. Such
                                                                           trademarks may also be registered or common
                                                                           law trademarks in other countries. A current list
                                                                           of IBM trademarks is available on the Web at
                                                                           “Copyright and trademark information” at
                                                                           ibm.com/legal/copytrade.shtml
                                                                           Intel and Itanium are trademarks or registered
                                                                           trademarks of Intel Corporation or its
                                                                           subsidiaries in the United States and other
                                                                           countries.
                                                                           UNIX is a registered trademark of The Open
                                                                           Group in the United States and other
                                                                           countries.
                                                                           Other product, company or service names may
                                                                           be trademarks or service marks of others.
                                                                           References in this publication to IBM products
                                                                           or services do not imply that IBM intends to
                                                                           make them available in all countries in which
                                                                           IBM operates.
                                                                       1
                                                                           Based on testing performed in an IBM lab




                                                                           SDW03013-USEN-00

Weitere ähnliche Inhalte

Was ist angesagt?

Introduction to parallel_computing
Introduction to parallel_computingIntroduction to parallel_computing
Introduction to parallel_computingMehul Patel
 
A service platform for development deployment and runtime management of real-...
A service platform for development deployment and runtime management of real-...A service platform for development deployment and runtime management of real-...
A service platform for development deployment and runtime management of real-...dmeil
 
Chapter 19 - Real Time Systems
Chapter 19 - Real Time SystemsChapter 19 - Real Time Systems
Chapter 19 - Real Time SystemsWayne Jones Jnr
 
Parallel computing in india
Parallel computing in indiaParallel computing in india
Parallel computing in indiaPreeti Chauhan
 
network ram parallel computing
network ram parallel computingnetwork ram parallel computing
network ram parallel computingNiranjana Ambadi
 
Real Time Operating System
Real Time Operating SystemReal Time Operating System
Real Time Operating SystemSharad Pandey
 
Indian Contribution towards Parallel Processing
Indian Contribution towards Parallel ProcessingIndian Contribution towards Parallel Processing
Indian Contribution towards Parallel ProcessingAjil Jose
 
Full introduction to_parallel_computing
Full introduction to_parallel_computingFull introduction to_parallel_computing
Full introduction to_parallel_computingSupasit Kajkamhaeng
 
Graphics processing uni computer archiecture
Graphics processing uni computer archiectureGraphics processing uni computer archiecture
Graphics processing uni computer archiectureHaris456
 
九月.點點.滴滴
九月.點點.滴滴九月.點點.滴滴
九月.點點.滴滴elusiveboy
 
Operating Systems 1 (10/12) - Scheduling
Operating Systems 1 (10/12) - SchedulingOperating Systems 1 (10/12) - Scheduling
Operating Systems 1 (10/12) - SchedulingPeter Tröger
 
Chapter 1 - introduction - parallel computing
Chapter  1 - introduction - parallel computingChapter  1 - introduction - parallel computing
Chapter 1 - introduction - parallel computingHeman Pathak
 
process management
 process management process management
process managementAshish Kumar
 

Was ist angesagt? (20)

Introduction to parallel_computing
Introduction to parallel_computingIntroduction to parallel_computing
Introduction to parallel_computing
 
A service platform for development deployment and runtime management of real-...
A service platform for development deployment and runtime management of real-...A service platform for development deployment and runtime management of real-...
A service platform for development deployment and runtime management of real-...
 
Chapter 19 - Real Time Systems
Chapter 19 - Real Time SystemsChapter 19 - Real Time Systems
Chapter 19 - Real Time Systems
 
Parallel computing in india
Parallel computing in indiaParallel computing in india
Parallel computing in india
 
design_flow
design_flowdesign_flow
design_flow
 
Rtos slides
Rtos slidesRtos slides
Rtos slides
 
network ram parallel computing
network ram parallel computingnetwork ram parallel computing
network ram parallel computing
 
Real Time Operating System
Real Time Operating SystemReal Time Operating System
Real Time Operating System
 
Indian Contribution towards Parallel Processing
Indian Contribution towards Parallel ProcessingIndian Contribution towards Parallel Processing
Indian Contribution towards Parallel Processing
 
Full introduction to_parallel_computing
Full introduction to_parallel_computingFull introduction to_parallel_computing
Full introduction to_parallel_computing
 
Graphics processing uni computer archiecture
Graphics processing uni computer archiectureGraphics processing uni computer archiecture
Graphics processing uni computer archiecture
 
九月.點點.滴滴
九月.點點.滴滴九月.點點.滴滴
九月.點點.滴滴
 
Operating Systems 1 (10/12) - Scheduling
Operating Systems 1 (10/12) - SchedulingOperating Systems 1 (10/12) - Scheduling
Operating Systems 1 (10/12) - Scheduling
 
ucOS
ucOSucOS
ucOS
 
RT linux
RT linuxRT linux
RT linux
 
Lecture 4 process cpu scheduling
Lecture 4   process cpu schedulingLecture 4   process cpu scheduling
Lecture 4 process cpu scheduling
 
Vxworks
VxworksVxworks
Vxworks
 
Chapter 1 - introduction - parallel computing
Chapter  1 - introduction - parallel computingChapter  1 - introduction - parallel computing
Chapter 1 - introduction - parallel computing
 
process management
 process management process management
process management
 
20090720 smith
20090720 smith20090720 smith
20090720 smith
 

Ähnlich wie Implementing nondisruptivedata migrations for UNIX usingTDMF technology.

Zenon The integrated solution
Zenon The integrated solutionZenon The integrated solution
Zenon The integrated solutionCopa-Data UK
 
Parallel language & compilers
Parallel language & compilersParallel language & compilers
Parallel language & compilersdikshagupta111
 
parallel language and compiler
parallel language and compilerparallel language and compiler
parallel language and compilerVignesh Tamil
 
Towards Software Defined Persistent Memory
Towards Software Defined Persistent MemoryTowards Software Defined Persistent Memory
Towards Software Defined Persistent MemorySwaminathan Sundararaman
 
Presentation date -20-nov-2012 for prof. chen
Presentation date -20-nov-2012 for prof. chenPresentation date -20-nov-2012 for prof. chen
Presentation date -20-nov-2012 for prof. chenTAIWAN
 
The evolution of data center network fabrics
The evolution of data center network fabricsThe evolution of data center network fabrics
The evolution of data center network fabricsCisco Canada
 
Clusters (Distributed computing)
Clusters (Distributed computing)Clusters (Distributed computing)
Clusters (Distributed computing)Sri Prasanna
 
On mp so c software execution at the transaction level
On mp so c software execution at the transaction levelOn mp so c software execution at the transaction level
On mp so c software execution at the transaction levelTAIWAN
 
443029825 cloud-computing-week8-9-pptx
443029825 cloud-computing-week8-9-pptx443029825 cloud-computing-week8-9-pptx
443029825 cloud-computing-week8-9-pptxAbdulqader Al-kaboudei
 
Citrix command lines
Citrix command linesCitrix command lines
Citrix command linesprincesly
 
DB2 for z/O S Data Sharing
DB2 for z/O S  Data  SharingDB2 for z/O S  Data  Sharing
DB2 for z/O S Data SharingSurekha Parekh
 
Ericsson Technology Review: The future of cloud computing: Highly distributed...
Ericsson Technology Review: The future of cloud computing: Highly distributed...Ericsson Technology Review: The future of cloud computing: Highly distributed...
Ericsson Technology Review: The future of cloud computing: Highly distributed...Ericsson
 

Ähnlich wie Implementing nondisruptivedata migrations for UNIX usingTDMF technology. (20)

Best Practices for Migration
Best Practices for MigrationBest Practices for Migration
Best Practices for Migration
 
Storage Migration Made Simple
Storage Migration Made SimpleStorage Migration Made Simple
Storage Migration Made Simple
 
Zenon The integrated solution
Zenon The integrated solutionZenon The integrated solution
Zenon The integrated solution
 
Parallel language & compilers
Parallel language & compilersParallel language & compilers
Parallel language & compilers
 
Mainframe
MainframeMainframe
Mainframe
 
parallel language and compiler
parallel language and compilerparallel language and compiler
parallel language and compiler
 
Towards Software Defined Persistent Memory
Towards Software Defined Persistent MemoryTowards Software Defined Persistent Memory
Towards Software Defined Persistent Memory
 
Presentation date -20-nov-2012 for prof. chen
Presentation date -20-nov-2012 for prof. chenPresentation date -20-nov-2012 for prof. chen
Presentation date -20-nov-2012 for prof. chen
 
The evolution of data center network fabrics
The evolution of data center network fabricsThe evolution of data center network fabrics
The evolution of data center network fabrics
 
Upper layer protocol
Upper layer protocolUpper layer protocol
Upper layer protocol
 
Crusoe processor
Crusoe    processorCrusoe    processor
Crusoe processor
 
Clusters (Distributed computing)
Clusters (Distributed computing)Clusters (Distributed computing)
Clusters (Distributed computing)
 
On mp so c software execution at the transaction level
On mp so c software execution at the transaction levelOn mp so c software execution at the transaction level
On mp so c software execution at the transaction level
 
Live migration
Live migrationLive migration
Live migration
 
Computer Port IT Solutions JNTU Lecture
Computer Port IT Solutions JNTU LectureComputer Port IT Solutions JNTU Lecture
Computer Port IT Solutions JNTU Lecture
 
443029825 cloud-computing-week8-9-pptx
443029825 cloud-computing-week8-9-pptx443029825 cloud-computing-week8-9-pptx
443029825 cloud-computing-week8-9-pptx
 
CT
CTCT
CT
 
Citrix command lines
Citrix command linesCitrix command lines
Citrix command lines
 
DB2 for z/O S Data Sharing
DB2 for z/O S  Data  SharingDB2 for z/O S  Data  Sharing
DB2 for z/O S Data Sharing
 
Ericsson Technology Review: The future of cloud computing: Highly distributed...
Ericsson Technology Review: The future of cloud computing: Highly distributed...Ericsson Technology Review: The future of cloud computing: Highly distributed...
Ericsson Technology Review: The future of cloud computing: Highly distributed...
 

Mehr von IBM India Smarter Computing

Using the IBM XIV Storage System in OpenStack Cloud Environments
Using the IBM XIV Storage System in OpenStack Cloud Environments Using the IBM XIV Storage System in OpenStack Cloud Environments
Using the IBM XIV Storage System in OpenStack Cloud Environments IBM India Smarter Computing
 
TSL03104USEN Exploring VMware vSphere Storage API for Array Integration on th...
TSL03104USEN Exploring VMware vSphere Storage API for Array Integration on th...TSL03104USEN Exploring VMware vSphere Storage API for Array Integration on th...
TSL03104USEN Exploring VMware vSphere Storage API for Array Integration on th...IBM India Smarter Computing
 
A Comparison of PowerVM and Vmware Virtualization Performance
A Comparison of PowerVM and Vmware Virtualization PerformanceA Comparison of PowerVM and Vmware Virtualization Performance
A Comparison of PowerVM and Vmware Virtualization PerformanceIBM India Smarter Computing
 
IBM pureflex system and vmware vcloud enterprise suite reference architecture
IBM pureflex system and vmware vcloud enterprise suite reference architectureIBM pureflex system and vmware vcloud enterprise suite reference architecture
IBM pureflex system and vmware vcloud enterprise suite reference architectureIBM India Smarter Computing
 

Mehr von IBM India Smarter Computing (20)

Using the IBM XIV Storage System in OpenStack Cloud Environments
Using the IBM XIV Storage System in OpenStack Cloud Environments Using the IBM XIV Storage System in OpenStack Cloud Environments
Using the IBM XIV Storage System in OpenStack Cloud Environments
 
All-flash Needs End to End Storage Efficiency
All-flash Needs End to End Storage EfficiencyAll-flash Needs End to End Storage Efficiency
All-flash Needs End to End Storage Efficiency
 
TSL03104USEN Exploring VMware vSphere Storage API for Array Integration on th...
TSL03104USEN Exploring VMware vSphere Storage API for Array Integration on th...TSL03104USEN Exploring VMware vSphere Storage API for Array Integration on th...
TSL03104USEN Exploring VMware vSphere Storage API for Array Integration on th...
 
IBM FlashSystem 840 Product Guide
IBM FlashSystem 840 Product GuideIBM FlashSystem 840 Product Guide
IBM FlashSystem 840 Product Guide
 
IBM System x3250 M5
IBM System x3250 M5IBM System x3250 M5
IBM System x3250 M5
 
IBM NeXtScale nx360 M4
IBM NeXtScale nx360 M4IBM NeXtScale nx360 M4
IBM NeXtScale nx360 M4
 
IBM System x3650 M4 HD
IBM System x3650 M4 HDIBM System x3650 M4 HD
IBM System x3650 M4 HD
 
IBM System x3300 M4
IBM System x3300 M4IBM System x3300 M4
IBM System x3300 M4
 
IBM System x iDataPlex dx360 M4
IBM System x iDataPlex dx360 M4IBM System x iDataPlex dx360 M4
IBM System x iDataPlex dx360 M4
 
IBM System x3500 M4
IBM System x3500 M4IBM System x3500 M4
IBM System x3500 M4
 
IBM System x3550 M4
IBM System x3550 M4IBM System x3550 M4
IBM System x3550 M4
 
IBM System x3650 M4
IBM System x3650 M4IBM System x3650 M4
IBM System x3650 M4
 
IBM System x3500 M3
IBM System x3500 M3IBM System x3500 M3
IBM System x3500 M3
 
IBM System x3400 M3
IBM System x3400 M3IBM System x3400 M3
IBM System x3400 M3
 
IBM System x3250 M3
IBM System x3250 M3IBM System x3250 M3
IBM System x3250 M3
 
IBM System x3200 M3
IBM System x3200 M3IBM System x3200 M3
IBM System x3200 M3
 
IBM PowerVC Introduction and Configuration
IBM PowerVC Introduction and ConfigurationIBM PowerVC Introduction and Configuration
IBM PowerVC Introduction and Configuration
 
A Comparison of PowerVM and Vmware Virtualization Performance
A Comparison of PowerVM and Vmware Virtualization PerformanceA Comparison of PowerVM and Vmware Virtualization Performance
A Comparison of PowerVM and Vmware Virtualization Performance
 
IBM pureflex system and vmware vcloud enterprise suite reference architecture
IBM pureflex system and vmware vcloud enterprise suite reference architectureIBM pureflex system and vmware vcloud enterprise suite reference architecture
IBM pureflex system and vmware vcloud enterprise suite reference architecture
 
X6: The sixth generation of EXA Technology
X6: The sixth generation of EXA TechnologyX6: The sixth generation of EXA Technology
X6: The sixth generation of EXA Technology
 

Kürzlich hochgeladen

A Call to Action for Generative AI in 2024
A Call to Action for Generative AI in 2024A Call to Action for Generative AI in 2024
A Call to Action for Generative AI in 2024Results
 
From Event to Action: Accelerate Your Decision Making with Real-Time Automation
From Event to Action: Accelerate Your Decision Making with Real-Time AutomationFrom Event to Action: Accelerate Your Decision Making with Real-Time Automation
From Event to Action: Accelerate Your Decision Making with Real-Time AutomationSafe Software
 
Advantages of Hiring UIUX Design Service Providers for Your Business
Advantages of Hiring UIUX Design Service Providers for Your BusinessAdvantages of Hiring UIUX Design Service Providers for Your Business
Advantages of Hiring UIUX Design Service Providers for Your BusinessPixlogix Infotech
 
Axa Assurance Maroc - Insurer Innovation Award 2024
Axa Assurance Maroc - Insurer Innovation Award 2024Axa Assurance Maroc - Insurer Innovation Award 2024
Axa Assurance Maroc - Insurer Innovation Award 2024The Digital Insurer
 
Data Cloud, More than a CDP by Matt Robison
Data Cloud, More than a CDP by Matt RobisonData Cloud, More than a CDP by Matt Robison
Data Cloud, More than a CDP by Matt RobisonAnna Loughnan Colquhoun
 
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...Miguel Araújo
 
04-2024-HHUG-Sales-and-Marketing-Alignment.pptx
04-2024-HHUG-Sales-and-Marketing-Alignment.pptx04-2024-HHUG-Sales-and-Marketing-Alignment.pptx
04-2024-HHUG-Sales-and-Marketing-Alignment.pptxHampshireHUG
 
Bajaj Allianz Life Insurance Company - Insurer Innovation Award 2024
Bajaj Allianz Life Insurance Company - Insurer Innovation Award 2024Bajaj Allianz Life Insurance Company - Insurer Innovation Award 2024
Bajaj Allianz Life Insurance Company - Insurer Innovation Award 2024The Digital Insurer
 
Boost Fertility New Invention Ups Success Rates.pdf
Boost Fertility New Invention Ups Success Rates.pdfBoost Fertility New Invention Ups Success Rates.pdf
Boost Fertility New Invention Ups Success Rates.pdfsudhanshuwaghmare1
 
A Domino Admins Adventures (Engage 2024)
A Domino Admins Adventures (Engage 2024)A Domino Admins Adventures (Engage 2024)
A Domino Admins Adventures (Engage 2024)Gabriella Davis
 
[2024]Digital Global Overview Report 2024 Meltwater.pdf
[2024]Digital Global Overview Report 2024 Meltwater.pdf[2024]Digital Global Overview Report 2024 Meltwater.pdf
[2024]Digital Global Overview Report 2024 Meltwater.pdfhans926745
 
Exploring the Future Potential of AI-Enabled Smartphone Processors
Exploring the Future Potential of AI-Enabled Smartphone ProcessorsExploring the Future Potential of AI-Enabled Smartphone Processors
Exploring the Future Potential of AI-Enabled Smartphone Processorsdebabhi2
 
IAC 2024 - IA Fast Track to Search Focused AI Solutions
IAC 2024 - IA Fast Track to Search Focused AI SolutionsIAC 2024 - IA Fast Track to Search Focused AI Solutions
IAC 2024 - IA Fast Track to Search Focused AI SolutionsEnterprise Knowledge
 
Artificial Intelligence: Facts and Myths
Artificial Intelligence: Facts and MythsArtificial Intelligence: Facts and Myths
Artificial Intelligence: Facts and MythsJoaquim Jorge
 
Automating Google Workspace (GWS) & more with Apps Script
Automating Google Workspace (GWS) & more with Apps ScriptAutomating Google Workspace (GWS) & more with Apps Script
Automating Google Workspace (GWS) & more with Apps Scriptwesley chun
 
The Role of Taxonomy and Ontology in Semantic Layers - Heather Hedden.pdf
The Role of Taxonomy and Ontology in Semantic Layers - Heather Hedden.pdfThe Role of Taxonomy and Ontology in Semantic Layers - Heather Hedden.pdf
The Role of Taxonomy and Ontology in Semantic Layers - Heather Hedden.pdfEnterprise Knowledge
 
Handwritten Text Recognition for manuscripts and early printed texts
Handwritten Text Recognition for manuscripts and early printed textsHandwritten Text Recognition for manuscripts and early printed texts
Handwritten Text Recognition for manuscripts and early printed textsMaria Levchenko
 
Scaling API-first – The story of a global engineering organization
Scaling API-first – The story of a global engineering organizationScaling API-first – The story of a global engineering organization
Scaling API-first – The story of a global engineering organizationRadu Cotescu
 
Boost PC performance: How more available memory can improve productivity
Boost PC performance: How more available memory can improve productivityBoost PC performance: How more available memory can improve productivity
Boost PC performance: How more available memory can improve productivityPrincipled Technologies
 
08448380779 Call Girls In Friends Colony Women Seeking Men
08448380779 Call Girls In Friends Colony Women Seeking Men08448380779 Call Girls In Friends Colony Women Seeking Men
08448380779 Call Girls In Friends Colony Women Seeking MenDelhi Call girls
 

Kürzlich hochgeladen (20)

A Call to Action for Generative AI in 2024
A Call to Action for Generative AI in 2024A Call to Action for Generative AI in 2024
A Call to Action for Generative AI in 2024
 
From Event to Action: Accelerate Your Decision Making with Real-Time Automation
From Event to Action: Accelerate Your Decision Making with Real-Time AutomationFrom Event to Action: Accelerate Your Decision Making with Real-Time Automation
From Event to Action: Accelerate Your Decision Making with Real-Time Automation
 
Advantages of Hiring UIUX Design Service Providers for Your Business
Advantages of Hiring UIUX Design Service Providers for Your BusinessAdvantages of Hiring UIUX Design Service Providers for Your Business
Advantages of Hiring UIUX Design Service Providers for Your Business
 
Axa Assurance Maroc - Insurer Innovation Award 2024
Axa Assurance Maroc - Insurer Innovation Award 2024Axa Assurance Maroc - Insurer Innovation Award 2024
Axa Assurance Maroc - Insurer Innovation Award 2024
 
Data Cloud, More than a CDP by Matt Robison
Data Cloud, More than a CDP by Matt RobisonData Cloud, More than a CDP by Matt Robison
Data Cloud, More than a CDP by Matt Robison
 
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
 
04-2024-HHUG-Sales-and-Marketing-Alignment.pptx
04-2024-HHUG-Sales-and-Marketing-Alignment.pptx04-2024-HHUG-Sales-and-Marketing-Alignment.pptx
04-2024-HHUG-Sales-and-Marketing-Alignment.pptx
 
Bajaj Allianz Life Insurance Company - Insurer Innovation Award 2024
Bajaj Allianz Life Insurance Company - Insurer Innovation Award 2024Bajaj Allianz Life Insurance Company - Insurer Innovation Award 2024
Bajaj Allianz Life Insurance Company - Insurer Innovation Award 2024
 
Boost Fertility New Invention Ups Success Rates.pdf
Boost Fertility New Invention Ups Success Rates.pdfBoost Fertility New Invention Ups Success Rates.pdf
Boost Fertility New Invention Ups Success Rates.pdf
 
A Domino Admins Adventures (Engage 2024)
A Domino Admins Adventures (Engage 2024)A Domino Admins Adventures (Engage 2024)
A Domino Admins Adventures (Engage 2024)
 
[2024]Digital Global Overview Report 2024 Meltwater.pdf
[2024]Digital Global Overview Report 2024 Meltwater.pdf[2024]Digital Global Overview Report 2024 Meltwater.pdf
[2024]Digital Global Overview Report 2024 Meltwater.pdf
 
Exploring the Future Potential of AI-Enabled Smartphone Processors
Exploring the Future Potential of AI-Enabled Smartphone ProcessorsExploring the Future Potential of AI-Enabled Smartphone Processors
Exploring the Future Potential of AI-Enabled Smartphone Processors
 
IAC 2024 - IA Fast Track to Search Focused AI Solutions
IAC 2024 - IA Fast Track to Search Focused AI SolutionsIAC 2024 - IA Fast Track to Search Focused AI Solutions
IAC 2024 - IA Fast Track to Search Focused AI Solutions
 
Artificial Intelligence: Facts and Myths
Artificial Intelligence: Facts and MythsArtificial Intelligence: Facts and Myths
Artificial Intelligence: Facts and Myths
 
Automating Google Workspace (GWS) & more with Apps Script
Automating Google Workspace (GWS) & more with Apps ScriptAutomating Google Workspace (GWS) & more with Apps Script
Automating Google Workspace (GWS) & more with Apps Script
 
The Role of Taxonomy and Ontology in Semantic Layers - Heather Hedden.pdf
The Role of Taxonomy and Ontology in Semantic Layers - Heather Hedden.pdfThe Role of Taxonomy and Ontology in Semantic Layers - Heather Hedden.pdf
The Role of Taxonomy and Ontology in Semantic Layers - Heather Hedden.pdf
 
Handwritten Text Recognition for manuscripts and early printed texts
Handwritten Text Recognition for manuscripts and early printed textsHandwritten Text Recognition for manuscripts and early printed texts
Handwritten Text Recognition for manuscripts and early printed texts
 
Scaling API-first – The story of a global engineering organization
Scaling API-first – The story of a global engineering organizationScaling API-first – The story of a global engineering organization
Scaling API-first – The story of a global engineering organization
 
Boost PC performance: How more available memory can improve productivity
Boost PC performance: How more available memory can improve productivityBoost PC performance: How more available memory can improve productivity
Boost PC performance: How more available memory can improve productivity
 
08448380779 Call Girls In Friends Colony Women Seeking Men
08448380779 Call Girls In Friends Colony Women Seeking Men08448380779 Call Girls In Friends Colony Women Seeking Men
08448380779 Call Girls In Friends Colony Women Seeking Men
 

Implementing nondisruptivedata migrations for UNIX usingTDMF technology.

  • 1. IBM Global Technology Services December 2009 Implementing nondisruptive data migrations for UNIX using TDMF technology. A technical implementation guide
  • 2. Implementing nondisruptive data migrations for UNIX using TDMF technology. Page 2 Introduction Contents IBM Softek Transparent Data Migration Facility (TDMF®) technology for UNIX® is a host-based software that moves data at the block level between 2 Introduction logical volumes (file system managed or raw volumes) without interrupting the 2 TDMF technology for UNIX capabilities applications that read or write to those volumes. TDMF software works with 6 TDMF architecture any volume manager, file system or database raw volume. 7 TDMF technology modes of operation This software is ideal for the following types of local data migrations: 7 TDMF virtual migration device ● Technology refresh, especially in heterogeneous or high-availability 9 Migration phases for environments dynamic activation mode ● Direct attached storage (DAS) to storage area network (SAN) migrations 13 Migration phases for ● Implementing tiered storage dynamic switchover mode ● Consolidating storage locally 17 Performance considerations 21 Tips for conducting large ● Dynamic data movement to fix performance problems (for example, migrations moving volumes from storage hot spots) 23 Comparison with volume manager mirroring This white paper presents an overview of Softek TDMF for UNIX capabilities, 24 Other IBM Softek TDMF its mode of operation and performance considerations. migration solutions 25 Platform support TDMF technology for UNIX capabilities: Designed for simple installation 25 Summary The distinct value of the TDMF for UNIX solution is its simplicity—three com- ponents designed to execute flawlessly together. It is designed to be simple to install, simple to use and simple to maintain.
  • 3. Implementing nondisruptive data migrations for UNIX using TDMF technology. Page 3 Dynamic driver activation Highlights Dynamic driver activation (DDA) enables TDMF technology to be installed and configured without an application outage and allows a data migration to occur on demand. Patent-pending nondisruptive switchover A key differentiator for the TDMF software is its ability to nondisruptively switch the source and target volumes, which moves the applications dynami- cally onto new storage. This switchover feature, which can occur automatically or on command, results in redirection of an application’s read function (that is, from the original source to the target). The switchover is designed to occur with no disruption to the application. TDMF technology continues to synchronously write to source and target vol- umes, allowing switchback at any time. Switchback simply refers to the ability to repeatedly switch the source and target volumes. A switchback might be used, for example, to direct application reads back to the original source while tuning the configuration of the new storage volumes to improve performance. Volumes that are no longer in an applications’ read path can be taken offline without disruption. With TDMF technology, administra- Flexible target configuration tors can proactively prepare a sys- Once the TDMF software is installed and the source volumes are configured, tem for its future migration before target volumes can be identified at any time. This capability allows an admin- the details are defined. istrator to proactively prepare a system for a future migration event, even when the details of that event are still undefined (for example, new storage hardware has not yet been selected).
  • 4. Implementing nondisruptive data migrations for UNIX using TDMF technology. Page 4 TDMF software supports the definition of up to eight target volumes for each Highlights source volume. This ability to create and synchronize multiple copies of data (known as “cloned data”) is useful when supporting multiple test environ- ments or a variety of backup scenarios. Finally, target volume characteristics can be different from source volume characteristics. For example, the source may be a simple volume while the tar- get may be a software redundant array of independent disks (RAID) or striped configuration. TDMF software allows configura- When TDMF software is used, the number of volumes concurrently migrating tion of the number of volumes con- data is configurable. It is also important to note that logical unit numbers currently migrating data. (LUNs) and storage layout can be changed during the data migration, because logical volumes are being moved. Migration tuning parameters To adjust performance impact on applications during a migration, TDMF pro- vides the ability to control migration rate and processor impact through per- formance tuning parameters, including: ● Block size—determines the amount of data moved for each single read and write during the copy phase. ● Sleep interval—determines the pause interval between block movements. ● Concurrent copies in progress—determines the number of threads concur- rently allocated to the copy phase of the migration. TDMF technology allows up to 50 concurrent volumes to be copied on a single copy command. ● Concurrent copy processes allowed on a single volume—TDMF technology allows up to four copy processes to be copying data on a single volume. ● Maximum speed allowable during a copy—sets the ceiling to the highest point that can be achieved during a copy by throttling the maximum speed.
  • 5. Implementing nondisruptive data migrations for UNIX using TDMF technology. Page 5 Should business conditions require a change to these parameters, TDMF soft- Highlights ware supports the ability to pause the migration, reconfigure the parameters and then resume the migration from the point it left off. TDMF software can pause the migration, reconfigure the parame- Migration groups ters and resume the migration from TDMF supports the definition and concurrent migration of thousands of vol- where it left off. umes. Because migration projects may involve several hundred volumes sup- porting a range of application types, TDMF technology provides the ability to logically group volumes for efficient operational control. Each group’s migra- tion parameters are independently configured and controlled to best suit the business applications supported. Monitoring features TDMF technology provides the ability to monitor the migration process from start to finish. Statistics that may be monitored include details such as esti- mated time to completion, elapsed time, time remaining, copy rate, application read and write rate and percentage complete. This data is kept on disk for his- torical review. TDMF technology is designed to Cluster interoperability work with any clustered system. TDMF technology is designed to work with virtually any clustered system. Automation is available for HP Service Guard clusters, IBM HACMP™ clus- ters, VERITAS clusters and Sun clusters. This cluster integration automates the majority of the processes needed to successfully complete data migrations in highly available cluster environments. It can also reduce the complexity and risk in the most important production environments.
  • 6. Implementing nondisruptive data migrations for UNIX using TDMF technology. Page 6 TDMF architecture Highlights As stated earlier, one of the greatest distinctions of the TDMF for UNIX solu- tion is its ability to execute its components together smoothly. TDMF technol- The TDMF solution consists of ogy consists of three components: three components: CLI, filter driver and configuration files. ● Command line interface (CLI)—provides a user interface to issue com- mands and control migrations. ● Filter driver—captures application updates during the data migration. ● Configuration files—contain system and migration configuration details ref- erenced by the TDMF software. The TDMF software is transparent to the application(s) running on the appli- cation server. Because the TDMF driver is located between the file systems and the volume manager, there are no driver, microcode or multipathing dependencies. Application server (host) App TDMF CLI File system TDMF Softek TDMF filter driver TDMF config files Volume manager Figure 1: TDMF architecture
  • 7. Implementing nondisruptive data migrations for UNIX using TDMF technology. Page 7 TDMF technology modes of operation Highlights TDMF technology has two possible modes of operation: dynamic activation and dynamic switchover. The advantages of each mode are explained in greater detail in the following sections. Dynamic activation mode Dynamic activation mode allows TDMF for UNIX to be dynamically activated without any application outage. The TDMF driver is dynamically inserted for the volumes that are specified on the TDMF create command. This allows a data migration to begin on demand without advance setup or outages before the beginning of the data migration. This mode of operation requires a short application outage to cutover to the new storage at the end of the data migration. Dynamic switchover mode Dynamic switchover mode enables new storage to migrate without an applica- tion outage once TDMF for UNIX has been installed and configured. In this mode of operation, an initial application outage is required to configure the TDMF devices so that the dynamic switchover feature is enabled. After the TDMF devices are configured, data migrations can occur on demand with vir- tually no outages. TDMF for UNIX uses a virtual TDMF virtual migration device migration device to support data TDMF for UNIX uses a virtual migration device to support data integrity dur- integrity during dynamic swap and ing dynamic swap and to capture the application updates during the migra- to capture the application updates tion. This virtual migration device is created using the CLI. One virtual device during the migration. is created for each logical volume that will be migrated.
  • 8. Implementing nondisruptive data migrations for UNIX using TDMF technology. Page 8 The directory structure for the virtual migration device is /dev/mig. There are two entries created for each volume; a block device and a special character device. The name of the virtual device is chosen by the administrator of the migration, but the directory path must be /dev/mig. For example, if a migra- tion volume is created with the name of oracledb, then the following virtual devices are created: ● /dev/mig/roracledb ● /dev/mig/oracledb To activate the TDMF technology, the file system entries for the volumes to be migrated must be changed in /etc/fstab to point to the TDMF virtual devices. A script automates this process, saving the current fstab and then making the appropriate changes, to put the TDMF software in the data path. To activate these changes, the source file systems require an unmount and a remount, which normally requires a short application outage. This is only a requirement when using the “Dynamic Switchover” mode of operation. If the volumes to be migrated are raw volumes, it is recommended that the TDMF dynamic activation mode be used. This makes it possible to leave the application and any symbolic links completely untouched. The source vol- umes are monitored for input/output (IO) and mirrored to the target without any changes in the application.
  • 9. Implementing nondisruptive data migrations for UNIX using TDMF technology. Page 9 Migration phases for dynamic activation mode Installation Install TDMF Create migration volumes Volume Add new targets migrations Copy/synchronize Stop application Unmount file systems Delete TDMF definitions Cutover to new storage Rename volume groups Start applications Uninstall TDMF Figure 2: TDMF for UNIX phases for dynamic activation mode Phase 1: Install TDMF software The first phase of the migration is to install the software on the production application servers. This is done with the platform provided installation tool, for example, installp for IBM AIX®. The installation dynamically installs the TDMF software and loads the driver so it is ready for use. This normally takes about five minutes per server. It is recommended to put the TDMF dis- tribution on a network share or network file share (NFS) mount for ease of installation.
  • 10. Implementing nondisruptive data migrations for UNIX using TDMF technology. Page 10 Phase 2: Migration volume creation The second phase of a migration is to define the source volumes to be migrated using the TDMF “create” subcommand. Once the migration volumes have been identified, they can be grouped at any time for more efficient oper- ational control. Note that installing TDMF software and creating or grouping the migration volumes can occur before target storage systems have even been decided upon. This lets an administrator proactively prepare a system for a future migration event. This phase requires no outage for activation when used with the dynamic driver mode of operation. Phase 3: Addition of target volumes Each migration volume requires at least one associated target volume of equal or greater size to start a migration copy process. Target volumes can be dynamically added to a migration configuration at any time, mapping a target volume to a migration volume (and its correlating source device). Once the target volume has been added, the TDMF driver begins to send all updates synchronously to both the source and the target volumes. This helps ensure that, during the complete data migration progress, all updates that occur by the application will be applied to both the source and target volumes.
  • 11. Implementing nondisruptive data migrations for UNIX using TDMF technology. Page 11 The initial status of a newly added target volume is SCRATCH, which means Highlights that the target volume is not usable by an application until a copy is completed in the next phase, and both the target and the source are synchronized. Phase 4: Data copy The copy phase is started on command by an administrator. The copy occurs asynchronously in the background while the applications are up and running. During the copy process, the status of target volume is COPYING. The copy process can be paused and resumed during a migration. When paused and resumed, the copy operation begins from the point at which it was paused. It is also possible to start a migration from the beginning when paused if desired. A migration can be paused or A migration can be paused or resumed at any time to modify parameters that resumed at any time to modify increase or decrease the speed of the data copy process (called “throttling”). parameters that increase or Throttling parameters allows an administrator to control subsystem I/O to decrease the speed of the data adjust the impact that a migration has on an application’s performance, copy process. including: ● Block size of each read and write request (the greatest effect on speed) ● Sleep time used between each read and write copy request (10 ms default) ● Concurrent copy threads (four default) ● Concurrent copy processes used on a single volume (four default)
  • 12. Implementing nondisruptive data migrations for UNIX using TDMF technology. Page 12 When the copy phase is completed, the migration volumes will automatically transition into the UPTODATE state. Once this transition occurs, then the migration volumes are eligible for the cutover phase. The TDMF provides the ability to view the status of a migration at any time. Phase 5: Cutover to new storage The cutover phase is the only phase in the dynamic driver mode of operation that requires a short application outage. The purpose of the cutover phase is to discontinue using the old storage and to rename the new storage to take on the exact identity of the old discontinued storage. It is very common for clients to use volume names and volume group names to build their backup infrastructure. Therefore, it is important not to change storage names during a data migration. Following are the steps required to perform the cutover phase: 1. Stop the application(s). 2. Unmount file systems. 3. Delete the TDMF definitions using the TDMF delete command. 4. Rename the volume groups by exporting the old volume groups and importing the new LUNs with the old volume group names. 5. Mount the file systems. 6. Start the application(s).
  • 13. Implementing nondisruptive data migrations for UNIX using TDMF technology. Page 13 This marks the completion of a successful data migration using the dynamic driver mode. If the TDMF software is to be uninstalled, then that process can be performed at this time. After the TDMF devices have been deleted, no allocations exist and TDMF software can be uninstalled. Migration phases for dynamic switchover mode Create migration devices Installation Re-mount file system Add new targets Copy/synchronize Online Volume migrations migrations Switchover Remove old volumes Removal of Re-mount file systems TDMF (optional) Uninstall TDMF Figure 3: TDMF for UNIX phases for dynamic switchover mode
  • 14. Implementing nondisruptive data migrations for UNIX using TDMF technology. Page 14 Phase 1: Install TDMF software Highlights As noted above, this is a typical software platform package installation. Once the migration volumes have Phase 2: Migration volume creation been identified, they can be The first phase of a migration is to create the virtual migration devices for grouped at any time for more effi- each source volume and update /etc/fstab as described in the TDMF virtual cient operational control. device section. Once the migration volumes have been created, they can be grouped at any time for more efficient operational control. Note that installing TDMF software and creating or grouping the migration volumes can occur before target storage systems have even been decided upon. This lets an administrator proactively prepare a system for a future migration event. Often, this preparation is conducted during a normally scheduled downtime window or during off-peak hours (because of the appli- cation bounce) and can take less than 15 minutes. Phase 3: Addition of target volumes Each migration volume requires at least one associated target volume of equal or greater size to start a migration copy process. Target volumes can be dynamically added to a migration configuration at any time, mapping a target volume to a migration volume (and its correlating source device). Once the target volume has been added, the TDMF driver begins to send all updates synchronously to both the source and the target volumes. This helps ensure that, during the complete data migration progress, all updates that occur by the application will be applied to both the source and target volumes.
  • 15. Implementing nondisruptive data migrations for UNIX using TDMF technology. Page 15 The initial status of a newly added target volume is SCRATCH, which means that the target volume is not usable by an application until a copy is com- pleted and both the target and the source are synchronized. Phase 4: Data copy The copy phase is started on command by an administrator. The copy occurs asynchronously in the background while the applications are up and running. During the copy process, the status of target volume is COPYING. The copy process can be paused and resumed during a migration. When paused and resumed, the copy operation begins from the point at which it was paused. It is also possible to start a migration from the beginning when paused if desired. A migration can be paused or resumed at any time to modify parameters that increase or decrease the speed of the data copy process (called “throttling”). Throttling parameters allows an administrator to control subsystem I/O to adjust the impact that a migration has on an application’s performance, including: ● Block size of each read and write request (the greatest effect on speed) ● Sleep time used between each read and write copy request (10 minutes default) ● Concurrent copy threads (four minutes default) ● Concurrent copy processes used on a single volume (four minutes default). When the copy phase is completed, the migration volumes will automatically transition into the UPTODATE state. Once this transition occurs, then the migration volumes are eligible for the switchover phase. The TDMF software provides the ability to view the status of a migration at any time.
  • 16. Implementing nondisruptive data migrations for UNIX using TDMF technology. Page 16 Phase 5: Switchover Highlights Switchover is a patent-pending feature of TDMF for UNIX software. This process is designed to help enable the applications to remain online and The switchover process is designed 100 percent available. During the copy phase, all read requests are satisfied by to enable applications to remain the source disk device. When the switchover occurs, the source and target online and 100 percent available. relationship is reversed. Following the switchover, all read requests will be sat- isfied by the new volume (old target), while all updates (writes) are still applied to both the source and target volumes. The switchover is abstracted by the TDMF driver, which helps ensure full data integrity. All updates before and after the switchover are applied to both source and target volumes. A switchover can occur by command when migration and target volumes are in an UPTODATE state (fully synchronized). Switchover can also occur auto- matically following a copy operation by using the –x option, which is auto- matic switchover after the copy is complete. A switchover command can be repeated (called a “switchback”) in the event that a new storage volume is not performing as expected or configuration problems are encountered. Switchback is not possible after the old storage is removed. If a switchover command is attempted while any of the migration volumes are not in the UPTODATE status, a switchover for that logical volume will not occur. Instead, an error message will be generated so that the migration administrator will be aware of the situation.
  • 17. Implementing nondisruptive data migrations for UNIX using TDMF technology. Page 17 Phase 6: Old volume removal Highlights When the switchover process is complete, the original source volumes can be removed from the migration pairs. When the old volumes are removed, all reads and writes are satisfied from the new storage device. By removing the original source volumes, the TDMF driver will no longer be writing to both the source and target volumes. The process of removing the original source volumes removes all TDMF allocation to these volumes. The volumes that have been removed from the migration can be varied offline and discontinued without an application outage. This marks the completion of a successful data migration. If the TDMF software is to be uninstalled at a future time, then the fstab will need to be restored or changed to point to the new volumes. After planning, the data copy phase Performance considerations is normally the longest migration Data copy speed phase. The data copy phase will normally be the longest elapsed time phase in the migration. Some recommendations for initial throttle settings include: ● Leave the –s parameter (sleep) value at the default for normal migrations (current default). ● Set the –b parameter (block size) to 2048, which is a good general setting for data migrations (current default). ● Set the –t parameter (thread count) to 25, which gives the migration copy 25 concurrent copy threads. ● Set the –n (concurrent copy processes) parameter to four for volumes that are greater than 500 gigabytes in size (current default).
  • 18. Implementing nondisruptive data migrations for UNIX using TDMF technology. Page 18 The limitation of concurrent copy threads is 50 per copy command. If this is Highlights not enough for the migration environment, multiple copy commands can be run concurrently, and there is no limitation in the number of concurrent Multiple copy commands can be copies that can be run. Of course, the storage subsystem eventually will be run concurrently. saturated, and no work other than the migration will occur. Below is a sample table of results showing the impact of modifying throttle parameters during a sample data migration. These measurements were taken on an IBM DS8000® system attached to an AIX 5.3 system. Sleep Block size in KB Speed in MB/s Elapsed time to copy 20 GB (h:m:s) 0 32 206.8 0:1:39 0 64 235.4 0:1:24 0 128 255 0:1:20 0 256 256 0:1:20 0 512 243.8 0:1:24 0 1024 235.4 0:1:27 0 2048 227.5 0:1:30 0 4096 252.8 0:1:21 0 8192 252.8 0:1:21 10 8192 4.9 1:09:06 10 64 9.8 0:34:36 10 128 19.4 0:17:36 10 256 37.7 0:9:03 10 512 71.9 0:4:45 10 1024 127.2 0:2:41 10 2048 193.8 0:01:43 10 4096 227.5 0:01:30 10 8192 243.8 0:01:24
  • 19. Implementing nondisruptive data migrations for UNIX using TDMF technology. Page 19 Processor and memory consumption Highlights The impact on the application and storage systems from TDMF software is very low. The majority of the resources consumed are I/O resources. The TDMF software is designed to con- processor consumption will normally be less than 3 percent of a single proces- sume minimal processor and mem- sor for the software. The memory consumed by a single copy process can typi- ory resources. cally be less than 3 MB but can vary based on the adjustment of thread count and block size.1 High thread count values and higher block size can cause the memory and processor consumption to increase. Below is an example of the resource consumption used during a migration and the recorded data transfer speed. Environment setup Migration test Migration results AIX 5.3 Copy using four threads on Processor consumption – a single copy with block 3 percent of a single size of 2048K processor TDMF technology for UNIX 3.1 Memory consumption – software IBM multipath 3344K input/output software 2 Fiber host bus adapters Sustained I/O rate – 61 MB/s Source: IBM DS8000 Target: IBM DS8000 TDMF driver overhead IBM System Services Software Development has measured the overhead of the TDMF driver when in synchronous mode (during the copy phase) with a 100 percent write workload. The results indicate an additional throughput and transfer speed overhead of about 25 percent.1
  • 20. Implementing nondisruptive data migrations for UNIX using TDMF technology. Page 20 To put these results into perspective: Most software mirroring drivers can add Highlights up to 100 percent overhead in the I/O latency. Given that a typical commer- cial workload has about a 20 percent write ratio, the overhead of the synchro- nous driver is actually greatly reduced in a live environment. Note that the described overhead is related to that period of time in which TDMF software is copying data to synchronize the source and target systems, which is a few days on average. At all other times, TDMF software adds extremely minimal overhead, especially when compared with traditional vol- ume manager mirrors. With TDMF software, migration vol- Grouping umes can be added or deleted from TDMF technology has the ability to group volumes together for operational groups without affecting the active control. The migration volumes can be put into groups so that it is easier to migration. control migrations with a single command(s) applied to all volumes in the group. All migration volumes will always be in the group of “all” so that com- mands can be entered with the scope of all or –a. The grouping is decided upon by the migration administrator and can be changed dynamically. Migration volumes can be added or deleted from groups without affecting the active migration. It is recommended that the grouping follows some conventions such as by volume group name or by application or by array that is being migrated.
  • 21. Implementing nondisruptive data migrations for UNIX using TDMF technology. Page 21 Tips for conducting large migrations Highlights When multiple application requirements exist or when large servers are migrated, it is recommended that the migration be controlled through the use of groups and dynamic switchover features. When a migration uses the dynamic switchover feature, you have the ability to start redirecting I/O application by application over to new hardware systems. Combined, these capabilities can produce several favorable results: The synchronous driver overhead is ● The overhead of the synchronous driver can be minimized because it is minimized because the dynamic working by group, rather than slowing down the entire system by moving switchover feature works by group, all volumes simultaneously. rather than by moving all volumes ● As volumes are migrated from the older system and application I/O is simultaneously. redirected to the new system, active I/O to the volumes on the original source can be reduced, which can improve application performance through load balancing. ● As the migration proceeds and I/O is redirected volume by volume, appli- cations can begin taking advantage of performance enhancements on the newer storage system for potential service level improvements. When a large migration uses the dynamic driver feature, the migration can be started with virtually no outages. This allows the migration to require less planning to schedule an initial outage to activate TDMF software and to acti- vate the migration much closer to the required cutover date. Highly active volumes should be migrated later in the migration to reduce mirroring overhead.
  • 22. Implementing nondisruptive data migrations for UNIX using TDMF technology. Page 22 Migration involving multiple applications The following is an example of a migration where a storage hardware lease is expiring and data is being moved to a new higher-performing storage system. The storage system has 90 logical volumes defined, with 40 of them support- ing a mission-critical, highly available Oracle application and 50 volumes sup- porting a data warehouse with predictable access patterns. Variable access exists during the week, but no access is available on weekends. The administrator groups the 40 Oracle volumes into Source Group A and the 50 warehouse volumes into Source Group B. ● Source Group A: Migration is configured to occur nondisruptively during the week (that is, running in the background) with synchronization expected in three business days. After peak business hours, the administra- tor accelerates migration speed slightly before going home; the adjust- ments affect all volumes in the group. In the morning, the administrator throttles back the migration of all volumes by modifying a single group parameter. By Friday, the target is synchronized with the source and a switchover occurs automatically. ● Source Group B: This group is configured to migrate as swiftly and rap- idly as possible during weekend hours when the warehouse is rarely accessed and when the Oracle application from Group A is scheduled to be offline for two four-hour maintenance windows. This configuration maximizes the speed and I/O usage of the machine while the Oracle appli- cation is offline. On Saturday, the administrator starts the copy operation for Group B volumes during the first four-hour downtime window. With a single command, the administrator pauses the migration when the Oracle application is put back online. The next day, the administrator resumes the copy operation for this group with a single command; migration is complete before the second four-hour downtime window expires and a switchover occurs to the new hardware.
  • 23. Implementing nondisruptive data migrations for UNIX using TDMF technology. Page 23 With both groups fully migrated and both applications now reading from the Highlights target device, the original source device can be removed at any time. The grouping feature allows users Without the grouping feature, the administrator would have had to issue copy, to independently control different pause and throttle-change pause commands for each of the 90 volumes. The workloads during a data migration. grouping feature allows for independent control of different workloads during a data migration. Larger-scale migrations A similar approach can be used for larger migration scenarios. For example, suppose the migration involves 2,000 volumes supporting five applications. One approach could be to group volumes by application and conduct migra- tions one application at time, using dynamic switchover to maintain a per- formance balance. If the migration involves 2,000 volumes supporting a single application, another approach might be to group the volumes in 10 groups of 200. As each group of 200 is moved, an automatic switchover lets the old source vol- umes be removed or repurposed right away. This also removes any overhead introduced by synchronously writing to both the source and target volumes. Several key factors make TDMF Comparison with volume manager mirroring software a stronger migration Some of the reasons why using the TDMF software can be a better migration solution than volume manager solution than volume manager mirroring include the following: mirroring. ● Volume manager mirroring copies a single volume from start to finish. TDMF software allows a volume to be copied concurrently with up to four copy processes. ● Volume manager mirroring is not always included in the base volume manager and can add monetary cost and complexity to a migration project.
  • 24. Implementing nondisruptive data migrations for UNIX using TDMF technology. Page 24 ● Volume manager mirrors only mirror data within a single volume group. Often, this is not an option when the new LUNs do not fit into existing volume groups. TDMF technology allows the copying of data between vol- ume groups. ● Volume manager mirroring normally only allows copying one volume at a time within a volume group, while TDMF software allows many volumes to be copied at a time within volume groups. ● Volume manager mirroring does not allow the enterprise to change the volume or volume group characteristics during the migration. ● There are volume group types such as RAID 5 that are not supported by volume mirroring. TDMF software can copy RAID 5 volume groups to other RAID 5 volume groups or to simple volumes. ● Volume manager mirroring has no pacing or restart option built in. These limitations can hurt application performance during a migration and can extend the length of a migration project when problems occur. ● Most enterprises have a mix of volume managers and operating environ- ments. TDMF software provides a single, standard method to perform a data migration in these environments. Other IBM Softek TDMF migration solutions IBM Softek TDMF software was designed and optimized specifically for local data migrations in which data will not be moved across a Transmission Control Protocol/Internet Protocol (TCP/IP) network. Softek provides TDMF (IP) for migrations involving movement over distances (known as “global migrations”). TDMF (IP) uses TCP/IP to deliver data over a local area net- work (LAN) or wide area network (WAN).
  • 25. Implementing nondisruptive data migrations for UNIX using TDMF technology. Page 25 Both IBM Softek TDMF software and IBM Softek TDMF (IP) software can be Highlights used in the same environment to address different migration project require- ments. TDMF software provides a very fast, simplified, optimized migration TDMF software is designed to pro- solution for local migrations; TDMF (IP) software provides a broader set of vide a fast, simplified, optimized features ideal for moving data asynchronously over distances. migration solution for local migra- tions, while TDMF (IP) software pro- Platform support vides a broader set of features ideal TDMF for UNIX is currently generally available on: for moving data over distances. ● HP-UX on the PA-RISC and Intel® Itanium® platforms ● Solaris on SPARC platforms ● IBM AIX on RISC platforms Summary TDMF technology is optimized for local data migrations involving heteroge- neous storage devices. It is designed to be lightweight, easy to install and easy to use. TDMF can scale as a solution for small- to large-scale data migration projects. This software can be controlled locally on each server via the CLI or centrally via the Data Mobility Console. TDMF technology can help ensure that applications maintain maximum avail- ability even as the underlying storage infrastructure is phased out and taken offline, enabling customers, storage vendors and integrators to successfully and quickly move to new storage technology.
  • 26. Notes
  • 27. Notes
  • 28. For more information © Copyright IBM Corporation 2009 IBM Global Services To learn more about implementing TDMF for UNIX, please contact your Route 100 IBM sales representative, or visit the following Web site: Somers, NY 10589 U.S.A. ibm.com/services/datamobility Produced in the United States of America December 2009 All Rights Reserved IBM, the IBM logo, ibm.com and TDMF are trademarks or registered trademarks of International Business Machines Corporation in the United States, other countries, or both. If these and other IBM trademarked terms are marked on their first occurrence in this information with a trademark symbol (® or ™), these symbols indicate U.S. registered or common law trademarks owned by IBM at the time this information was published. Such trademarks may also be registered or common law trademarks in other countries. A current list of IBM trademarks is available on the Web at “Copyright and trademark information” at ibm.com/legal/copytrade.shtml Intel and Itanium are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States and other countries. UNIX is a registered trademark of The Open Group in the United States and other countries. Other product, company or service names may be trademarks or service marks of others. References in this publication to IBM products or services do not imply that IBM intends to make them available in all countries in which IBM operates. 1 Based on testing performed in an IBM lab SDW03013-USEN-00