This is the paper to go with the Migrating from Single Instance to RAC Data guard

1. Database
MIGRATING FROM SINGLE INSTANCE TO RAC AND UPGRADING: A DATA GUARD
SUCCESS STORY
Fuad Arshad, Walgreens Company
INTRODUCTION
• Oracle has provided many methods of migrating and upgrading their database systems. These migration methods include
Basic migration methods like CTAS, Export/Import, Datapump, Streams, GoldenGate and Data Guard. The purpose of
this paper is to discuss the use of Data Guard to migrate from a single instance to a RAC Database and explain the
rationale behind this approach. This Paper will also talk about what the benefits we found in designing the architecture
and infrastructure and how we were able to successfully migrate to new servers with only 1 hour of downtime . I will
also discuss the challenge of migrating to the newest patchset with the software stack we were using.
OLD INFRASTRUCTURE:
• The old infrastructure consisted of Active/Passive Nodes using Veritas Clustering to provide Failover capabilities
• The old infrastructure did not allow us to scale and we had maxed out both CPU and memory capacity in the frame
• The old infrastructure could not connect to newer Disk Frames due to antiquated HBA’s
• The old infrastructure was running Oracle 10.2.0.4 with Solaris 9
• There was an idle node that was not being used due to the Active/Passive design of the environment
• Due to the application design we have a Logical Standby that was configured for reporting purposes only
THE PROBLEM
• The application being an Ecommerce application was experiencing scalability issues during busy times with high I/O
Wait times as well as application concurrency issues. The database server was maxing out of CPU which in turn caused
the application to bottleneck. Various application tuning attempts yielded minimal gain since this was a 3rd party
application with customized code
• Application team expected higher concurrency and transaction growth rates due to newer technologies being deployed at
the application's front end
• The cost and timelines to upgrade the current infrastructure would have caused us to miss the holiday season and thus
lose significant revenue
• Our application team needed a plan that would have the least amount of downtime with the least amount of money being
spent
• The Platform had to be scalable to meet our future needs
• The Platform had to be certified by the 3rd Party application we are using
DESIGN GOALS:
• Scalable architecture that can support our business growth
• Minimal code changes
• Needed to support our business and be available in time for the holiday season
1 371

2. Database
ARCHITECTURE BEFORE MIGRATION
The Pre-migration architecture consisted of
• 2 Sun Servers utilizing Veritas Cluster services for Active/Passive Failover running Solaris 9
• Oracle Database 10.2.0.4
• EMC SAN about 6 Years Old
The Pre-migration architecture consisted of hardware that had outgrown the business and a model that required downtime for
any changes, which was not beneficial to the business. The need to be highly available and provide a model to be able to
patch with minimal downtime as well as load balance the environment and utilize all hardware on the floor caused us to look
for better strategies to help our business.
ENVIRONMENT FOR MIGRATION
As part of the design process, and to meet design goals, the team worked towards Oracle RAC as the technology/scalability
platform for the application. This technology was selected after careful consideration since it was certified by the ATG
Product team, (ATG is Art Technology Group, now part of Oracle) as well as verifying that it made sense as the technology
platform to provide uptime and economies of scale.
The new environment selected was:
• 3 Oracle SPARC Enterprise M5000 servers running Solaris 10
• Oracle Grid Infrastructure 11.2.0.2 Oracle RAC 11.2.0.2
• EMC VMAX SAN
The environment was sized for an N+1 configuration which allowed the application to function without any issues in the case
of 1 node being out of service. The capacity planning exercise involved various tools from various vendors including AWR
reports to assess peak and nonpeak capacity models.
MIGRATION PREPARATION
The migration goals were simple. We needed a platform to migrate to with very little downtime. To do this task the team
started working on strategies that would allow us to provide not only a migration platform but a pre-migration live test.
The technologies that were considered were:
• Disk Based Replication
Disk Based replication would work since it was between 2 similar frames but we were moving from single
instance to RAC and that would have caused complications
• Backup & Restore
Backup and restore was one of the options considered but this option was time consuming and the downtime
window would have been extensive and thus unacceptable to the business
• Data Guard
Oracle Data Guard was a technology that fit the bill of being able to not only provide for faster migration time
but be a repeatable process that would allow us to do a test run and perform pre-migration testing. As part of our
Data Guard strategy we looked at both Logical and physical Data Guard as a means to migrate. Logical was ruled
out due to the fact that our primary had objects without primary keys and some load operations would have
caused issues in using the Transient Logical Standby upgrade methodology. We had a logical standby attached to
2 371

3. Database
the single instance and our understanding of the implications made our decision to use physical standby much
easier.
BUILDING THE ENVIRONMENT
The Environment build was an interesting challenge. Since the goal was to use the newest version of Oracle Software at the
time i.e 11.2.0.2 . We hit a couple of interesting issues. Some important documents we used to build the environment were:
1. Operating Systems Requirement for various OS
2. RDA RAC Guide
3. Patch Set Updates for Oracle Products
4. RAC And Oracle Clusterware Starter Kit
5. Master Note for RAC
6. RAC check
These documents as well as the Oracle documentation were used to build the environment and to ensure the environment
was built to meet all Oracle requirements. During the build we hit a couple of bugs including the multicast bugs . The
Multicast bug explained (now fixed in 11.2.0.3) used a multicast address and created role separation issues between the
clusterware and the database as documented in the guide.
The Clusterware installation had to have Oracle 10.2 DB installation since that was needed for the migration. To get the
install working we followed Metalink Doc # 948456.1. There also is an issue during install where /usr/local/bin/ssh did
not exist so a symlink was needed to get the 10.2 software installed on Solaris.
The environment build process included design of the infrastructure from network design to physical hardware setup. This
involves very close coordination with the UNIX, Network and Database teams. The infrastructure build was followed by a
complete OS check to ensure that all RAC related "Best Practices" were met on the OS level. Some important "Best
Practices" that are also documented in the above references include:
1. udp_recv_hiwat & udp_xmit_hiwat must be adjusted to 65536 to account for interconnect traffic.
2. For Jumbo Frames the Network Switch and HBA cards must be able to support an MTU of 9000.
VALIDATING THE ENVIRONMENT
It is very important to ensure that the environment is validated and ready prior to migration. Oracle provides tools for
environment validation. A Cluster Verification Utility (CVU) as well as RDA can be used to validate the environment.
The Cluster Verification Utility can help in discovering the network interfaces and ensuring that the environment is fully
configured prior to the install. The Oracle 11gR2 (11.2.0.2+) Installer includes Pre-checks that invoke the CVU utility and
validate all components of the environment prior to the creating the cluster.
SOFTWARE INSTALLATION
After the environment has been validated it is time to install the software. Oracle 11gR2 is different from previous version in
installation as a lot of the checks that were manual in previous versions are incorporated into the installer, the binaries
structure has changed too. Oracle 11gR2 breaks the Oracle binaries into 2 distinct sets.
1. Oracle Clusterware or Grid Infrastructure Binaries
The Oracle Clusterware or Grid Infrastructure (GI) binaries are now separate and include the Clusterware as well as
the Automatic Storage Management (ASM) binaries. These binaries can be installed for RAC Installation as well as
for single instance installation where ASM and Oracle Restart are needed. These binaries can also be installed as a
separate user for job role separation.
2. Oracle Database
3 371

4. Database
The Oracle Database binaries are the standard binaries that are used for the operation and management of the Oracle
Database.
Oracle GI has specific issues at install time (11.2.0.2 only) which require a patch that can be applied at software install time
due to Multicast Issues. This issue is documented in MyOracleSupport (formerly Metalink) doc #1212703.1 and something
that is very frustrating to debug. The GI installation gets a little complicated if you create Job Role Separation by creating a
separate grid user and separating the clusterware install from the DB Install. MyOracleSupport has a lot of useful documents
that explain, in detail, how to setup the clusterware and what the caveats are for job role separation. Note #1376731.1 will
help here.
Once the install is completed a validation must to be done to ensure all components are installed correctly. Oracle provides an
audit tool called Raccheck. Raccheck can be downloaded at #1268927.1.
Raccheck is a very useful tool and should be used not only for validating the install but also for routine validation of the
environment.
The following is an example of the checks Oracle has incorporated in raccheck.
- Begin RACCHECK example output
- =============================================================
- Node name - test
- =============================================================
- Collecting - ASM DIsk I/O stats
- Collecting - ASM Disk Groups
- Collecting - ASM disk partnership imbalance
- Collecting - ASM diskgroup attributes
- Collecting - ASM initialization parameters
- Collecting - Active sessions load balance for test database
- Collecting - Archived Destination Status for test database
- Collecting - CONNECT Role Grantees for test database
- Collecting - Cluster Interconnect Config for test database
- Collecting - Data Files In Backup Mode for test database
- Collecting - Database Archive Destinations for test database
- Collecting - Database Component Status for test database
- Collecting - Database Files for test database
- Collecting - Database Instance Settings for test database
- Collecting - Database Parameters for test database
- Collecting - Database Properties for test database
- Collecting - Database Registry for test database
- Collecting - Database Sequences for test database
- Collecting - Database Undocumented Parameters for test database
- Collecting - Database Workload Services for test database
- Collecting - Dataguard Status for test database
- Collecting - Files Needing Media Recovery for test database
- Collecting - Files not opened by ASM
- Collecting - INVALID SYS and SYSTEM objects for test database
- Collecting - INVALID application objects for test database
4 371

5. Database
- Collecting - Invalid Java Objects for test database
- Collecting - Invalid Registry Components for test database
- Collecting - JVM Roles for test database
- Collecting - JVM Roles for test database
- Collecting - Log Sequence Numbers for test database
- Collecting - Objects Duplicated in SYS and SYSTEM Schema for test database
- Collecting - Percentage of asm disk Imbalance
- Collecting - Process for shipping Redo to standby for test database
- Collecting - Redo Log information for test database
- Collecting - Standby redo log creation status before switchover for test database
- Collecting - CRS active version
- Collecting - CRS oifcfg
- Collecting - CRS software version
- Collecting - CSS Reboot time
- Collecting - CSS diagwait
- Collecting - CSS disktimout
- Collecting - CSS miscount
- Collecting - Cluster interconnect (clusterware)
- Collecting - Clusterware OCR healthcheck
- Collecting - Clusterware Resource Status
- Collecting - Kernel parameters
- Collecting - Multipath configuration
- Collecting - Netstat for tcp and udp protocols
- Collecting - OS Packages
- Collecting - OS Patches
- Collecting - Shared memory segments
- Collecting - Solaris10 kernel parameters
- Collecting - Solaris9 kernel parameters
- Collecting - System configuration information
- Collecting - Table of file system defaults
- Collecting - Voting disks (clusterware)
The following is an Example of a sample raccheck output.
--
INFO => $CRS_HOME/log/hostname/client directory has too many older log files.
WARNING => Value of remote_listener parameter is not able to tnsping for test
WARNING => Value of remote_listener parameter is not able to tnsping for test
INFO => core_dump_dest has too many older core dump files for test
INFO => user_dump_dest has trace files older than 30 days for test
INFO => ORA-00600 errors found in alert log for test
INFO => ORA-07445 errors found in alert log for test
INFO => background_dump_dest has files older than 30 days for pwagdb
5 371

6. Database
INFO => Some tablespaces do not have allocation type as SYSTEM for test
WARNING => Some tablespaces are not using Automatic segment storage management. for test
--
THE MIGRATION
We have talked about getting the environment ready and now we are going to talk about the actual process of migration from
single instance to RAC using Data Guard. Since this migration was from the Oracle 10.2.0.4 database the first thing was to
install the 10.2.0.4 DB software on the new 11.2.0.2 RAC cluster.
Once the Software is successfully Follow these steps to get the environment up:
Pre Migration Steps
1. Add REDO threads to correspond to the number of instances in the new RAC cluster
2. Add Undo Tablespaces to correspond with the number of RAC instances you will have
3. Run “?/rdbms/admin/catclust.sql” on the single instance during a quiet time to get the RAC catalog views in place
4. Backup The Database as below
“RUN {
set command id to 'stdby_test';
allocate channel ch1 type disk format ='/usr/local/oracle/migration/standby_%U.bak';
allocate channel ch2 type disk format ='/usr/local/oracle/migration/standby_%U.bak';
BACKUP DATABASE PLUS ARCHIVELOG tag for_standby;
}
BACKUP DEVICE TYPE DISK FORMAT '/usr/local/oracle/migration/test_ctl%U' CURRENT CONTROLFILE FOR STANDBY”;
5. Configure Oracle tnsnames & sqlnet.ora on Standby & Primary to have both the primary and standby node
information on both
6. Setup the Standby init.ora with parameters from primary and add-ons as below
-- Setup the init file on the standby
-- copy base file from primary
-- modify the following
control_files
*.log_archive_config
*.log_archive_dest_2
*.db_file_name_convert
*.log_file_name_convert
*.standby_file_management=auto
*.fal_server
*.fal_client
*.service_names
*.cluster_database=true
*.db_unique_name
*.instance_name
*.instance_number
*.thread
*.REMOTE_LISTENER=
*.LOCAL_LISTENER
6 371

7. Database
7. On the New RAC node as the grid user $GRID_HOME/bin/setasmgidwrap
o=/opt/oracle/product/10.2.0/db/bin/oracle to get the Oracle binary to work with 11.2.0.2 clusterware
Migration Steps
1. Copy the backup to a location available by the new nodes
2. Restore the database as below
“$ rman target sys/oracle@pwagdb01 auxiliary /
DUPLICATE TARGET DATABASE FOR STANDBY;”
3. Once the duplicated database exists you can perform the setup to let the clusterware/OCR be aware of the database
“srvctl add database -d test -o /usr/local/oracle/product/10.2 -r PHYSICAL_STANDBY” -- Let the Clusterware
know that this is physical Standby
“srvctl add instance -d test –i test1 -n testdb01” -- Add node 1 instance
“srvctl add instance -d test –i test2 -n testdb02” -- Add node 2 instance
“srvctl modify instance -d test -i test1 -s +ASM1” – Add node 1 ASM as a dependency
“srvctl modify instance -d test -i test2 -s +ASM2” -- Add Node 2 ASM as a dependency
4. Add Standby Redo Logs that should be identical to the online redo log sizes that the primary database has. The
standby redo logs can be created prior to the duplicate as well as after and ae needed for real time apply to function
5. Run “ALTER DATABASE RECOVER MANAGED STANDBY DATABASE USING CURRENT LOGFILE
DISCONNECT” on instance 1 of the new RAC DB to start the physical Standby apply
At this point we have a complete running Standby Database on the new RAC Cluster. The Database is 10.2.0.4 and is
in complete sync with the primary. The Data Guard Broker should be used to manage and maintain this database and
the primary single instance database is being synced with the new RAC clustered physical Standby. After this point
the upgrade portion of the exercise begins. In our scenario we chose to take downtime since there was application
configuration that was needed. So the following was the upgrade scenario.
Upgrade Steps.
1. On The Primary and standby ensure all redo logs have been shipped
2. Put the database in restrict and bring the application
3. Bounce the database to ensure all app connections are completed and put it in restricted mode. Switch a couple of
redo logs and then defer the redo log apply
4. Ensure on the Standby database all received archivelogs are applied
5. Stop Standby redo apply “ALTER DATABASE RECOVER MANAGED STANDBY DATABASE CANCEL;”
6. Cleanup old Archive Logs
7. Activate Standby Database “Alter database activate standby database;”
8. Open The Database
9. Shutdown the Database to enable flashback.
10. Remove any obsolete parameters identified by utl112i.sql
11. Create a restore point
12. Open the Database
13. Run DBUA to upgrade the Database
7 371

8. Database
14. Validate The Upgrade
15. GO Live
While the steps look tedious if well planned and choreographed, the total upgrade time can be 1-3 hrs. This in our case
coincided with the application changes required to support the new architecture including the addition of services to load
balance the application architecture.
The upgrade was completed, the move to infrastructure was completed without any hitches and our Go-Live from single
instance using RAC was a success.
REFERENCES AND FURTHER READING
The references are provided courtesy of Oracle RAC Assurance group
Oracle RAC Assurance Group is a team within Oracle whose goal is to provide
input to customers with new and existing CRS,RAC,ASM and MAA
implementation
– References
22.1 Notes for 'My Oracle Support' (MOS) (The New MetaLink):
o Doc ID 736737.1 My Oracle Support - The Next Generation Support Platform
o Doc ID 730283.1 Get the most out of My Oracle Support
o Doc ID 747242.1 My Oracle Support Configuration Management FAQ
22.2 Process-Oriented and Self-Service MOS Notes:
o Doc ID 374370.1 New Customers Start Here
o Doc ID 166650.1 Working Effectively With Global Customer Support
o Doc ID 199389.1 Escalating Service Requests with Oracle Support Services
22.3 RAC Master Note:
o Doc ID 1096952.1 Master Note for RAC, Oracle Clusterware and Oracle Grid Infrastructure
22.4 RAC Installation Notes:
The process used to install, configure, and create an Oracle Real Application Cluster (RAC) database shares much in
common shares much in common all supported operating system platforms. Despite these commonalities, many aspects
of the deployment process are O/S specific. All RAC deployments configure shared storage resources, network
resources, and software services that enable RAC node participation, operation and communication.
o Doc ID 169706.1 Oracle Database on AIX,HP-UX,Linux,MacOSX,Solaris,Tru64 Operating Systems
Installation and Configuration Requirements Quick Reference (8.0.5 to 11.1)
o Doc ID 298891.1 Configuring Linux for the Oracle 10g VIP or private interconnect using bonding driver
o Doc ID 296874.1Configuring the HP-UX Operating System for the Oracle 10g VIP
o Doc ID 361323.1 HugePages on Linux: What It Is... and What It Is Not...
o Doc ID 401132.1 How to install Oracle Clusterware with shared storage on block devices
o Doc ID 357472.1 Configuring device-mapper for CRS/ASM
o Doc ID 332257.1 Using Oracle Clusterware with Vendor Clusterware FAQ
o Doc ID 397460.1 Oracle's Policy for Supporting RAC 10g with Symantec SFRAC
o Doc ID 790189.1 Oracle Clusterware and Application Failover Management
o Doc ID 391771.1 OCFS2 - Frequently Asked Questions
o Doc ID 359515.1 Mount Options for Oracle files when used with NAS devices
8 371

9. Database
o Doc ID 759565.1 Turning NUMA on can cause database hangs
o Doc ID 220970.1 RAC: Frequently Asked Questions (9.2 through 11.2)
o Click here. Automatic Workload Management with Oracle Real Application Clusters (FAN/FCF).
o Click here. Oracle Clusterware 11g Release 2
22.5 RAC Assurance RAC Starter Kits
The goal of the Oracle Real Application Clusters (RAC) Starter Kit is to provide you with the latest information on
generic and platform specific best practices for implementing an Oracle RAC cluster. This document is compiled and
provided based on Oracle's experience with its global RAC customer base.
o Doc ID 810394.1 RAC Assurance Support Team: RAC Starter Kit and Best Practices (Generic)
o Doc ID 811306.1 RAC Assurance Support Team RAC Starter Kit (Linux)
o Doc ID 811280.1 RAC Assurance Support Team RAC Starter Kit (Solaris)
o Doc ID 811271.1 RAC Assurance Support Team RAC Starter Kit (Windows)
Oracle Global Customer Support, RAC Assurance Walgreens Page 156
9 371

10. Database
o Doc ID 811293.1 RAC Assurance Support Team RAC Starter Kit (AIX)
o Doc ID 811303.1 RAC Assurance Support Team RAC Starter Kit (HP-UX)
22.6 RAC Troubleshooting, Data Collection and Monitoring Notes:
To effectively manage your RAC cluster, it is important to know how to monitor, troubleshoot and collect data to
support a service request.
o Doc ID 556679.1 Data Gathering for Troubleshooting RAC Issues
o Doc ID 289690.1 Data Gathering for Troubleshooting RAC and CRS issues
o Doc ID 301137.1 OS Watcher User Guide
o Doc ID 459694.1 Procwatcher: Script to Monitor and Examine Oracle and CRS
o Doc ID 726446.1 What Is The Difference Between RDA and Configuration Manager
o Doc ID 316900.1 Oracle 10g Release 2 (10.2) Support Status and Alerts
o Doc ID 454507.1 ALERT Oracle 11g Release 1 (11.1) Support Status and Alerts
o Doc ID 559365.1 Using Diagwait as a Diagnostic.
o Doc ID 556679.1 Data Gathering for Troubleshooting RAC Issues
22.7 RAC Testing and Performance Tuning
Seventy to eighty percent of all RAC performance tuning is “single instance” tuning. The other twenty to thirty percent
of tuning has to do with gcs, ges, UDP and inter-instance tuning.
o Doc ID 390374.1 Oracle Performance Diagnostic Guide (OPDG)
o Doc ID 181489.1 Tuning Inter-Instance Performance in RAC and OPS
o Doc ID 563566.1 gc lost blocks diagnostics
o Doc ID 280939.1 Checklist for Performance Problems with Parallel Execution
22.8 RMAN with RAC and Data Guard
o Doc ID 311068.1 RMAN Performance Tuning Diagnostics
o Click here. Using RMAN with DG in 10g Database
22.9 RAC One Node
New with Oracle Database 11g Release 2, Oracle Real Application Clusters One Node (Oracle RAC One Node) is a one-
node version of Oracle RAC which enables customers to standardize on a single deployment model for all their database
needs. It offers far greater benefits than traditional server virtualization to single instance databases running on a
hardware cluster.
o Click here. Oracle RAC One Node 11g Release 2 User Guide
22.10 RAC Maintenance Notes:
o Doc ID 428681.1 How to ADD/REMOVE/REPLACE/MOVE Oracle Cluster Registry (OCR) and Voting Disk
o Doc ID 363254.1Applying one-off Oracle Clusterware patches in a mixed version home environment
o Doc ID 399031.1 Step by step installation of Oracle Clusterware one-off and bundle patches for 10.1 and 10.2
o Doc ID 209768.1 Database, FMW, EM Grid Control and OCS Software Error Correction Support Policy
o Doc ID 429855.1 CRSCTL STOP CRS issues Shutdown Abort in both ASM and database instances
o Doc ID 4598992.8 Bug 4598992:"Action script for resource 'ora.xxx.vip' stdout redirection" errors in crsd.log
o Doc ID 563905.1 Implementing LIBUMEM for CRS on Solaris 64 with 3rd Party Clusterware
o Doc ID 92602.1 How to Password Protect the Listener
o Doc ID 403743.1 VIP Failover Take Long Time After Network Cable Pulled
o Doc ID 359515.1 Mount Options for Oracle files when used with NAS devices
o Doc ID 561414.1 Transactional Sequences in Applications in a RAC environment
o Doc ID 395314.1 RAC Hangs due to small cache size on SYS.AUDSES$
o Doc ID 563566.1 gc lost blocks diagnostics
Oracle Global Customer Support, RAC Assurance Walgreens Page 157
10 371

11. Database
o Doc ID 949322.1 Oracle11g Data Guard: Database Rolling Upgrade Shell Script
o Doc ID 1053147.1 11gR2 Clusterware and Grid Home - What You Need to Know
o Doc ID 742060.1 Release Schedule of Current Database Releases
o Doc ID 276434.1 Modifying the VIP or VIP Hostname of a 10g or 11g Oracle Clusterware Node
o Doc ID 219361.1 Troubleshooting ORA-29740 in a RAC Environment
o Doc ID 226880.1 Configuration of Load Balancing and Transparent Application Failover
o Doc ID 864633.1 Enable Oracle NUMA support with Oracle Server Version 11.2.0.1
o Click here. RAC on IBM AIX Best practices in memory tuning and configuring for system stability
o Click here. XA and Oracle controlled Distributed Transactions
22.11 Upgrade and Platform Migration for RAC
Migrate a RAC cluster to a new platform or upgrade a RAC cluster from an older version of Oracle to the current
version of Oracle can be relatively simple if you know the correct techniques. The right planning, preparation and
upgrade steps will make the upgrade process simpler, faster and more predictable from start to finish.
o Doc ID 785351.1 Oracle 11gR2 Upgrade Companion
o Doc ID 601807.1 Oracle 11gR1 Upgrade Companion
o Doc ID 429825.1 Complete Checklist for Manual Upgrades to 11gR1.
o Doc ID 338706.1 Cluster Ready Services (CRS) rolling upgrade
o Click here. Methods for Upgrading to Oracle Database 11g Release 2
o Click here. Database Rolling Upgrades Made Easy by Using a Data Guard Physical Standby Database
o Click here. Database Rolling Upgrade Using Transient Logical Standby: Oracle Data Guard 11g
o Click here. Database Rolling Upgrade Using Transient Logical Standby Oracle Database 10gR2
o Click here. Database Upgrade Guide 11gR2
o Click here. Methods for Upgrading to Oracle Database 11g Release 2
o Click here. Database Rolling Upgrades Made Easy by Using a Data Guard Physical Standby Database
o Click here. Database Upgrade Using Transportable Tablespaces: Oracle Database 11g Release 1
o Click here. Platform Migration Using Transportable Database Oracle Database 11g and 10g Release 2
22.12 RAC ASM Documentation
Automatic Storage Management (ASM) is an evolution in file system and volume management functionality for Oracle
database files. ASM further enhances automation and simplicity in storage management that is critical to the success of
the Oracle grid architecture. ASM also improves file system scalability and performance, manageability and databases
availability for RAC environments. Use of ASM with RAC is an Oracle Best Practice.
o Doc ID 888017.1 ACFS/ADVM Support Information for 11gR2
o Doc ID 885363.1 What diagnostic information to collect for ADVM/ACFS related issues
o Doc ID 371556.1 How move tablespaces across platforms using Transportable Tablespaces with RMAN.
o Doc ID 370921.1 ASM - Scalability and Limits
o Doc ID 6453944.8 Bug 6453944 - ORA-15196 with ASM disks larger than 2TB
o Doc ID 294869.1 Oracle ASM and Multi-Pathing Technologies.
o Doc ID 30286.1 I/O Tuning with Different RAID Configurations
o Click here. Oracle Database 11g R2 Extending ASM to Manage All Data
o Click here. ASM and Multipathing Generic Best Practices and Information Matrix
22.13 RAC - Windows Notes:
o Doc ID 811151.1 How to install Oracle Cluster Health Monitor (former IPD/OS) on Windows.
Oracle Global Customer Support, RAC Assurance Walgreens Page 158
11 371

12. Database
o Doc ID 810915.1 How to Monitor, Detect and Analyze OS and RAC Resource Related Degradation and Failures on
Windows.
o Doc ID 433472.1 OS Watcher For Windows (OSWFW) User Guide.
22.14 OCFS2 Documentation:
o Doc ID 603080.1 Linux OCFS2 - Best Practices.
o Doc ID 457423.1 OCFS2 Fencing, Network, and Disk Heartbeat Timeout Configuration
o Doc ID 434255.1 Common reasons for OCFS2 Kernel Panic or Reboot Issues.
o Doc ID 565574.1 How to Fix the IO Errors Reading an Oracle Datafile On an OCFS2 Filesystem.
o Doc ID 395878.1 Heartbeat/Voting/Quorum Related Timeout Configuration for Linux, OCFS2, RAC Stack to avoid
unnecessary node fencing, panic and reboot.
o Doc ID 468923.1 OCFS2: Disk Space is not Released After Deleting Many Files
o Doc ID 224586.1 FAQ: Oracle Cluster File System (OCFS) for 2.4 Kernels
22.15 MAA/Standby Documentation:
Data Guard provides the management, monitoring, and automation software infrastructure to create and maintain one
or more standby databases to protect Oracle data from failures, disasters, errors, and data corruptions. As users commit
transactions at a primary database, Oracle generates redo records and writes them to a local online log file.
o Doc ID 380449.1 Creating a RAC Physical Standby for a RAC Primary
o Doc ID 239100.1 Data Guard Protection Modes
o Doc ID 275977.1 Data Guard Broker High Availability
o Doc ID 1085687.1 Data Guard Support for Heterogeneous Primary and Logical Standbys in Same Data Guard
Configuration
o Doc ID 454848.1 Installing and Using Standby Statspack in 11gR1. Unless a query was run on the Primary it will not
be in the stats. You cannot run AWR on a standby. You can configure Standby Statspack.
o Doc ID 729551.1 Redo Transport Compression in a Data Guard Environment
o Doc ID 312434.1Oracle10g Data Guard SQL Apply Troubleshooting
o Doc ID 1265884.1Resolving ORA-752 or ORA-600 [3020] During Standby Recovery
o Doc ID 565535.1Flashback Database Best Practices & Performance
o Doc ID 387450.1 MAA - SQL Apply Best Practices 10gR2
o Doc ID 273015.1Migrating to RAC using Data Guard
o Doc ID 413484.1 Data Guard Support for Heterogeneous Primary and Standby Systems in Same Data Guard
Configuration
o Doc ID 414043.1 Role Transitions for Data Guard Configurations Using Mixed Oracle Binaries
o Doc ID 751600.110.2 Data Guard Physical Standby Switchover
o Doc ID 753893.1Degraded performance on standby due to waits on controlfile enqueue
o Doc ID 459411.1 Steps to recreate a Physical Standby Controlfile
o Doc ID 858975.1 How To Create Standby Control File Placed In A Raw Device
o Doc ID 949322.1 Oracle11g Data Guard: Database Rolling Upgrade Shell Script
o Doc ID 736755.1 How To Calculate The Required Network Bandwidth Transfer Of Archivelogs In Dataguard
Environments
o Doc ID 278641.1 How do you apply a Patchset, PSU or CPU in a Data Guard Physical Standby configuration
o Click here. Database Rolling Upgrade Using Transient Logical Standby: Oracle Data Guard 11g
o Click here. Database Rolling Upgrades Made Easy Using a Data Guard Physical Standby Database
o Click here. Installation Companion Oracle Data Guard on Amazon EC2
o Click here. Automatic Workload Management with Oracle Real Application Clusters
o Click here. Client Failover Best Practices for HA Oracle Databases: Oracle Database 11gR2
Oracle Global Customer Support, RAC Assurance Walgreens Page 159
12 371

13. Database
o Click here. Rapid Oracle RAC Standby Deployment: Oracle Database 11g Release 2
o Click here. Oracle Active Data Guard Oracle Data Guard 11g Release 1
o Click here. Oracle Data Guard: Disaster Recovery for Sun Oracle Database Machine.
o Click here. Rapid Oracle RAC Standby Deployment: Oracle Database 11g Release 2
o Click here. Platform Migration Using Transportable Database Oracle Database 11g and 10g Rel 2
o Click here. Switchover and Failover Best Practices: Oracle Data Guard 10g Release 2
o Click here. Client Failover Best Practices for Highly Available Databases: Oracle Database 10gR2
o Click here. Oracle Active Data Guard Oracle Data Guard 11g Release1
o Click here. Configuring Oracle BI EE Server with Oracle Active Data Guard
o Click here. Fast-Start Failover Best Practices: Data Guard 10gR2.
o Click here. Data Guard Redo Apply and Media Recovery Best Practices Oracle Database 10gR2
o Click here. Switchover and Failover Best Practices: Oracle Data Guard 10g Release 2
o Click here. SQL Apply Best Practices: Oracle Data Guard 11g Release 1
o Click here. SQL Apply Best Practices: Oracle Data Guard 10g Release 2.
o Click here. Rapid Oracle RAC Standby Deployment: Oracle Database 11g Release 2
22.16 Cloning
o Click here. DB Cloning Solution Using Oracle's Sun ZFS Storage Appliance And Oracle Data Guard
o Click here. Cloning an Oracle Database to the Same Server Using Snapshot and Volume Copy
22.17 Exadata
o Doc ID 888828.1 Database Machine and Exadata Storage Server 11g Release 2 (11.2) Supported Versions
o Click here. Oracle Exadata Tips, Tricks, and Best Practices: Backup and Recovery (S316821)
o Click here. Oracle Data Guard: Disaster Recovery for Sun Oracle Database Machine
o Click here. Migrating Oracle E-Business Suite to Sun Oracle Database Machine Using Oracle Data Pump
o Click here. Tape Backup Performance and Best Practices for Exadata Storage and the HP Oracle Database Machine
o Click here. Oracle Data Guard: Disaster Recovery for Sun Oracle Database Machine
22.18 RAC and PeopleSoft
o Click here. Deploying a PeopleSoft Maximum Availability Architecture
o Click here. Reducing PeopleSoft Downtime Using a Local Standby Database
22.19 RAC and SAP
o Click here. Providing High Availability for SAP Resources with Oracle Clusterware 11 Release 2
o Click here. Moving your SAP database to Oracle 11gR2 ASM Best Practices
22.20 RAC and Oracle E-Business
o Doc ID 362135.1 Configuring Oracle Applications Release 11i with Oracle10g Release 2 Real Application Clusters and
Automatic Storage Management.
o Doc ID 823587.1 Using Oracle 11gR2 RAC with Oracle E-Business Suite Release 12
o Doc ID 380489.1 Using Load-Balancers with Oracle E-Business Suite Release 12
o Doc ID 727171.1 Implementing Load Balancing On Oracle E-Business Suite - Documentation For Specific Load
Balancer Hardware
o Doc ID 217368.1 Advanced Configurations and Topologies for Enterprise Deployments of E-Business Suite 11i
o Doc ID 403347.1 MAA Roadmap for the E-Business Suite
Oracle Global Customer Support, RAC Assurance Walgreens Page 160
13 371

14. Database
o Doc ID 388577.1 Using Oracle 10gR2 RAC and Automatic Storage Management with Oracle E-Business Suite Release
12
o Doc ID 455398.1 Using Oracle Real Application Clusters and Automatic Storage Management with Oracle E-Business
Suite Release 11i and Oracle Database 11g
o Doc ID 466649.1 Using Oracle 11g Release 1 (11.1.0.7) Real Application Clusters and Automatic Storage Management
with Oracle E-Business Suite Release 12
o Doc ID 762024.1 How To Ensure Load Balancing Of Concurrent Manager Processes In PCP-RAC Configuration
o Doc ID 241370.1 Concurrent Manager Setup and Configuration Requirements in an 11i RAC Environment
o Doc ID 1057802.1 Best Practices for Performance for Concurrent Managers in E-Business Suite
o Doc ID 552028.1 How to Ensure That Source Nodes Are Not Used on Failover From RAC to Single Tier Dataguard
Environment
o Doc ID 271090.1 Parallel Concurrent Processing Failover/Failback Expectations
o –Click here. Maximum Availability Architecture(MAA):Oracle E-Business Suite Release 12
22.21 Patching
As part of an overall maintenance strategy, it is critical that customers have a formal strategy to stay in front of known
issues and bugs. To make it easier for customers to obtain and deploy fixes for known critical issues we have created.
o Doc ID 1082394.1 11.2.0.X Grid Infrastructure PSU Information
o Doc ID 438314.1 Critical Patch Update - Introduction to Database n-Apply CPUs.
o Doc ID 756671.1 Oracle Recommended Patches -- Oracle Database.
o Doc ID 850471.1 Oracle Announces First Patch Set Update For Oracle Database Release 10.2.
o Doc ID 761111.1 Online Patches (Hot Patching).
o Doc ID 405820.1 10.2.0.X CRS Bundle Patch Information.
o Doc ID 844983.1 Apply CRS Bundle Patch Using opatch auto Option.
o Doc ID 850471.1 Oracle Announces First Patch Set Update For Oracle Database Release 10.2
22.22 Weblogic and RAC
o Doc ID 1086009.1 Grid Control 11g Fusion Middleware and Weblogic Server Management New Features
22.23 Websphere and RAC
o Click here. Using Oracle Real Application Cluster (RAC) with WebSphere Process Server.
o Click here. Enabling Oracle pooling in WebSphere Application Server V6.1.
22.24 RAC 11gR2 Documentation:
o Click here. 11gR2 Documentation Library.
o Click here. 2 Day + Real Application Clusters Guide E10743-02.
o Click here. New Features Guide E10881-03.
o Click here. Upgrade Guide E10819-02.
o Click here. High Availability Overview E10804-03.
o Click here. Real Application Clusters Administration and Deployment Guide E10718-08.
o Click here. Oracle Clusterware Administration and Deployment Guide E10717-08.
o Click here. (ASM) Storage Administrator's Guide E10500-04.
o Click here. Real Application Testing User’s Guide E12254-02.
o Click here. Oracle Data Guard Concepts and Administration E10700-02.
o Click here. Oracle Data Guard Broker E10702-02.
Oracle Global Customer Support, RAC Assurance Walgreens Page 161
14 371

15. Database
o Click here. Event notification and database connection failover support that is available to database clients when a
broker-managed failover occurs.
o Click here. Oracle RAC on Extended Distance Clusters
22.25 RAC 11gR1 Documentation:
o Click here. 11gR1 Documentation Library.
o Click here. 2 Day + Real Application Clusters Guide B28252-03.
o Click here. New Features Guide B28279-03.
o Click here. Oracle Real Application Clusters Administration and Deployment Guide B28254-07.
o Click here. Oracle Clusterware administration and Deployment Guide B28255-06.
o Click here. Oracle Data Guard Concepts and Administration B28294-03.
o Click here. Oracle Data Guard Broker B28295-03.
o Click here. (ASM) Storage Administrator's Guide B31107-05.
o Click here. Real Application Testing User’s Guide E12253-01.
o Click here. Upgrade Guide B28300-03.
15 371