Maa goldengate-rac-2007111

An Oracle White Paper
August 2013
Oracle Goldengate With Oracle Real
Application Clusters Configuration

Oracle Maximum Availability Architecture Oracle GoldenGate with Oracle Real Application Clusters Configuration
Executive Overview........................................................................... 2
Configuration Overview ..................................................................... 3
Oracle GoldenGate........................................................................ 3
Oracle Real Application Clusters (RAC) ........................................ 3
Oracle Clusterware........................................................................ 4
Oracle Database File System ........................................................ 4
Configuration Best Practices ............................................................. 5
Step 1: Set Up DBFS on Oracle RAC............................................ 5
Step 2: Install Oracle GoldenGate ............................................... 10
Step 3: Configure Oracle GoldenGate and Database Parameters10
Step 4: Set Up Checkpoint Files and Trail Files in DBFS............. 12
Step 5: Configure Replicat Commit Behavior............................... 14
Step 6: Configure Autostart of Extract, Data Pump, and Replicat Processes
.................................................................................................... 14
Step 7: Oracle Clusterware Configuration.................................... 15
Appendix A: Creating Oracle GoldenGate Clusterware Resource ... 20
Recommendations When Deploying on Oracle RAC ................... 23
Appendix B: Example Agent Script.................................................. 24
References...................................................................................... 30

Executive Overview
Oracle Real Application Clusters (Oracle RAC) and Oracle Clusterware allow Oracle Database
to run any packaged or custom application across a set of clustered servers. This capability
provides the best availability for node and instance failures and most planned maintenance
activities, and the most flexible scalability. If a clustered node fails, the Oracle database
continues running on the surviving nodes. When more processing power is needed, another
node can be added without interrupting user access to data.
Oracle Clusterware is a cluster manager that is designed specifically for the Oracle database.
In an Oracle RAC environment, Oracle Clusterware monitors all Oracle resources (such as
database instances and listeners). If a failure occurs, then Oracle Clusterware automatically
attempts to restart the failed resource. During outages, Oracle Clusterware relocates the
processing performed by the inoperative resource to a backup resource. For example, if a
node fails, then Oracle Clusterware relocates the database services being used by the
application to a surviving node in the cluster.
This white paper describes best practices for configuring Oracle GoldenGate to work with
Oracle RAC, Oracle Clusterware and Oracle Database File System (DBFS). Oracle
GoldenGate is instrumental for many reasons, including the following:
 To migrate to an Oracle Database, incurring minimal downtime
 As part of an application architecture that requires Oracle RAC plus the flexible availability
features provided by Oracle GoldenGate, such as active-active database for data distribution
and continuous availability, and zero or minimal downtime during planned outages for
system migrations, upgrades, and maintenance
 To implement a near real-time data warehouse or consolidated database on Oracle RAC,
sourced from various, possibly heterogeneous source databases, populated by Oracle
GoldenGate
 To capture from an OLTP application running on Oracle RAC to support further downstream
consumption such as a SOA type integration

3
This paper focuses on configuring Oracle GoldenGate to run on Oracle RAC, which can act as
the source database, as the target database, or in some cases as both source and target
databases for Oracle GoldenGate processing.
Configuration Overview
This section introduces Oracle GoldenGate, Oracle RAC, Oracle Clusterware, and Oracle Database
File System (DBFS). For more information about these features, see the References section at the end
of this white paper.
Oracle GoldenGate
Oracle GoldenGate provides real-time, log-based change data capture and delivery between
heterogeneous systems. This technology enables you to construct a cost-effective and low-impact real-
time data integration and continuous availability solution.
Oracle GoldenGate moves committed transactions with transaction integrity and minimal overhead on
your existing infrastructure. The architecture supports multiple data replication topologies such as one-
to-many, many-to-many, cascading, and bidirectional. Its wide variety of use cases includes real-time
business intelligence; query offloading; zero-downtime upgrades and migrations; and active-active
databases for data distribution, data synchronization, and high availability. Figure 1 shows the Oracle
GoldenGate architecture.
Figure 1. Oracle GoldenGate Architecture
Oracle Real Application Clusters
Oracle Real Application Clusters (Oracle RAC) enables multiple instances that are linked
by an interconnect to share access to an Oracle database. In an Oracle RAC environment,
Oracle Database runs on two or more systems in a cluster while concurrently accessing a

4
single shared database. The result is a single database system that spans multiple hardware
systems, enabling Oracle RAC to provide high availability and redundancy during failures
in the cluster. Oracle RAC accommodates all system types, from read-only data
warehouse systems to update-intensive online transaction processing (OLTP) systems.
Oracle Clusterware
Oracle Clusterware enables servers to communicate with each other, so that they appear to function as
a collective unit. This combination of servers is commonly known as a cluster. Although the servers are
standalone servers, each server has additional processes that communicate with other servers. In this
way the separate servers appear as if they are one system to applications and end users.
Oracle Clusterware provides the infrastructure necessary to run Oracle RAC. Oracle Clusterware also
manages resources, such as virtual IP (VIP) addresses, databases, listeners, services, and so on.
There are APIs to register an application and instruct Oracle Clusterware regarding the way an
application is managed in a clustered environment. You use the APIs to register the Oracle
GoldenGate Manager process as an application managed through Oracle Clusterware. The Manager
process should then be configured to automatically start or restart other Oracle GoldenGate processes.
Oracle Database File System
The Oracle Database File System (DBFS) creates a file system interface to files stored in the database.
DBFS is similar to NFS in that it provides a shared network file system that looks like a local file
system. Because the data is stored in the database, the file system inherits all the high availability and
disaster recovery capabilities provided by the database.
With DBFS, the server is the Oracle Database. Files are stored as SecureFiles LOBs. PL/SQL
procedures implement file system access primitives such as create, open, read, write, and list directory.
The implementation of the file system in the database is called the DBFS SecureFiles Store. The DBFS
SecureFiles Store allows users to create file systems that can be mounted by clients. Each file system
has its own dedicated tables that hold the file system content.

5
Configuration Best Practices
Step 1: Set Up DBFS on Oracle RAC
When setting up the configuration, the best practice is to store the Oracle GoldenGate trail files,
checkpoint files, bounded recovery and configuration files in DBFS to provide the best performance,
scalability, recoverability, and failover capabilities in the event of a system failure.
Using DBFS is fundamental to the continuing availability of the checkpoint and trail files in the event
of a node failure. Ensuring the availability of the checkpoint files clusterwide is essential to ensure that
after a failure occurs the Extract process can continue mining from the last known archived redo log
file position, and Replicat processes can start applying from the same trail file position before a failure
occurred. Using DBFS allows one of the surviving database instances to be the source of an
Extract/Data Pump processes or a destination for the Replicat processes.
It is recommended to run the DBFS database in ARCHIVELOG mode, so that recoverability is not
compromised in the event of media failures or corruptions.
Follow instructions in My Oracle Support note 869822.1 to install the required FUSE libraries. And
use the instructions in My Oracle Support note 1054431.1 to configure the DBFS database, required
users, and permissions on source or target environments. My Oracle Support note 1054431.1 also
provides detailed instructions for configuring DBFS with Oracle Clusterware which is covered later in
this paper.
Source Environment (Extract/Data Pump)
Create a single file system for storing the Oracle GoldenGate trail files, checkpoint files, bounded
recovery files, temp files, discard files, and parameter files.
It is recommended that you allocate enough trail file disk space to permit storage of up to 12 hours of
trail files. Doing this will give sufficient space to Extract trail file generation should a problem occur
with the target environment that prevents it from receiving new trail files. The amount of space needed
for 12 hours can only be determined by testing trail file generation rates with real production data.
Create the DBFS tablespace:
-- Connect to the DBFS database
SQL> connect system/<passwd>@<source_dbfs_tns_alias>
-- Create the tablespace:
SQL> create bigfile tablespace dbfs_gg_source_tbs datafile ‘+DBFS_DG’ size
200g autoextend on next 8g maxsize 400g LOGGING EXTENT MANAGEMENT LOCAL
AUTOALLOCATE SEGMENT SPACE MANAGEMENT AUTO;
Substitute the size parameters with your required trail file storage size.

6
Create the file system:
% cd $ORACLE_HOME/rdbms/admin
% sqlplus dbfs_user/dbfs_password
SQL> start dbfs_create_filesystem dbfs_gg_source_tbs gg_source
The LOB segment used by DBFS should be configured with the storage options NOCACHE
LOGGING which is the default:
SQL> connect system/<passwd>@<dbfs_tns_alias>
-- View current LOB storage:
SQL> SELECT table_name, segment_name, logging, cache
FROM dba_lobs WHERE tablespace_name='DBFS_GG_SOURCE_TBS';
-- More than likely it will be something like this:
--
-- TABLE_NAME SEGMENT_NAME LOGGING CACHE
-- ------------------ ---------------------- ------- ----------
-- T_GOLDENGATE LOB_SFS$_FST_73 YES NO
If the LOB segment is not using NOCACHE LOGGING, alter it:
SQL> ALTER TABLE DBFS.<TABLE_NAME> MODIFY LOB (FILEDATA)
(NOCACHE LOGGING);
-- View the new LOB storage:
FROM dba_lobs WHERE tablespace_name='DBFS_GG_SOURCE_TBS';
-- ------------------ ---------------------- ------- ----------
-- T_GOLDENGATE LOB_SFS$_FST_73 YES NO
Starting with Oracle GoldenGate release 11.2.1 the dirtmp directory can be placed on DBFS. With
earlier releases of Oracle GoldenGate, the dirtmp directory was normally on the local file system or
another non-DBFS shared file system since data stored in dirtmp is transient and not required for any
Oracle GoldenGate startup. However, by placing dirtmp on DBFS, you get the additional benefit of
larger storage potential.
If dirtmp is on DBFS, use a NOCACHE NOLOGGING tablespace. For example:

7
Create the DBFS tablespace:
SQL> connect system/<passwd>@<source_dbfs_tns_alias>
-- Create the tablespace:
SQL> create bigfile tablespace dbfs_gg_dirtmp_tbs datafile ‘+DBFS_DG’ size
200g autoextend on next 8g maxsize 400g NOLOGGING EXTENT MANAGEMENT LOCAL
AUTOALLOCATE SEGMENT SPACE MANAGEMENT AUTO;
Substitute the size parameters with your required trail file storage size.
Create the file system:
% cd $ORACLE_HOME/rdbms/admin
% sqlplus dbfs_user/dbfs_password
SQL> start dbfs_create_filesystem dbfs_gg_dirtmp_tbs gg_dirtmp
The LOB segment used by DBFS should be configured with the storage options NOCACHE
NOLOGGING which is the default when the tablespace is created with the NOLOGGING option:
FROM dba_lobs WHERE tablespace_name='DBFS_GG_DIRTMP_TBS';
-- More than likely it will be something like this:
--
-- ------------------ ---------------------- ------- ----------
-- T_GG_DIRTMP LOB_SFS$_FST_73 NO NO
Follow the instructions in My Oracle Support note 1054431.1 for configuring the newly created DBFS
file system so that the DBFS instance and mount point resources are automatically started by Cluster
Ready Services (CRS) after a node failure. When registering the resource with Oracle Clusterware, be
sure to create it as a cluster_resource instead of a local_resource as specified in the My
Oracle Support note. The reason for using cluster_resource is so the resource can only be
running on a single node preventing the accidental mounting of DBFS from concurrent nodes.
Example command to register the resource:
crsctl add resource $RESNAME
-type cluster_resource
-attr "ACTION_SCRIPT=$ACTION_SCRIPT,
CHECK_INTERVAL=30,RESTART_ATTEMPTS=10,
START_DEPENDENCIES='hard(ora.$DBNAMEL.db)pullup(ora.$DBNAMEL.db)',

8
STOP_DEPENDENCIES='hard(ora.$DBNAMEL.db)',
SCRIPT_TIMEOUT=300"
Once the file system is mounted, create directories in the newly created file system for storing the
Oracle GoldenGate files.
Example:
% cd /mnt/gg_source/goldengate
% mkdir dirchk
% mkdir dirpcs
% mkdir dirprm
% mkdir dirdat
% mkdir BR
Create symbolic links for the directories that are not controlled by Oracle GoldenGate parameters:
% ln –s /mnt/gg_source/goldengate/dirprm $GG_HOME/dirprm
% ln –s /mnt/gg_source/goldengate/dirchk $GG_HOME/dirchk
% ln –s /mnt/gg_source/goldengate/dirpcs $GG_HOME/dirpcs
% ln –s /mnt/gg_dirtmp $GG_HOME/dirtmp
The Bounded Recovery (BR) feature was added to Extract in Oracle GoldenGate version 11.1.1. This
feature guarantees an efficient recovery after Extract stops for any reason, planned or unplanned, no
matter how many open (uncommitted) transactions there were at the time that Extract stopped and no
matter how old they were. Bounded Recovery sets an upper boundary for the maximum amount of
time that it would take for Extract to recover to the point where it stopped and then resume normal
processing. The Bounded Recovery checkpoint files should be placed on a shared file system such that
in an event of a failover when there are open long running transactions, Extract can use Bounded
Recovery to reduce the time to taken to perform recovery. Starting in Oracle GoldenGate version
11.2.1 the Bounded Recovery files are supported and it is recommended that you place them on DBFS.
With earlier releases the Bounded Recovery files need to be stored on NFS storage such as Oracle ZFS
Storage Appliance. It is possible to store the checkpoint files on the local file system, but when Extract
performs recovery after a node failure, the standard checkpoint mechanism will be used until new local
Bounded Recovery checkpoint files are subsequently created. This will only be noticeable if there are
long running transactions at the time of the failure.
To set the Bounded Recovery file directory use the following Extract parameter:
BR BRDIR /mnt/gg_source/goldengate/BR

9
For more information on Bounded Recovery refer to the Oracle GoldenGate Windows and UNIX Reference
Guide:
http://docs.oracle.com/cd/E35209_01/doc.1121/e29399.pdf
The location of the Extract/Data Pump trail file directory is specified during process creation. For
Extract it is also specified in the parameter file with the EXTTRAIL.
Target Environment (Replicat)
On the target environment, where the Replicat processes read the trail files and apply the data to the
target database there is a requirement for two separate DBFS file systems to separate the different I/O
requirements of the trail and checkpoint files.
Trail files are written to by the Collection Server process on the target host using consecutive serial
I/O’s from the start to the end of the file, sized according to your Data Pump configuration. The same
trail files are read by each Replicat process, also using consecutive serial I/O requests. Once a portion
of the trail is read by a Replicat process it will not normally be read a second time by the same process.
When using multiple Replicat processes reading from the same trail files, it is rare that they remain in
sync, reading from the same portion of the trail file at the same time. Because of this, the best
configuration for DBFS is with NOCACHE LOGGING storage options. This is described above in
configuring the source environment.
The checkpoint files are small (approximately 4KB) but written to frequently, overwriting previous
data. The file doesn’t grow in size and is only read during process startup to determine the proper
starting point for recovery or initiation. Because the checkpoint file is written to over and over,
performance is best when the file is stored in DBFS with the CACHE LOGGING storage option.
Setting the CACHE option causes the small amount of data being written to the checkpoint files to be
written into the buffer cache of the DBFS instance, and not issuing direct writes to disk causing higher
waits on I/O. In testing this has shown to increase checkpoint performance by a factor of 2 to 5 times
compared to using the NOCACHE configuration with DBFS.
Create the second DBFS file system for the checkpoint files in much the same way as the file system
on the source environment (above). Some important notes:
 The file system is only for checkpoint files, so it can be sized less than 100 MB.
 Create the file system using the same user as the first file system created. It is important to make sure
the same user creates both file systems.
 Change the LOB storage parameters to CACHE LOGGING:
SQL> SELECT table_name, segment_name,logging
FROM dba_lobs WHERE tablespace_name='DBFS_GG_CKPT_TBS';
-- Likely it will be something like this:
--

10
-- ------------------ ---------------------- ------- ----------
-- T_GOLDENGATE2 LOB_SFS$_FST_75 YES NO
SQL> ALTER TABLE DBFS.<TABLE_NAME> MODIFY LOB (FILEDATA)
(NOCACHE LOGGING);
-- View the new LOB storage:
SQL> SELECT table_name, segment_name,logging
FROM dba_lobs WHERE tablespace_name='DBFS_GG_CKPT_TBS';
TABLE_NAME SEGMENT_NAME LOGGING CACHE
------------------ ---------------------- ------- ----------
T_GOLDENGATE2 LOB_SFS$_FST_75 YES YES
Note: If you are using an Oracle GoldenGate Data Pump process to transfer the trail files from a
source host on the database machine using DBFS, then contact Oracle GoldenGate Support to obtain
the fix to Bug 10146318. This bug fix improves trail file creation performance on DBFS by the Oracle
GoldenGate server/collector process. This only affects Oracle GoldenGate versions earlier than
11.1.1.0.5.
Step 2: Install Oracle GoldenGate
1. Download the Oracle GoldenGate software from Oracle Technology Network (OTN) at:
http://www.oracle.com/technetwork/middleware/goldengate/downloads/index.html
2. Install Oracle GoldenGate locally on the primary source and target nodes in the Oracle RAC
configuration. Make sure the installation directory is the same on all nodes.
3. Once you have successfully configured Oracle GoldenGate on the primary source and/or
target nodes, shut down Extract/Replicat and copy the entire Oracle GoldenGate home
directory to the other source and target nodes.
4. Follow the generic installation instructions for the source and target machine installations
available in Chapter 2, “Installing GoldenGate” at:
Step 3: Configure Oracle GoldenGate and Database Parameters
It is recommended that you configure Oracle GoldenGate Extract in Integrated Capture Mode to take
advantage of the increased data type support, TDE, and data compression.
Introduced in Oracle GoldenGate version 11.2.1, Extract can be used in integrated capture mode. Extract
integrates with an Oracle database log mining server to receive change data from that server in the
form of logical change records (LCR). Extract can be configured to capture from a local or
downstream mining database. Because integrated capture is fully integrated with the database, no
additional setup is required to work with Oracle RAC, ASM, TDE, and data compression. Integrated
capture can be used starting with Oracle 11.2.0.3 with the patches described in My Oracle Support note

11
1411356.1. It can also be used to capture changes from Oracle versions starting with 10.2.0.4 in a
downstream mining deployment.
Extract can still be configured to capture directly from the redo logs for any supported Oracle version.
This configuration is now called classic capture mode.
1. Extract using Integrated Capture mode.
a. Set the database initialization parameter STREAMS_POOL_SIZE = 1.25GB X
#Integrated Capture Processes.
For further details about configuring Extract in Integrated Capture mode, refer to the
Oracle GoldenGate Installation and Setup Guide:
2. Extract in Classic Capture mode.
a. Use the default Oracle Automatic Storage Manager (Oracle ASM) naming convention for
the archived redo log files.
b. Configure the Oracle GoldenGate Extract parameter for the newer Oracle ASM log read
API.
Oracle GoldenGate release 11.1.1 introduces a new method of reading log files stored in
Oracle ASM. This new method uses the database server to access the redo and archived
redo log files, instead of connecting directly to the Oracle ASM instance. The database
must contain the libraries with the API modules. The libraries are currently included with
Oracle Database release 10.2.0.5, 11.2.0.2 and 11.2.0.3.
To successfully mine the Oracle archived redo log files located on the storage cells that
are managed by Oracle ASM, configure the Oracle GoldenGate Extract parameter as
follows:
Set the TRANLOGOPTIONS parameter to specify use of the new log read API. For
example:
TRANLOGOPTIONS DBLOGREADER
3. Configure Data Pump.
Configure the Data Pump with the PASSTHRU parameter if the process is not carrying out
any mappings or conversions. Using PASSTHRU reduces CPU by the Data Pump because it
does not have to look up table definitions, either from the database or from a data definitions
file.
For further details on Extract configuration or Data Pump with PASSTHRU, refer to the
Oracle GoldenGate Windows and UNIX Reference Guide:

12
Step 4: Set Up Checkpoint Files and Trail Files in DBFS
1. Set up checkpoint files.
Checkpoint files contain the current read and write positions of the Extract and Replicat
processes. Checkpoints provide fault tolerance by preventing the loss of data should the
system, the network, or an Oracle GoldenGate process need to be restarted.
Placing the checkpoint files on the local file system will not provide high availability in the
event of a database node failure. A checkpoint table can be used to record Replicat checkpoint
information to provide an alternative method of fault tolerance.
To store the checkpoint files on DBFS, the best practice is to create a symbolic link from the
Oracle GoldenGate home directory to a directory in DBFS. This is detailed in the DBFS
configuration in Step 1: Set Up DBFS on Oracle RAC.
Note: Oracle GoldenGate uses file locking on the checkpoint files to determine if the Extract
or Replicat processes are already running. This would normally prevent the process from
being started a second time on another Oracle RAC node that has access to the checkpoint
files. DBFS does not support this method of file locking. Mounting DBFS on a single Oracle
RAC node prevents access to the checkpoint files from other nodes. This will in turn prevent
the Extract or Replicat from being started concurrently on multiple nodes.
2. Set up trail files.
Trail files contain the data extracted from the archived redo log files. The trail files are
automatically generated by the Extract process.
To configure Oracle GoldenGate trail files on DBFS for the source database:
1. Create and mount the DBFS file system.
This is detailed in Step 1: Set Up DBFS on Oracle RAC.
2. Set the EXTTRAIL Extract parameter:
EXTTRAIL /mnt/dbfs/goldengate/dirdat/aa
3. After creating the Extract, use the same EXTTRAIL parameter value to add the local trail:
% ggsci
GGSCI (ggtest.oracle.com) 1> ADD EXTTRAIL
/mnt/dbfs/goldengate/dirdat/aa, EXTRACT ext_db, Megabytes 500

13
Further instructions about creating the Extract are available in the Oracle GoldenGate Windows
and UNIX Administration Guide at
To configure Oracle GoldenGate trail files on DBFS for the target database:
1. Make sure the DBFS directory is already created on the target environment.
2. Set the EXTTRAIL Replicat parameter, as follows:
EXTTRAIL /mnt/dbfs/goldengate/dirdat/aa
3. When adding the Replicat, use the same EXTTRAIL parameter value:
% ggsci
GGSCI (ggtest.oracle.com) 1> ADD REPLICAT rep_db1, EXTTRAIL
/mnt/dbfs/goldengate/dirdat/aa
Do not place trail files on the local file system because it will lengthen restart times in the
event of a node failure, reducing availability.
To configure Data Pump between a source and target database:
1. Make sure Extract and Replicat are configured
2. Set the RMTHOST Data Pump parameter to the IP or hostname that will be used for
connecting to the target. In Step 7: Oracle Clusterware Configuration, the Application Virtual
IP address is created with Cluster Ready Services (CRS) so that a single IP address can be
moved between compute nodes, so that Data Pump can continue to connect to the target
host when it moves from a failed node to a surviving node:
RMTHOST gg_dbmachine, MGRPORT 8901
3. Set the RMTTRAIL Data Pump parameter to the trail file location on the target host:
RMTTRAIL /mnt/dbfs/goldengate/dirdat/aa

14
4. Create a Data Pump process using the local trail file location on the source host:
% ggsci
GGSCI (ggtest.oracle.com) 1> ADD EXTRACT dpump_1, EXTTRAILSOURCE
/mnt/dbfs/goldengate/dirdat/aa
5. Use the ADD RMTTRAIL command to specify the remote trail file location on the target
host:
% ggsci
GGSCI (ggtest.oracle.com) 1> ADD RMTTRAIL
/mnt/dbfs/goldengate/dirdat/aa EXTRACT dpump_1, MEGABYTES 500
Further instructions about creating the Data Pump process are available in the Oracle
GoldenGate Administration Guide at
Step 5: Configure Replicat Commit Behavior
With Oracle GoldenGate version 11.2.1 and above, if a checkpoint table is configured, Replicat will
automatically operate using COMMIT NOWAIT. The Replicat processes will no longer wait at each
commit when applying transactions, increasing throughput performance.
For instructions on creating the checkpoint table, refer to the Oracle GoldenGate Oracle Installation and
Setup Guide, Chapter 4:
If you are using Oracle GoldenGate version 11.1.1 or earlier, setting the Replicat commit behavior to
COMMIT NOWAIT is configured separately from creating a checkpoint table. This should only be
considered when using a checkpoint table due to protection of recovery data during a checkpoint.
Set the Replicat parameter file to COMMIT NOWAIT as follows:
SQLEXEC "ALTER SESSION SET COMMIT_WRITE='NOWAIT'";
Step 6: Configure Autostart of Extract, Data Pump, and Replicat Processes
Configure the Extract, Data Pump (if used), and Replicat processes to automatically start when the
Manager process is started. Add the following parameter to the Manager parameter file:
AUTOSTART ER *
AUTORESTART ER *

15
If required, instead of using the Oracle GoldenGate process name wildcard (*), explicitly name the
processes you want to be restarted automatically.
Example:
AUTOSTART EXTRACT EXT_1A
AUTOSTART EXTRACT DPUMP_1A
AUTORESTART EXTRACT EXT_1A
AUTORESTART EXTRACT DPUMP_1A
Step 7: Oracle Clusterware Configuration
The following step-by-step procedure shows how to instruct Oracle Clusterware to start Oracle
GoldenGate using the 11.2.0.3 Oracle Grid Infrastructure Bundled Agent. If using a Grid
Infrastructure release earlier than 11.2.0.3, use the instructions in Appendix A: Creating Oracle
GoldenGate Clusterware Resourceto create and register an Oracle GoldenGate resource with Oracle
Clusterware.
1. Create an Application Virtual IP Address (VIP).
An application VIP is required to ensure the remote data pumps can communicate with the
target database machine, regardless of which node is hosting Oracle GoldenGate.
The VIP is a cluster resource that Oracle Clusterware manages. The VIP is assigned to a
cluster node and will be migrated to another node in the event of a node failure. This allows
Oracle GoldenGate data pump to continue transferring data to the newly assigned target
node.
Perform the following configuration steps:
a. To create the application VIP, run the following as the root user:
$GRID_HOME/bin/appvipcfg create -network=1
-ip=10.1.41.93
-vipname=gg_vip_source
-user=root
In the example:
 $GRID_HOME is the Oracle home in which Oracle 11g Release 2 Grid
infrastructure components have been installed (for example:
/u01/app/grid).
 network is the network number that you want to use. With Oracle Clusterware
release 11.2.0.1, you can find the network number using the following command:
crsctl stat res -p |grep -ie .network -ie subnet |grep -ie
name -ie subnet

16
Consider the following sample output:
NAME=ora.net1.network
USR_ORA_SUBNET=10.1.41.0
net1 in NAME=ora.net1.network indicates this is network 1, and the
second line indicates the subnet on which the VIP will be created.
 ip is the IP address provided by your system administrator for the new
Application VIP. This IP address must be in the same subnet as determined
above.
 gg_vip_source is the name of the application VIP that you will create.
b. Run the following command to give the Oracle Database installation owner
permission to start the VIP:
$GRID_HOME/bin/crsctl setperm resource gg_vip_source -u
user:oracle:r-x
c. As the Oracle Database installation owner, start the VIP resource:
$GRID_HOME/bin/crsctl start resource gg_vip_source
d. To validate whether the VIP is running and on which node it is running, execute:
$GRID_HOME/bin/crsctl status resource gg_vip_source
See the Oracle Clusterware documentation for further details about creating an Application
VIP: http://docs.oracle.com/cd/E11882_01/rac.112/e16794/crschp.htm#CHDGHGJA
2. Configure Oracle Grid Infrastructure Bundled Agent.
Introduced in release 11.2.0.3 for 64bit Linux, the Oracle Infrastructure Bundled Agents
provide pre-defined Oracle Clusterware resources for Oracle GoldenGate, Siebel, and Apache
applications. Using the bundled agent for Oracle GoldenGate, it is simple to create
dependencies on the source/target database, the application VIP, and the DBFS mount point.
The agent command line utility (AGCTL) is used to start and stop Oracle GoldenGate and
can also be used to relocate Oracle GoldenGate between the nodes in the cluster.
The bundled agent for Oracle GoldenGate fully supports use of Extract running in classic or
integrated capture modes.
The current version certification matrix:
Grid Infrastructure Oracle GoldenGate Oracle Database
11.2.0.3.+ 11.2.1.+ 10.2.0.5, 11.2.0.2+
The bundled agent software should be downloaded from the following location:

17
http://www.oracle.com/technetwork/products/clusterware/downloads/index.html
Follow the installation instructions provided in the readme.txt file.
Example configuration of the bundled agent:
a. Configure DBFS as detailed in My Oracle Support note 1054431.1 and test to make
sure CRSCTL can be used to mount and unmount the file system. Mounting the file
system should also start the DBFS instance if it is not already running.
NOTE: It is important that the DBFS resource is registered with Oracle
Clusterware for mounting and unmounting DBFS. If the Oracle Clusterware
resource is not registered and tested, as described in MOS note 1054431.1, it
will not be possible to use the Bundled Agent to automatically mount and
unmount DBFS.
b. Use AGCTL to create the Oracle Clusterware resource:
On the Source environment
% agctl add goldengate GG_Source --gg_home
/home/oracle/goldengate
--instance_type source
--nodes dbm01db05,dbm01db06
--vip_name gg_vip_source
--filesystems dbfs_mount --databases ggs
--oracle_home /u01/app/oracle/product/11.2.0.3/dbhome_1
--monitor_extracts ext_1a,dpump_1a
On the Target environment:
% agctl add goldengate GG_Target --gg_home
/home/oracle/goldengate
--instance_type target
--nodes dbm03db01,dbm03db02
--vip_name gg_vip_target
--filesystems dbfs_mount --databases ggt
--oracle_home /u01/app/oracle/product/11.2.0.3/dbhome_1
--monitor_replicats rep_1a,rep_2a,rep_3a,rep_3b
NOTE: The dbfs_mount value for the --filesystems parameter is the name of
the resource registered with Oracle Clusterware. This should have been
carried out as described in My Oracle Support note 1054431.1.
Once the Oracle GoldenGate processes have been added to a bundled they should
only be started and stopped using AGCTL. The bundled agent starts the Oracle
GoldenGate processes by starting Manager which in turn will automatically start the
processes (Extract, Data Pump, Replicat) that have been configured to autostart. If
an Oracle GoldenGate process aborts due to a problem, as long as the manager
process is still running it is okay to use GGSCI to restart the failed process.
To check the status of Oracle GoldenGate:

18
% agctl status goldengate GG_Source
Goldengate instance 'GG_Source' is running on dbm01db06
GGCSI can be used to check the status of individual Oracle GoldenGate processes.
To start Oracle GoldenGate manager, and all processes that have autostart enabled:
% agctl start goldengate GG_Target [--serverpool serverpool_name
| --node node_name]
Note: Oracle GoldenGate will start up on the node you issue the command from,
unless a node name or serverpool name is specified.
To stop all Oracle GoldenGate processes:
% agctl stop goldengate GG_Target [--serverpool serverpool_name
| --node node_name]
To relocate Oracle GoldenGate to another node:
% agctl relocate goldengate GG_Source [--serverpool
serverpool_name | --node node_name]
Note: The Oracle GoldenGate resource MUST be running before relocating it.
To view the configuration parameters for the Oracle GoldenGate resource:
% agctl config goldengate GG_Target
GoldenGate location is: /home/oracle/goldengate
GoldenGate instance type is: source
Configured to run on Nodes: dbm01db05 dbm01db06
ORACLE_HOME location is:
/u01/app/oracle/product/11.2.0.3/dbhome_1
Databases needed: ggs
File System resources needed: dbfs_mount
Extracts to monitor: ext_1a,dpump_1a
Replicats to monitor:
To view more detailed clusterware configuration information:

19
% crsctl stat res -w "NAME = xag.GG_Source.goldengate" –p
To delete the Oracle GoldenGate resource:
% agctl remove goldengate GG_Source
Further information on the Oracle Grid Infrastructure Bundled Agent:
http://www.oracle.com/technetwork/products/clusterware/overview/ogiba-
reference-guide-v1-1844341.html

20
Appendix A: Creating Oracle GoldenGate Clusterware Resource
If the Grid Infrastructure release is earlier than 11.2.0.3, Oracle GoldenGate can still be registered as a
resource with Oracle Clusterware using the following instructions.
1. Create an agent script (if not using the Bundled Agent)
If you are not able to use the Oracle Grid Infrastructure Bundled Agent it is still possible to
use Oracle Clusterware to manage automatic starting and stopping of Oracle GoldenGate.
Oracle Clusterware runs resource-specific commands through an entity called an agent. The
agent script:
 Must be able to accept five parameter values: start, stop, check, clean and
abort.
 Must be stored in the same location on all nodes. In this example, it is stored in the
Grid Infrastructure ($GRID_HOME) ORACLE_HOME/crs/script directory.
 Must be owned by the Oracle Grid Infrastructure user and have execute permissions.
 Must be accessible at the same location on every node in the cluster.
 Must include environment variable settings for ORACLE_HOME, ORACLE_SID,
PATH, TNS_ADMIN and LD_LIBRARY path so that CRS will be able to find the
correct program executables and Oracle Net configuration. If the correct sqlnet.ora,
tnsnames.ora or dbfs_client executable cannot be found when CRS tries to mount
DBFS it will fail.
See Appendix B: Example Agent Script for an example agent script starts and stops the
Oracle GoldenGate Manager, Extract, Data Pump, and Replicat processes.
It is important to manually test the agent script to start and stop the Oracle GoldenGate
processes before moving onto the next step.
2. Register a resource in Oracle Clusterware.
Register Oracle GoldenGate as a resource in Oracle Clusterware using the CRSCTL utility.
When using DBFS to store Oracle GoldenGate files, follow the configuration guidelines
provided in My Oracle Support note 1054431.1 (detailed above). Mounting of DBFS is carried
out by an Oracle Clusterware resource which in turn has a start dependency on the DBFS
instance. IF the DBFS resource is started and the instance is not running, it will be started
automatically. The DBFS mount resource is named as a start dependency for the Oracle
GoldenGate resource such that the required file systems are mounted BEFORE the Oracle
GoldenGate processes are started.
It is also recommended to list the source or target databases as start dependencies for the
Oracle GoldenGate resource so that the Extract or Replicat processes won’t fail when it can’t
connect to the database.

21
1. Determine the name of the DBFS or source/target database resource for the start
dependency:
crsctl status resource | grep -i dbfs|source/target DB name
2. Use the Oracle Grid Infrastructure user (oracle, in this example) to execute the
following:
$GRID_HOME/bin/crsctl add resource GG_Source
-attr
"ACTION_SCRIPT=/u01/app/11.2.0/grid/crs/script/11gr2_gg_action.scr,
CHECK_INTERVAL=30,
START_DEPENDENCIES='hard(gg_vip_source,dbfs_mount,ora.ggs.db)
pullup(gg_vip_source)', STOP_DEPENDENCIES=’hard(gg_vip_source)’,
SCRIPT_TIMEOUT=300"
If an Application VIP is not used, issue the following command:
-attr
CHECK_INTERVAL=30,
START_DEPENDENCIES='hard(ora.ggs.db,dbfs_mount)
pullup(ora.ggs.db,dbfs_mount) attraction(ora.ggs.db)',
STOP_DEPENDENCIES='hard(intermediate:ora.ggs.db,intermediate:dbfs_m
ount)', SCRIPT_TIMEOUT=300"
If the source and target databases are within the same cluster, make sure the Extract and
Replicat is restricted to the designated cluster nodes:
-attr
CHECK_INTERVAL=30, START_DEPENDENCIES='hard(gg_vip_source,dbfs_mount)
pullup(gg_vip_source)', STOP_DEPENDENCIES=’hard(gg_vip_source)’,
HOSTING_MEMBERS=’dbm01db05 dbm01db06’, PLACEMENT=’restricted’,
SCRIPT_TIMEOUT=300"
For more information about the CRSCTL add resource command and its options, see
the Oracle Clusterware Administration and Deployment Guide at
http://docs.oracle.com/cd/E11882_01/rac.112/e16794/toc.htm
3. Start the resource.
Once the resource has been added, you should always use Oracle Clusterware to start Oracle
GoldenGate. Login as the Oracle Grid Infrastructure user (oracle) and execute the
following:
% $GRID_HOME/bin/crsctl start resource GG_Source

22
To check the status of the application, enter the command:
% $GRID_HOME/bin/crsctl status resource GG_Source
For example:
% crsctl status resource GG_Source
NAME=GG_Source
TYPE=cluster_resource
TARGET=ONLINE
STATE=ONLINE on dbm01db05
4. Manage the application.
To relocate Oracle GoldenGate onto a different cluster node, use the
‘$GRID_HOME/bin/crsctl relocate resource’ API and include the force (-f)
option. This command can be run on any node in the cluster as the Grid Infrastructure user
(oracle).
If there is a dependency on an Application VIP you need to relocate the VIP resource which
in turn will stop Oracle GoldenGate, relocate and restart it on the new node.
For example:
[oracle@dbm01db05 ~]$ crsctl relocate resource gg_vip_source –f
CRS-2673: Attempting to stop 'GG_Source' on 'dbm01db05'
CRS-2677: Stop of 'GG_Source' on 'dbm01db05' succeeded
CRS-2673: Attempting to stop 'gg_vip_source' on 'dbm01db05'
CRS-2677: Stop of 'gg_vip_source' on 'dbm01db05' succeeded
CRS-2672: Attempting to start 'gg_vip_source' on 'dbm01db06'
CRS-2676: Start of 'gg_vip_source' on 'dbm01db06' succeeded
CRS-2673: Attempting to stop 'dbfs_mount' on ' dbm01db05'
CRS-2677: Stop of 'dbfs_mount' on ' dbm01db05' succeeded
CRS-2672: Attempting to start 'dbfs_mount' on 'dbm01db06'
CRS-2676: Start of 'dbfs_mount' on 'dbm01db06' succeeded
CRS-2672: Attempting to start 'GG_Source' on 'dbm01db06'
CRS-2676: Start of 'GG_Source' on 'dbm01db06' succeeded
To stop the Oracle GoldenGate resource, enter the following command:
$GRID_HOME/bin/crsctl stop resource GG_Source
5. Perform CRS cleanup.
To remove Oracle GoldenGate from Oracle Clusterware management, perform the following
tasks:
a) Stop Oracle GoldenGate (login as the Oracle Grid Infrastructure (oracle) user):

23
$GRID_HOME/bin/crsctl stop resource GG_Source
b) As the root user, delete the Oracle GoldenGate resource:
$GRID_HOME/bin/crsctl delete resource GG_Source
c) If no longer needed, delete the agent action script: 11gr2_gg_action.scr.
This does not delete the Oracle GoldenGate or DBFS configuration, only the Oracle
Clusterware resource.
Recommendations When Deploying on Oracle RAC
When Oracle GoldenGate is configured in an Oracle RAC environment, follow these
recommendations:
 Ensure that the DBFS database has instances running on all the database nodes involved in the
Oracle RAC configuration.
This action provides access to Oracle GoldenGate if it is restarted after a node failure.
 Ensure that the DBFS file system is mountable on all database nodes in the Oracle RAC
configuration. Mounting DBFS using a Clusterware resource to prevent the Extract or Replicat
processes from being started on multiple nodes concurrently, mount the file system only on the
node where Oracle GoldenGate is running. Use the same mount point names on all the nodes to
ensure seamless failover.

24
Appendix B: Example Agent Script
The following example agent starts and stops the Oracle GoldenGate Manager, Extract, and Replicat
processes. The mounting and unmounting of DBFS is handled by the DBFS Clusterware resource,
created in step 2 of Appendix A: Creating Oracle GoldenGate Clusterware Resource. The agent script
accepts the parameter values: start, stop, check, clean, and abort.
#!/bin/bash
#11gr2_gg_action.scr
# Edit the following environment variables:
# NOTE: The ORACLE_SID will be different on each node
export ORACLE_HOME=/u01/app/oracle/product/11.2.0.3/dbhome_1
export ORACLE_SID=GGS1
GGS_HOME=/home/oracle/goldengate/latest
export PATH=$ORACLE_HOME/bin:$PATH
export TNS_ADMIN=$ORACLE_HOME/network/admin
#Include the GoldenGate home in the library path to start GGSCI
export LD_LIBRARY_PATH=${ORACLE_HOME}/lib:${LD_LIBRARY_PATH}:${GGS_HOME}
# Edit this to indicate the DBFS mount point
DBFS_MOUNT_POINT=/mnt/dbfs
# Edit this to indicate at least one of the file systems mounted in
# the DBFS mount point
DBFS_FILE_SYSTEM=/mnt/dbfs/goldengate
# Edit for correct Extract/Datapump name if running this script on
# the source (widlcard is appended to these names in the script):
EXTRACT=EXT
DATAPUMP=DPUMP
# Edit for current Replicat names if running script on the target
# (widlcard is appended to these names in the script):
REPLICAT=REP
# Specify delay after start before checking for successful start
start_delay_secs=5
### Syslog facility name (default user)

25
### Changed default from local3 to user for Solaris default support on 17-
FEB-2012
### This will allow us to log messages to the syslog
### (/var/log/messages on Linux, /var/adm/messages on Solaris)
LOGGER_FACILITY=user
###########################################
### No editing is required below this point
###########################################
### determine platform
UNAME_S=`uname -s`
if [ $UNAME_S = 'Linux' ]; then LINUX=1; SOLARIS=0;
elif [ $UNAME_S = 'SunOS' ]; then LINUX=0; SOLARIS=1;
fi
LOGGER="/bin/logger -t GoldenGate"
logit () {
### type: info, error, debug
type=$1
msg=$2
if [ "$type" = "info" ]; then
echo $msg
$LOGGER -p ${LOGGER_FACILITY}.info "$msg"
elif [ "$type" = "error" ]; then
echo $msg
$LOGGER -p ${LOGGER_FACILITY}.error "$msg"
elif [ "$type" = "debug" ]; then
echo $msg
$LOGGER -p ${LOGGER_FACILITY}.debug "$msg"
fi
}
# check_process validates that Manager/Extract/Replicat process is
# running #at PID that GoldenGate specifies.
check_process () {
PROCESS=$1
if [ ${PROCESS} = mgr ]
then
PFILE=MGR.pcm

26
elif [ ${PROCESS} = ext ]
then
PFILE=${EXTRACT}*.pce
elif [ ${PROCESS} = rep ]
then
PFILE=${REPLICAT}*.pcr
else
PFILE=${DATAPUMP}*.pce
fi
if ( [ -f "${GGS_HOME}/dirpcs/${PFILE}" ] )
then
pid=`cut -f8 "${GGS_HOME}/dirpcs/${PFILE}"`
if [ ${pid} = `ps -e |grep ${pid} |grep ${PROCESS} |cut -d " " -f2` ]
then
logit info "${SCRIPTNAME}(check_process) - Procese(s) ${PROCESS} IS
running"
exit 0
else
if [ ${pid} = `ps -e |grep ${pid} |grep ${PROCESS} |cut -d " " -f1` ]
then
logit info "${SCRIPTNAME}(check_process) - Procese(s) ${PROCESS} IS
running"
exit 0
else
logit error "${SCRIPTNAME}(check_process) - Procese(s) ${PROCESS} is
NOT running"
exit 1
fi
fi
else
logit error "${SCRIPTNAME}(check_process) - Procese(s) ${PROCESS} is NOT
running - no pid file"
exit 1
fi
}
# call_ggsci is a generic routine that executes a ggsci command
call_ggsci () {
ggsci_command=$1
ggsci_output=`${GGS_HOME}/ggsci << EOF
${ggsci_command}
exit

27
EOF`
}
stop_everything () {
# Before starting, make sure everything is shutdown and process files are
removed
#attempt a clean stop for all non-manager processes
logit info "${SCRIPTNAME}(stop_everything) - Stopping all processes"
call_ggsci 'stop er *'
#ensure everything is stopped
call_ggsci 'stop er *!'
#in case there are lingering processes
call_ggsci 'kill er *'
#stop Manager without (y/n) confirmation
call_ggsci 'stop manager!'
#Remove the process files:
rm -f $GGS_HOME/dirpcs/MGR.pcm
rm -f $GGS_HOME/dirpcs/*.pce
rm -f $GGS_HOME/dirpcs/*.pcr
}
case $1 in
'start')
# stop all GG processes and remove process files
logit info "${SCRIPTNAME} - start - Starting all processes"
stop_everything
sleep ${start_delay_secs}
#Now can start everything...
#start Manager
logit info "${SCRIPTNAME} - start - Starting Manager, autostarting
processes"
call_ggsci 'start manager'

28
#there is a small delay between issuing the start manager command
#and the process being spawned on the OS <96> wait before checking
#start Extract or Replicats
call_ggsci 'start er *'
#check whether Manager is running and exit accordingly
logit info "${SCRIPTNAME} - start - Checking Manager status"
check_process mgr
#Check whether Extract is running
logit info "${SCRIPTNAME} - start - Checking GoldenGate statuses"
check_process ext
check_process dpump
;;
'stop')
logit info "${SCRIPTNAME} - stop - Stopping all processes"
stop_everything
#exit success
exit 0
;;
'check')
logit info "${SCRIPTNAME} - check - Checking all processes"
check_process mgr
check_process ext
check_process dpump
check_process rep
;;
'clean')
logit info "${SCRIPTNAME} - clean - Stopping all processes"
stop_everything

29
#exit success
exit 0
;;
'abort')
logit info "${SCRIPTNAME} - abort - Stopping all processes"
stop_everything
#exit success
exit 0
;;
esac

30
References
 Oracle GoldenGate Windows and UNIX Administration Guide version 11.2.1.0.1
 Oracle GoldenGate Oracle Installation and Setup Guide version 11.2.1.0.1
 Oracle GoldenGate Windows and UNIX Reference Guide version 11.2.1.0.1
 Oracle Database SecureFiles and Large Object Developer’s Guide (DBFS)
 Oracle Clusterware Administration and Deployment Guide
Oracle Maximum Availability Architecture Web site
http://www.otn.oracle.com/goto/maa

Oracle GoldenGate with Oracle Real
Application Clusters Configuration
August 2013
Author: Stephan Haisley
Contributing Authors: MAA team
Oracle Corporation
World Headquarters
500 Oracle Parkway
Redwood Shores, CA 94065
U.S.A.
Worldwide Inquiries:
Phone: +1.650.506.7000
Fax: +1.650.506.7200
oracle.com
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. This document is provided for information purposes only and the
contents hereof are subject to change without notice. This document is not warranted to be error-free, nor subject to any other
warranties or conditions, whether expressed orally or implied in law, including implied warranties and conditions of merchantability or
fitness for a particular purpose. We specifically disclaim any liability with respect to this document and no contractual obligations are
formed either directly or indirectly by this document. This document may not be reproduced or transmitted in any form or by any
means, electronic or mechanical, for any purpose, without our prior written permission.
Oracle and Java are registered trademarks of Oracle and/or its affiliates. Other names may be trademarks of their respective owners.
AMD, Opteron, the AMD logo, and the AMD Opteron logo are trademarks or registered trademarks of Advanced Micro Devices.
Intel and Intel Xeon are trademarks or registered trademarks of Intel Corporation. All SPARC trademarks are used under license
and are trademarks or registered trademarks of SPARC International, Inc. UNIX is a registered trademark licensed through X/Open
Company, Ltd. 1010

Maa goldengate-rac-2007111

Empfohlen

Empfohlen

Weitere ähnliche Inhalte

Was ist angesagt?

Was ist angesagt? (20)

Andere mochten auch

Andere mochten auch (14)

Ähnlich wie Maa goldengate-rac-2007111

Ähnlich wie Maa goldengate-rac-2007111 (20)

Kürzlich hochgeladen

Kürzlich hochgeladen (20)

Maa goldengate-rac-2007111