5. 5Patterns
Introduction
Oracle Golden Gate enables the exchange and manipulation of data at the
transaction level among multiple, heterogeneous platforms across the
enterprise.
Oracle Golden Gate provides low-impact capture, routing, transformation, and
delivery of database transactions across heterogeneous environments in near-
real time.
Oracle Golden Gate moves committed transactions from redo logs and maintains
transaction integrity with sub-second latency
6. 6Patterns
Oracle GoldenGate – Supported Platforms
Choice can be made from 2 types of Oracle GoldenGate (OGG):
Supported Databases Supported Operating Systems
z/OS, iSeries, z/LinuxAlways check the latest Certification Matrix
8. 8Patterns
Oracle GoldenGate Topologies
UNIDIRECTIONAL QUERY
OFFLOADING
BIDIRECTIONAL STANDBY
DB OR ACTIVE-ACTIVE FOR
HIGH-AVAILABILITY
PEER-TO-PEER LOAD
BALANCING,
MULTIMASTER
BROADCAST DATA
DISTRIBUTION
INTEGRATION / CONSOLIDATION
DATA WAREHOUSE CASCADING MARTS
10. 10Patterns
Benefits Of Oracle GoldenGate
High Availability (Standby Database)
Zero Down-Time Upgrades and Migrations
Live Reporting (Reporting Database)
Operational Business Intelligence
Transactional Data Integration
11. 11Patterns
Oracle Golden Gate Solutions for Oracle Database
Upgrade Oracle Database versions 8i, 9i, 10g or 11g to 12c
Migrate from non-Oracle databases to Oracle 12c
Upgrade or migrate the database server or operating system
Perform database maintenance
12. 12Patterns
SERVER: Golden Gate 1 (Source)
Oracle GoldenGate 12c – Source and TargetSCHEMAS
AP
AR
SYS
HR
WH
LGWR
Process
DATABASE: ORACLE
Writes
database
Changes to
redo log
files
Online Redo
Log Files
GoldenGate
extract process
(HREXT)
- - -
only reads redo
for HR schema
Reads
redo
log
files
GoldenGate
data pump
process
GoldenGate
Trail Files
File grows until
records are
processed
Reads
Trail
Files
Extract
Writes
HR schema
changes
to
SERVER: Golden Gate 2 (Target)
SCHEMAS
SYS
HR
DATABASE: ORACLE
GoldenGate
Replicat (RPHR01)
process
Changes written to
HR Schema
GoldenGate
Collector
process
Trail files
grow until
data
processed
Reads Trail
Files
Writes to
Trail Files
GoldenGate
Data Pump
writes
Writes to
collectors
13. 13Patterns
Oracle GoldenGate Solutions for Oracle Database
Eliminate Down-Time During Oracle Database Upgrades
Eliminate Unplanned Down-Time With Active Data Guard
Improve Production System Performance
Real-Time Reporting from a Lower-Cost System
Increase ROI On Existing Servers and Synchronize Global Data
14. 14Patterns
ELIMINATE DOWN-TIME DURING ORACLE DATABASE UPGRADES
ZERO DOWN-TIME DATABASE UPGRADES
CAPTURE
DELIVERY
CAPTURE
DELIVERY
REAL-TIME UPDATES
POST-SWITCHOVER
DATA FLOW
Route (LAN/WAN/Web/IP)
ORACLE
8i / 9i / 10g
ORACLE
11g
APPLICATION
SWITCH OVER
COMPARE & VERIFY
15. 15Patterns
ELIMINATE UNPLANNED DOWN-TIME WITH ACTIVE DATA GUARD
CAPTURE
DELIVERY
CAPTURE
DELIVERY
REAL-TIME UPDATES
POST-SWITCHOVER
DATA FLOW
Route (LAN/WAN/Web/IP)
APPLICATION
SWITCH OVER
SOURCE
STANDBY SERVER
COULD BE USED FOR
REPORTING,
QUERING, TESTING….
DISASTER RECOVERY & DATA PROTECTION
STANDBY
16. 16Patterns
IMPROVE PRODUCTION SYSTEM PERFORMANCE & LOWER COSTS
QUERY OFFLOADING
CAPTURE DELIVERY
REAL-TIME UPDATES
Route (LAN/WAN/Web/IP)
APPLICATION
TRANSACTION
PROCESSING
ACTIVITY
LEGACY PRODUCTION
OLTP
ORACLE REPLICA
READ-ONLY
ACTIVITY
18. 18Patterns
INCREASE ROI ON EXISTING SERVERS & SYNCHRONIZE GLOBAL DATA
ACTIVE - ACTIVE
SOURCE & TARGET DB SOURCE & TARGET DB
CAPTURE
DELIVERY
REAL-TIME UPDATES
Route (LAN/WAN/Web/IP)
APPLICATION APPLICATION
DELIVERY CAPTURE
19. 19Patterns
Replicat applies data with transaction integrity
ORACLE GOLDENGATE – THE WAY IT WORKS
EXTRACT
REPLICAT
TRAIL FILES
PUMP
• LAN
• WAN
• INTERNET
• OVER TCP / IP
SOURCE
ORACLE &
NON-ORACLE
DATABASE
CAPTURE
DELIVERY
TRAIL FILES
PUMP TARGET
ORACLE &
NON-ORACLE
DATABASE
REPLICAT
EXTRACT
Extract
Committed transactions are captured (and can be filtered) as they occur by reading the transaction logs.
Trail Stages and Queues Data for Routing
Pump Distributes Data for Routing to Target(s)
Route Data is Compressed, Encrypted for Routing to Target(s)
← B I - D I R E C T I O N A L →
21. 21Patterns
ORACLE GOLDEN GATE LOGICAL ARCHITECTURE
EXTRACT
EXTRACT
REPLICAT
REPLICAT
COLLECTOR
DATA
PUMP
MANAGER
MANAGER
NETWORK
DATA SOURCE
FOR INITIAL
LOAD: SOURCE
TABLES
INITIAL LOAD
TRAIL OR FILE
CHANGE SYNCHRONIZATION
DATA SOURCE
CHANGE
SYNCHRONIZA
-TION:
TRANSACTION
LOG
(OPTIONAL)
22. 22Patterns
Oracle GoldenGate Components
Source Database
Target Database
Server Collector: Reassembles and writes data to an OGG trail files.
Replicat: Queued transactions that are stored in the trail files, reads the trail file and
applies to the target database by using native SQL calls.
Extract: Captures transactional changes from transaction logs (Change Data Capture).
Trail: After the capture process, OGG converts the committed transactions into
canonical data format in “trail” files.
Pump: One more capture component can be used to pump the data to multiple
targets and is used for better recoverability.
Manager: Required to start and stop processes & responsible
for monitoring, restarting processes, allocating data storage, reporting errors and events
23. 23Patterns
Source Database – Configuration Files
GGSCI (ogg-wokshop) 27> view param mgr
PORT 15500
DYNAMICPORTLIST 15510-15520
PURGEOLDEXTRACTS ./dirdat/*, USECHECKPOINTS
GGSCI (ogg-wokshop) 28> view param extprm
EXTRACT extprm
TRANLOGOPTIONS DBLOGREADER
EXTTRAIL ./dirdat/ea
USERID ogguser, PASSWORD oracle
STATOPTIONS RESETREPORTSTATS
REPORT AT 12:00
REPORTROLLOVER AT 12:00
REPORTCOUNT EVERY 60 SECONDS, RATE
TABLE scott.*;
GGSCI (ogg-wokshop) 29> view param pmpprm
EXTRACT pmpprm
RMTHOST ogg-wokshop, MGRPORT 15000, COMPRESS
RMTTRAIL ./dirdat/pa
PASSTHRU
TABLE scott.*;
GGSCI (ogg-wokshop) 30> info all
Program Status Group Lag at Chkpt Time Since Chkpt
MANAGER RUNNING
EXTRACT RUNNING EXTPRM 00:00:00 00:00:10
EXTRACT RUNNING PMPPRM 00:00:00 00:00:09
24. 24Patterns
Target Database – Configuration Files
GGSCI (ogg-wokshop) 35> view param mgr
PORT 15000
DYNAMICPORTLIST 15010-15020
PURGEOLDEXTRACTS ./dirdat/*, USECHECKPOINTS
GGSCI (ogg-wokshop) 36> view param repdb
REPLICAT REPDB
ASSUMETARGETDEFS
DISCARDFILE ./dirout/REPDB.DSC, PURGE
USERID ogguser, PASSWORD oracle
MAP scott.*, TARGET scott.*;
GGSCI (ogg-wokshop) 37> info all
Program Status Group Lag at Chkpt Time Since Chkpt
MANAGER RUNNING
REPDB RUNNING REPDB 00:00:00 00:00:09
31. 31Patterns
Tune TCPBUFSIZE and TCPFLUSHBYTES
Calculate the bandwidth-delay product (BDP).
Example: if the network between the source and target databases is 155 megabits per second (Mbits) and
the latency is 39ms, the calculation would be as follows:
BDP = (155,000,000 / 8) * 0.039 = 755,625bytes
Multiply the result by 3 to determine 3xBDP.
Example: 3xBDP = 755,625 x 3 = 2,266,875
In this example, because the result is more than 1MB, set the TCPBUFSIZE and TCPFLUSHBYTES parameters
to 2,266,875.
The parameters are set in the Data Pump parameter file.
RMTHOST target, MGRPORT 7809, TCPBUFSIZE 2266875, TCPFLUSHBYTES 2266875
The two RMTHOST parameters, TCPBUFSIZE and TCPFLUSHBYTES, are very useful for increasing the buffer sizes and network
packets sent by Data Pump over the network from the source to the target system.
This is especially beneficial for high latency networks.
These parameters should be set to a value of 1MB (1,048,576 bytes) or the calculated value, whichever is larger.
To determine suitable values for these parameters, execute the following steps:
Use the ping command to obtain the average round trip time (RTT).
Output from PING command: Minimum = 31ms, Maximum = 61ms, Average = 39ms
32. 32Patterns
Startup Speed - DynamicResolution
EXAMPLE
EXTRACT EORA01
USERID ggadmin, PASSWORD ggadmin
EXTTRAIL ./dirdat/lt
DYNAMICRESOLUTION
TABLE SOURCE.TCUSTMER;
TABLE SOURCE.TCUSTORD;
STARTUP SPEED
When a process starts, it queries the database system tables to retrieve table metadata and builds a structure in
memory. Analyzing many tables takes time
Solution
DynamicResolution parameter adds table’s attributes to the object record the first time its object ID enters the
transaction log. This parameter builds out the columns only when a table is used.
33. 33Patterns
Monitoring : What is Lag?
GOLDEN GATE GOLDEN GATE
EXTRACT SOURCE TRAIL PUMP
TCP
TARGET TRAIL REPLICAT
SourceCommit
Timestamp
SystemTime
WritetoTrail
SystemTime
WritetoTrail
TargetCommit
Timestamp
REPLICATE LAG
“End-to-End” LATENCY
● ● ● ● ● ● ● ● ● ● ●
EXTRACT LAG
PUMP LAG
34. 34Patterns
● ● ● ● ●
If the lag in
EXTRACT, Start
Here!
● ● ● ● ●
If the lag
in PUMP,
check for
Lag in
EXTRACT
● ● ● ● ●
If the lag in
REPLICAT, check
for Lag in PUMP
Identifying the Bottlenecks
EXTRACT REPLICAT
SOURCE TRAIL PUMP
LAN /
WAN
INTERNET
TARGET TRAIL REPLICATSOURCE
DATABASE
TARGET
DATABASE
MANAGER MANAGER
← Monitoring Lag – Working from the Target back to the Source
▲
If the (latency) in
REPLICAT is acceptable
then All is Well
Throughput Statistics are also useful in identifying potential bottlenecks
35. 35Patterns
Finding Oracle GoldenGate Latency/Lag
For integrated Extract:
SQL> SELECT capture_name, 86400 *(available_message_create_time -
capture_message_create_time) latency_in_seconds
FROM GV$GOLDENGATE_CAPTURE;
For integrated Replicat:
SQL> SELECT r.apply_name, 86400 *(r.dequeue_time - c.lwm_message_create_time)
latency_in_seconds
FROM GV$GG_APPLY_READER r, GV$GG_APPLY_COORDINATOR c
WHERE r.apply# = c.apply# and r.apply_name = c.apply_name;
Latency or lag is the period of time that has passed between when a change (DML or DDL) occurred in
the source database and the current point in time.
Example: Extract latency is the time that has elapsed since the change occurred to the source table and
the time it was extracted and written to the trail file. Conversely, Replicat latency is the time that has
elapsed from the source table change to the time it was applied to the target database.
When using integrated Extract or integrated Replicat, the latency can be determined from the database
using the following queries:
36. 36Patterns
Monitoring - Lag
TO VIEW LAG, USE THE SEND OR LAG COMMANDS
GGSCI> send {Extract | Replicat} , GetLag (OR) GGSCI> lag {Extract | Replicat | ER}
YOU CAN USE INFOR, DETAIL COMMAND TO VIEW LAG
GGSCI> Info {Extract | Replicat | ER}, detail
CHECKING VOLUME STATISTICS
Using Report (OR) ReportCount parameters
Using the Stats command
GGSCI> send {Extract | Replicat} <group>, report
GGSCI> view report <group>
GGSCI> view report <filename>
GGSCI> send {Extract | Replicat} <group>, report
GGSCI> stats {Extract | Replicat | ER} {<group/wildcard>} [Table <name/wildcard>]
Regularly monitor lag and processing volume of extraction and replication
Continuous excess lag usually indicates a bottleneck in one or both processes
Compare Extract’s processing volume with Replicat’s volume
Extract usually faster. Replicat must construct and apply SQL statements
37. 37Patterns
Monitoring – Determining Lag using Checkpoints
To view the Extract checkpoint in the trail, use the following commands:
GGSCI> Info Extract <group>, ShowCh
GGSCI> Info Extract <group>, Detail
GGSCI> Info Replicat <group>, Detail
38. 38Patterns
Install the UTL_SPADV Package (Integrated Extract and Integrated Replicat)
To install SPADV, execute the following steps:
a. Grant the following privileges to a designated Oracle GoldenGate administrator database user:
SQL> exec DBMS_GOLDENGATE_AUTH.GRANT_ADMIN_PRIVILEGE( -
'<db user name>');
b. Connect to the database with the user name that was granted permissions in Step a.
c. Run the utlspadv.sql script. For example:
SQL> @$ORACLE_HOME/rdbms/admin/utlspadv.sql
The UTL_SPADV PL/SQL package provides subprograms to collect and analyze statistics for the LogMiner server
processes. The statistics help identify any current areas of contention such as CPU or I/O.
The Oracle Streams Performance Advisor (SPADV) enables monitoring of the integrated GoldenGate server
processes which are used by integrated Extract and integrated Replicat, and provides information about how these
processes are performing.
SPADV statistics are collected and analyzed using the UTL_SPADV package.
UTL_SPADV package, as the Oracle GoldenGate administrator user on the source database.
39. 39Patterns
Gather Statistics using the UTL_SPADV Package
To gather statistics every 15 seconds, run the following SQL*Plus command as the Oracle GoldenGate
administrator:
SQL> exec UTL_SPADV.START_MONITORING(interval=>15);
To stop statistics gathering, run the following command:
SQL> exec UTL_SPADV.STOP_MONITORING;
Run the following commands to determine if the monitoring job is currently running:
SET SERVEROUTPUT ON
DECLARE
is_mon BOOLEAN;
BEGIN
is_mon := UTL_SPADV.IS_MONITORING(
job_name => 'STREAMS$_MONITORING_JOB',
client_name => NULL);
IF is_mon=TRUE THEN
DBMS_OUTPUT.PUT_LINE('The monitoring job is running.');
ELSE
DBMS_OUTPUT.PUT_LINE('No monitoring job was found.');
END IF;
END;
/
Oracle recommends that you gather statistics for a 30-60 minute time period during which you are troubleshooting performance.
It is also recommended to gather statistics during a 30-60 minute time period where performance is good, serving as a baseline comparison.
40. 40Patterns
Generating Report - UTL_SPADV Package
spool /tmp/spadv.txt
begin
utl_spadv.show_stats(path_stat_table=>'STREAMS$_PA_SHOW_PATH_STAT',
bgn_run_id=> 1,
end_run_id=> 9999,
show_legend=> TRUE);
end;
After the reports have been generated, Oracle recommends purging the SPADV statistics using the
following
command:
SQL> exec UTL_SPADV.STOP_MONITORING(PURGE=>TRUE);
It is also possible to create a static report of SPADV statistics after monitoring for a period of time. The report can be
generated in text form much like the display of real-time statistics.
To generate a text report, from SQL*Plus as the Oracle GoldenGate administrator, execute the following:
41. 41Patterns
Tune the Manager Process
Manager can be tuned to restart processes that terminate abnormally:
Use the AutoRestart Manager parameter
(If the database goes down, autorestarting ensures that GoldenGate processes start again after recovery)
Use the AutoStart Manager parameter
(Starts processes when Manager starts)
Use the BootDelayMinutes parameter
(When database services must be started before starting Oracle GoldenGate processes)
42. 42Patterns
Tune the Extract Process
DATA FILTERING & DATA CONVERSION
Data filtering and Data conversion can cause performance degradation because we have to do extra work. Instead
of using Extract process, use Replicat or data pump to perform filtering and conversion.
We can use parallel pumps or Replicats
To remove the overhead from the source system, you can use data pumps on an intermediate system or the
target system.
This may not be the best option when the data pump is performing filtering
DATABASE FETCHING
Identify tables that generates fetches, generate long transactions, and use SQL procedures that fetch data
Separate into one or more separate extract groups
Keep tables with relational constraints to each other in the same group
Large tables with heavy LOB volume may require multiple extract groups for each
Use the @RANGE function to divide the load in a safe, scalable manner
Using multiple Extract requires multiple trails. You need a data pump and Replicat for each one
43. 43Patterns
Tune the Replicat Process
DIVIDE THE LOAD
Replication is a single threaded process that applies operation one at a time by using regular SQL. It is a bottleneck!
Divide the load into multiple Replicat groups that work in parallel
Try splitting the load by schema
You can try using a different group for each table
Keep those tables with relational constraints in the same group
@RANGE function: you can split a table among multiple groups so that row operations can be performed in
parallel
The @RANGE function is an easy, safe and scalable way to divide the rows among different processing groups.
It preserves data integrity by guaranteeing that the same row is always processed by the same group.
Note 1: The number of groups support depends on the memory resources available on the system. Each
Extract and Replicat process needs approximately 25 MB to 55 MB of memory.
Note 2: If you split one table among multiple replicats, you cannot do DDL replication
on that table, only DML.
44. 44Patterns
Tune the Replicat Process
Example Of DIVIDE THE LOAD
If there are a small number of tables that contain a large percentage of DML which, after dividing into their
own Replicat processes, are still not applying the data fast enough, these tables can be further distributed
among Replicat processes using the @RANGE function.
For example, distributing a table between two Replicat processes would use the following
MAP parameters:
Replicat #1:
MAP SOESMALL.ORDER_ITEMS , TARGET soesmall.ORDER_ITEMS
FILTER (@RANGE (1, 2, ORDER_ID));
Replicat #2:
MAP SOESMALL.ORDER_ITEMS , TARGET soesmall.ORDER_ITEMS
FILTER (@RANGE (2, 2, ORDER_ID));
The preceding example shows that the table SOESMALL.ORDER_ITEMS is partitioned using
the primary key column ORDER_ID between two Replicat processes.