The document describes a migration from an Oracle database topology to a PostgreSQL database topology at ACI. It discusses the starting Oracle topology with issues around operational complexity and non-ACID compliance. It then describes the target PostgreSQL topology with improved performance, availability and lower costs. The document outlines decisions around tools, extensions, code changes and testing approaches needed for the migration. It also discusses options for migrating the data and cutting over to the new PostgreSQL environment.

1. Oracle to PostgreSQL migration
journey
Dominic Mortimer
April 30, 2021

2. Agenda
About me
ACI Overview
Starting point
Target destination
Extensions
Technology decisions (DDL, code, data)
Application code changes
Cut over

3. About me
3
Confidential
• Worked in many parts of the payments industry for over 20 years
• Had a number of roles, both technical and non-technical.
• Naturally gravitate towards the database tier, covering DB2 Z/os, Oracle, MSSQL and PostgreSQL!
• Worked on Oracle to PostgreSQL migration for last 2 – 3 years

4. ACI Spans the Payments Ecosystem
4
Confidential
WHAT WE DO
PROCESS AND MANAGE
DIGITAL PAYMENTS
• Banks
• Processors
• Central Infrastructures
• Networks
• Acquirers
• Merchant Intermediaries
ENABLE OMNI-COMMERCE
PAYMENTS
• Merchants
PRESENT AND PROCESS
BILL PAYMENTS
• Billers
MANAGE FRAUD
AND RISK
• Banks
• Processors
• Central Infrastructures
• Networks
• Acquirers
• Billers
• Merchant Intermediaries

5. ACI by the Numbers
5
Confidential
EMEA
~450
customers
Americas
5,300+
customers
Asia/Pacific
~250
customers
Support 19 of the
top 20 banks worldwide
Prevent fraud for 1,500+
banks, intermediaries
and merchants.
Serve 80,000+ merchants
directly and through payment
service providers
Support thousands
of billers
45
years
in payments
6,000+
global
organizations
rely on ACI
~4,000
employees
globally
$1.29B
2020
revenue

6. 6
Confidential
Starting point – Oracle topology
• Issues
− Operational complexity
▪ Replication latency
▪ Multiple replication techniques – multiple points of failure
▪ DR failover – need to set up replication in DR site
− Non ACID
− Code complexity
▪ Logic to know where to look for data
▪ DDL – must align table changes between instances and
custom code.
TX
processing
0-6 month old data
Historic DB1
6-18 months old
data
Historic DB2
18 + months old
data
Config
DB
Configuration
app
Payment
engine
Reporting
engine
Database link
Materialized views
Database link
Materialized views
Custom data movement code
Database link
Custom data movement code

7. 7
Confidential
Final destination
Lead
Master
Config + 0- 6 month
Logical
Standby
Config + all data
Configuration
app
Payment
engine
Reporting
engine
HAProxy
Shadow
Master
Config + 0- 6 month
HAProxy
Lead
Master
Config + 0- 6 month
Logical
Standby
Config + all data
Shadow
Master
Config + 0- 6 month
Configuration
app
Payment
engine
Reporting
engine
HAProxy
HAProxy
Primary Site Secondary Site
BDR
Sync rep
BDR
Sync rep
BDR
Async rep
BDR
Async rep
BDR
Async rep
BDR
Filter Deletes
BDR
Filter Deletes
• Benefits
• Reduced operational complexity
• Simplified DDL deployment
• Simplified application code
• All replication handled by BDR
• Improved performance (no network lag)
• Lower cost
• Still able to offload of read heavy queries
Jenkins
Ansible
Liquibase

8. Define business objective of migration
• Why do the project (reduce cost/operational complexity, increase availability,
make product more sellable, etc)
Define scope
• Which schemas/apps are to be impacted
Talk to budget holders and decision makers
• Set expectations around tools, infrastructure, risks and timescales
8
Confidential
Before you start

9. • Different approaches
− Reverse engineer (convert to generalised data model eg power designer)
− Convert already exported generated DDL (Text conversion)
− Read Oracle metadata and convert
• Things to consider
− Migration frequency
− Network speed
− Execute migration multiple times with different options
− Flexibility of index/FK creation – only want after data has been loaded
− Repeatability
− Is this a one off
9
Confidential
Migration technology decisions (DDL)
Tools
• Ora2PG
• AWS
• SQL Lines
• Easyfrom
• Pentaho

10. • orafce
− This is a very useful extension in your migration project
− Mimics a number of important (non iso standard functions)
− Download from git hub and compile against your version of PostgreSQL
− https://github.com/orafce/orafce
• Pgcrypto
− Replacement for DBMS_OBFUSCATION_TOOLKIT
10
Confidential
Code migration - Extensions
dbms_pipe
dbms_alert
dbms_sql (separate project)
dbms_assert
dbms_random
date functions
date operators
dbms_output
utl_file

11. 11
Confidential
DB code migration tools
Assess code
Report on
gaps
Run migration
Apply manual
changes
Compile/Test
Post migration
custom
process
Update tool
Update custom process
More manual changes
How to decide on which tool to use:
• Volume of gaps reported
• Is amount of manual code change acceptable?
• Frequency of migration
• Can the tool be updated?
• How much effort in creating post migration process
Tool Licence DDL Data Uses orafce
ora2pg Open source Y Y Y
AWS schema
conversion tool
Free Y Y N
Ispirer Migration
and modernisation
toolkit
Commercial Y Y Y
EDB PL
Translate tool
Commercial N (use
ora2pg)
N Y
EDB Advanced
server
Commercial Y Y N – Oracle
emulator
3rd Party Commercial Y Y Possible

12. • Public synonym/Synonym – use search path, assuming synonym name and object name is the same
• Collections – manual migration
• autonomous transactions – Use dblink
• Packages - Split into functions and procedures
− Object name contains package and function/package name schema.package__function
• Nested functions/procedures not supported – be careful with local variables
• DBMS_OBFUSCATION_TOOLKIT – use pgcrypto
12
Confidential
Can’t migrate – work arounds

13. • Unit
− Apples to apples (same results Oracle or PostgreSQL)
− Consider adding unit test to overnight test – pgTap framework
• Functional testing
− Use test automation
− Likely minimal changes to test automation suite
− Apples to apples
• Non functional requirements testing
− Performance – results may not be the same but still must meet NFR
− Availability – Same or better than Oracle
13
Confidential
Testing

14. • Things to consider
− Size of source data
− Rate of change in source system
− Migration time frame
− Impact on source system
− Acceptable latency
− Cost
− Replicate changes
• Tool options
− File based (eg Ora2pg)
− cdc replication tool (eg qlik/striim)
− SQL replication tool (eg nifi)
− Foreign data wrapper
14
Confidential
Data migration - Options
Approach/tool type Approach Source
data size
Cut over
timeframe
Impact on
source
system
Replicate changes Cost
Foreign Data Wrapper Insert into target select *
from FDW table
Can be iterative eg 1
day at a time
Small/Med Med Med Possible
Use MERGE
Need a way to
determine change,
eg last update date
Free
File based
Eg ora2pg
Create unload file
SFTP to target
copy in to target DB
Small/Med Related to
data size
Low/Med Possible but
complex.
Open
source
SQL replication
Eg nifi
Load then replicate
changes
Large Small Medium/high Yes
Replicate changes
by querying source
table.
Need a way to
determine change,
eg last update date
Open
source or
commercial
CDC replication
Eg qlik/striim
Initial load then replicate
changes
Large Small Low Yes – read log in
source/CDC
Commercial

15. • Same code base Oracle and PostgreSQL – Use ISO standard SQL or orafce supported
• Changes we made
− Rownum
▪ Rownum in select list – use row_number() over(order by ‘1’) window function
▪ Rownum as a predicate – fetch/offset
▪ SYSDATE – use current_timestamp
− Handle changed package names – replace package name/function name separator
▪ Oracle – schema.package_name.function_name
▪ PostgreSQL – schema.package_name__function_name
− New data source for PostgreSQL
15
Confidential
Application code changes

16. 16
Confidential
Cut over - Set up
TX
processing
0-6 month old data
Historic DB
6-18 months old
data
Historic DB
18 + months old
data
Config
DB
Configuration
app
Payment
engine
Reporting
engine
Lead
Master
Config + 0- 6 month
Logical
Standby
Config + all data
HAProxy
Shadow
Master
Config + 0- 6 month
Qlik
replication
(load only)
Qlik load and
changes
HAProxy

17. 17
Confidential
Cut over – RO
TX
processing
0-6 month old data
Historic DB
6-18 months old
data
Historic DB
18 + months old
data
Config
DB
Configuration
app
Payment
engine
Reporting
engine
Lead
Master
Config + 0- 6 month
Logical
Standby
Config + all data
HAProxy
Shadow
Master
Config + 0- 6 month
Qlik
replication
(load only)
Qlik load and
changes
HAProxy

18. 18
Confidential
Cut over – RW
TX
processing
0-6 month old data
Historic DB
6-18 months old
data
Historic DB
18 + months old
data
Config
DB
Configuration
app
Payment
engine
Reporting
engine
Lead
Master
Config + 0- 6 month
Logical
Standby
Config + all data
HAProxy
Shadow
Master
Config + 0- 6 month
Qlik
replication
(load only)
Qlik load and
changes
HAProxy