SlideShare ist ein Scribd-Unternehmen logo
1 von 34
Downloaden Sie, um offline zu lesen
Which Change Data Capture
Strategy Is Right for You?
Presented by
Paige Roberts
Sr. Product Marketing Manager
Data Integration, Data Quality
1
Choosing a Change Data Capture Strategy
1 What is Change Data Capture?
2 Why Do Change Data Capture?
3 Strategies for Change Data Capture
4 Examples of Change Data Capture
5 Q and A
CDC is the process that ensures that changes made over time in
one dataset are automatically transferred to another dataset.
Change Data Capture or CDC is most often used with databases that hold important
transactional data to make sure that organizations are working with up-to-date information
across the enterprise.
Source - often used to record transactions or other business occurrences as they happen.
Target - often used to create a report or do analysis to determine a course of action.
Sometimes, data is replicated bi-directionally so that a source is also a target and vice versa.
Which Change Data Capture Strategy Is Right for You?3
What is Change Data Capture?
Replication Options
One Way
Two Way
Cascade
Bi-Directional
Distribute
Consolidate
Choose a
topology or
combine them to
meet your data
sharing needs
5
Integrated Architecture Use Case
ERP SYSTEM
Customer Orders
Payment Details
Product Catalogue
Price List
eCOMMERCE &
WEB PORTALS
TEST & AUDIT
ENVIRONMENT
DATA EXCHANGE
WITH OUTSIDE VENDOR
(FLAT FILE)
DR /
BACKUP
6
Customer Example Architecture
EDGE
NODE
CLUSTER DATA NODES
DATABASE
SOURCES
MAINFRAME
SOURCES
VSAM
Db2
CAPTURE
AGENT
Reasons for CDC
1. Businesses have Multiple Databases
Multiple databases are the norm
• Merger or acquisition
• Choice of multiple apps or databases for best of breed solutions
• Combination of legacy and new databases
• Multi-organization supply chain
IT infrastructures are heterogeneous
• Database platforms
• Operating systems
• Hardware
8
Drivers Behind Change Data Capture
83%
10%
8% Does your organization rely on multiple databases?
Yes No I don't know.
73%of those with multiple databases
share data among them
Does your organization share data between multiple databases?
Source: Vision Solutions ‘ 2017 State of Resilience Report
2. Enabling Analytics, Reporting and BI
• Protecting performance of production
database by offloading data to a reporting
system for queries, reports, business
intelligence or analytics
• Consolidating data into centralized
databases, data marts or data warehouses
for decision making or business processing
Which Change Data Capture Strategy Is Right for You?9
Drivers Behind Change Data Capture
3. Enabling Machine Learning, Advanced Analytics and AI
• Growing data volumes lead to new architectures for
data consolidation – data lakes and enterprise data hubs
based on Hadoop or Spark.
• New types of data and larger amounts of data from
multiple sources combined together create an ideal
environment for training and employing machine
learning and artificial intelligence.
• Businesses across many industries seek competitive
edge from these new technologies in use cases from
fraud detection to targeted marketing.
• ML and AI systems have a constant, voracious need for
more data, and must constantly have the latest, most
current data available to provide the promised insights.
Which Change Data Capture Strategy Is Right for You?10
Drivers Behind Change Data Capture
4. Varied Business and IT Goals
• Offloading data for maintenance, backup, or testing
on a secondary system without production impact
• Maintaining synchronization between siloed
databases or branch offices
• Feeding segmented data to customer or partner
applications
• Migrating data to new databases
• Re-platforming databases to new database or
operating system platforms
11
Drivers Behind Change Data Capture
Source: Vision Solutions ‘ 2017 State of Resilience Report
For what business purpose does your organization share data
between databases?
Consolidating data from multiple sources into…
Reporting on data offloaded from the…
Synchronizing data between distributed…
Testing on offloaded data
Running business processes on offloaded data
I don’t know
0% 10% 20% 30% 40% 50% 60% 70%
Why do you need to capture and move the changes in your data?
• Populating centralized databases, data marts, data warehouses, or data lakes
• Enabling machine learning, advanced analytics and AI on modern data architectures like Hadoop and Spark
• Enabling queries, reports, business intelligence or analytics without production impact
• Feeding real-time data to employee, customer or partner applications
• Keeping data from siloed databases in sync
• Reducing the impact of database maintenance, backup or testing
• Re-platforming to new database or operating systems
• Consolidating databases
12
Goals for Change Data Capture
Strategies for CDC
Which Change Data Capture Strategy Is Right for You?14
Timestamps or Version Numbers
Advantages
• Simple
• Nearly every database can query
with a where clause.
Disadvantages
• Must be built into database
• Bloats database size
• Query requires considerable compute resources in source database
• Not always reliable
Which Change Data Capture Strategy Is Right for You?15
Table Triggers
Advantages
• Very reliable and detailed
• Changes can be captured, almost as fast as they are
made – real-time CDC.
Disadvantages
• Significant drag on database resources, both
compute and storage.
• Requires that the database have the capability.
• Negative impact on performance of applications that
depend on the source database.
Which Change Data Capture Strategy Is Right for You?16
Snapshot or Table Comparison
Advantages
• Relatively easy to implement with
good ETL software.
• Requires no specialized knowledge
of the source database.
• Very dependable and accurate.
Disadvantages
• Requires repeatedly moving all data in monitored tables. May impact
target or staging system resources and network bandwidth.
• Moving lots of data can be slow, may not meet SLA’s.
• Joining, comparing, and finding changes may also take time. Even
slower.
• Not a complete record of intermediate changes between snapshot
captures.
Which Change Data Capture Strategy Is Right for You?17
Log Scraping
Advantages
• Very reliable and detailed.
• Virtually no impact on database or application
performance.
• Changes captured in real-time.
• No database bloat.
Disadvantages
• Every RDMS has a different log format, often not
documented.
• Log formats often change between RDBMS
versions.
• Log files are frequently archived by the database.
CDC software must read them before they’re
archived, or be able to go read the archived logs.
• Requires specialized CDC software. Cannot be
easily accomplished with ETL software.
• Can fail if connectivity is lost on source or target,
causing lost data, duplicated data, or need to
restart from initial data load.
CDC with Syncsort
19
Syncsort DMX & DMX-h:
Simple and Powerful Big Data Integration Software
Syncsort Data Integration and Data Quality for the Cloud
DMX
• GUI for developing MapReduce & Spark jobs
• Test & debug locally in Windows; deploy on Hadoop
• Use-case Accelerators to fast-track development
• Broad based connectivity with automated parallelism
• Simply the best mainframe access and integration with Hadoop
• Improved per node scalability and throughput
High Performance
ETL Software
• Template driven design for:
o High performance ETL
o SQL migration/DB offload
o Mainframe data movement
• Light weight footprint on commodity hardware
• High speed flat file processing
• Self tuning engine
High Performance
Hadoop ETL SoftwareDMX-h
DMX Change Data Capture
Keep data in sync in real-time
• Without overloading networks.
• Without affecting source database
performance.
• Without coding or tuning.
Reliable transfer of data you can trust even if connectivity fails on either side.
• Auto restart.
• No data loss.
Real-Time Replication
with Transformation
Conflict Resolution,
Collision Monitoring,
Tracking and Auditing
Files
RDBMS
Streams
Streams
RDBMS
Data
Lake
Mainframe
Cloud
OLAP
DMX Change Data Capture Sources and Targets
SOURCES
• IBM Db2/z
• IBM Db2/i
• IBM Db2/LUW
• VSAM
• Kafka
• Oracle
• Oracle RAC
Real Application
Clusters
• MS SQL Server
• IBM Informix
• Sybase
TARGETS
• Kafka
• Amazon Kinesis
• Teradata
• HDFS
• Hive
(HDFS, ORC, Avro, Parquet)
• Impala
(Parquet, Kudu)
• IBM Db2
• SQL Server
• MS Azure SQL
• PostgreSQL
• MySQL
• Oracle
• Oracle RAC
• Sybase
• And more …
Real-Time Replication
with Transformation
Conflict Resolution,
Collision Monitoring,
Tracking and Auditing
Files
RDBMS
Streams
Streams
RDBMS
Data Hub
Mainframe
Cloud
OLAP
22
Design Once, Deploy Anywhere
Syncsort Data Integration and Data Quality for the Cloud
Intelligent Execution - Insulate your organization from underlying complexities of Hadoop.
Get excellent performance every time
without tuning, load balancing, etc.
No re-design, re-compile, no re-work ever
• Future-proof job designs for emerging compute
frameworks, e.g. Spark 2.x
• Move from development to test to production
• Move from on-premise to Cloud
• Move from one Cloud to another
Use existing ETL and data quality skills
No parallel programming – Java, MapReduce, Spark …
No worries about:
• Mappers, Reducers
• Big side or small side of joins …
Design Once
in visual GUI
Deploy Anywhere!
On-Premise,
Cloud
Mapreduce, Spark,
Future Platforms
Windows, Unix,
Linux
Batch,
Streaming
Single Node,
Cluster
Which Change Data Capture Strategy Is Right for You?23
Snapshot CDC with DMX/DMX-h
• Captures database changes on a
scheduled basis
• High speed sort and join
• Transforms and enhances data
during replication
• Supplies end-to-end lineage of data
for compliance, auditing
• Any source, any target, not limited
to sources with logging
• Fast development in template-
based GUI
• Latency – Usually hourly to weekly
Integration in
the Cloud with
DMX ETL
“DMX allows Dickey’s to rapidly
collect, transform and load
thousands of very large files, with
diverse data types from multiple
servers across all of Dickey’s
locations, without performance
bottlenecks.”
Laura Rea, Dickey’s, CIO
24
Modernize antiquated, Excel-based
Point of Sales system analytics.
Must function with minimal on-site
infrastructure and support personnel.
• Standardize software across 500+ stores.
• 1000’sof large files
• Diverse data types – financial, operations,
inventory, purchasing
• DMX ETL
• AWS cloud-based architecture designed and
implemented by iOLAP.
• Rapid job development in visual interface – no
hand coding or scripts to maintain.
• Everyday operations data available to non-
technical business users.
AWS Cloud scales with project needs
– Dickeys pays for only what they use
Redshift updated every 15-20
minutes for quick, easy, current data-
driven business insights.
Better reporting and analytics =
more dollars saved and earned.
SOLUTION:
25
Log-Based Anything to Hadoop
• Real-time capture
• Minimizes bandwidth usage with LAN/WAN
friendly replication
• Parallel load on cluster
• Updates HDFS, Hive or Impala, backed by HDFS,
Parquet, ORC, or Kudu.
• Updates even versions of Hive that did not
support updating
• Latency – Minutes (less than 5)
Real-Time Replication
with Transformation
Conflict Resolution,
Collision Monitoring,
Tracking and Auditing
Data
Lake
Cloud
Files
RDBMS
Streams
Mainframe
Case Study:
Guardian Life Insurance
"We found DMX-h to be very usable and
easy to ramp up in terms of skills. Most
of all, Syncsort has been a very good
partner in terms of support and listening
to our needs.“
– Alex Rosenthal, Enterprise Data Office
CHALLENGE
• Enable visualization and BI on broad range of data sets.
• Reduce data preparation, transformation times
• Reduce time-to-market for analytics projects.
• Make data assets available to whole enterprise – including Mainframe.
SOLUTION
• Created Amazon-style data marketplace, supported by data lake,
Hadoop, NoSQL. New projects reuse and build upon existing
data assets. DMX-h adds new data to the Data Lake with
each new project.
• DMX DataFunnel quickly ingested hundreds of database
tables at push of a button
• DMX Change Data Capture pushes changes from DB2 to the
data lake in real-time. Current data up-to-the minute.
BENEFITS
• Centralized standardized reusable data assets –
searchable, accessible and managed.
• DMX-h and DataFunnel accelerated
data acquisition, reduced time to
market for analytics and reporting.
27
Anything to Stream, or Stream to Anything
• Real-time capture
• Minimizes bandwidth usage with LAN/WAN
friendly replication
• Parallel load on cluster
• Updates HDFS, Hive or Impala, backed by
HDFS, Parquet, ORC, or Kudu.
• Updates even versions of Hive that did not
support updating
• Latency – Real-time, actual SLA varies
depending on update speed of target,
stream settings, etc. Usually, seconds.
Real-Time Replication
with Transformation
Conflict Resolution,
Collision Monitoring,
Tracking and Auditing
Files
RDBMS
Streams
Streams
RDBMS
Data
Lake
Mainframe
Cloud
OLAP
Case Study:
Global Hotel Data Kept Current On the Cloud
Syncsort Data Integration and Data Quality for the Cloud28
C H A L L E N G E
• More timely collection & reporting on room availability, event bookings,
inventory and other hotel data from 4,000+ properties globally
S O LU T I O N
• Near real-time reporting - DMX-h consumes property updates from Kafka
every 10 seconds
• DMX-h processes data on HDP, loading to Teradata every 30 minutes
• Deployed on Google Cloud Platform
• Productivity: Leveraging ETL team for Hadoop
(Spark), visual understanding of data pipeline
• Insight: Up-to-date data = better business decisions
= happier customers
B E N E F I T S
• Time to Value: DMX-h ease of use drastically cut development time
• Agility: Global reports updated every 30 min – before 24 hours
29
Log-Based Database to Database
• Captures database changes as they happen
• Transforms and enhances data during replication
• Minimizes bandwidth usage with LAN/WAN
friendly replication
• Ensures data integrity with conflict resolution
and collision monitoring
• Enables tracking and auditing of transactions for
compliance
• Latency – sub-second
Real-Time Replication
with Transformation
Conflict Resolution,
Collision Monitoring,
Tracking and Auditing
RDBMS
RDBMS
OLAP
Centralized Reporting Use Case
Casino 1
IBM i Db2
Casino 2 Casino 3 Casino 4 Casino 5 Casino 6
Single Data Warehouse Database
Windows Cluster
MS SQL Server
Business intelligence
Real time CDC replication
with transformation
• Customer loyalty
• Amounts paid
• Amounts won
• Time at the table
• Time at the machine
IBM i Db2 IBM i Db2 IBM i Db2 IBM i Db2 IBM i Db2
Gradual Database Re-Platforming Use Case
IBM i
Db2
Old System
Windows
SQL Server
New System
America II Corp
Active-Active replication eliminated need
for hard cutover and enabled partners to
move back and forth between systems
True zero downtime for
migration to new systems
Transformation between
different OS and database
platforms with completely
different schemas 100’s of partners moved to
new server after training at
their own pace
Syncsort Addresses All Your
Data Sharing Needs
✓ Enables centralization or consolidation of data
✓ Facilitates machine learning, advanced analytics and AI
✓ Facilitates real-time queries, reporting and business intelligence
✓ Transforms data for smooth data flow between databases
✓ Keeps distributed applications and data in sync
✓ Feeds real-time data to mission critical applications
✓ Offloads data for maintenance, testing and backup
✓ Migrates legacy data to new platforms
✓ And more!
33
Which Change Data Capture Strategy is Right for You?

Weitere ähnliche Inhalte

Was ist angesagt?

Data Lakehouse Symposium | Day 4
Data Lakehouse Symposium | Day 4Data Lakehouse Symposium | Day 4
Data Lakehouse Symposium | Day 4Databricks
 
Time to Talk about Data Mesh
Time to Talk about Data MeshTime to Talk about Data Mesh
Time to Talk about Data MeshLibbySchulze
 
Introducing the Snowflake Computing Cloud Data Warehouse
Introducing the Snowflake Computing Cloud Data WarehouseIntroducing the Snowflake Computing Cloud Data Warehouse
Introducing the Snowflake Computing Cloud Data WarehouseSnowflake Computing
 
Data Lakehouse Symposium | Day 1 | Part 2
Data Lakehouse Symposium | Day 1 | Part 2Data Lakehouse Symposium | Day 1 | Part 2
Data Lakehouse Symposium | Day 1 | Part 2Databricks
 
Modern Data architecture Design
Modern Data architecture DesignModern Data architecture Design
Modern Data architecture DesignKujambu Murugesan
 
Data Warehousing Trends, Best Practices, and Future Outlook
Data Warehousing Trends, Best Practices, and Future OutlookData Warehousing Trends, Best Practices, and Future Outlook
Data Warehousing Trends, Best Practices, and Future OutlookJames Serra
 
Modern Data Warehousing with the Microsoft Analytics Platform System
Modern Data Warehousing with the Microsoft Analytics Platform SystemModern Data Warehousing with the Microsoft Analytics Platform System
Modern Data Warehousing with the Microsoft Analytics Platform SystemJames Serra
 
Snowflake Data Science and AI/ML at Scale
Snowflake Data Science and AI/ML at ScaleSnowflake Data Science and AI/ML at Scale
Snowflake Data Science and AI/ML at ScaleAdam Doyle
 
Data Driven Advanced Analytics using Denodo Platform on AWS
Data Driven Advanced Analytics using Denodo Platform on AWSData Driven Advanced Analytics using Denodo Platform on AWS
Data Driven Advanced Analytics using Denodo Platform on AWSDenodo
 
Intro to Delta Lake
Intro to Delta LakeIntro to Delta Lake
Intro to Delta LakeDatabricks
 
Data lineage and observability with Marquez - subsurface 2020
Data lineage and observability with Marquez - subsurface 2020Data lineage and observability with Marquez - subsurface 2020
Data lineage and observability with Marquez - subsurface 2020Julien Le Dem
 
Data Lakehouse, Data Mesh, and Data Fabric (r2)
Data Lakehouse, Data Mesh, and Data Fabric (r2)Data Lakehouse, Data Mesh, and Data Fabric (r2)
Data Lakehouse, Data Mesh, and Data Fabric (r2)James Serra
 
Data Management vs Data Strategy
Data Management vs Data StrategyData Management vs Data Strategy
Data Management vs Data StrategyDATAVERSITY
 
Learn to Use Databricks for Data Science
Learn to Use Databricks for Data ScienceLearn to Use Databricks for Data Science
Learn to Use Databricks for Data ScienceDatabricks
 
Demystifying Data Warehousing as a Service - DFW
Demystifying Data Warehousing as a Service - DFWDemystifying Data Warehousing as a Service - DFW
Demystifying Data Warehousing as a Service - DFWKent Graziano
 
Changing the game with cloud dw
Changing the game with cloud dwChanging the game with cloud dw
Changing the game with cloud dwelephantscale
 
Zero to Snowflake Presentation
Zero to Snowflake Presentation Zero to Snowflake Presentation
Zero to Snowflake Presentation Brett VanderPlaats
 

Was ist angesagt? (20)

Data Lakehouse Symposium | Day 4
Data Lakehouse Symposium | Day 4Data Lakehouse Symposium | Day 4
Data Lakehouse Symposium | Day 4
 
Data Mesh
Data MeshData Mesh
Data Mesh
 
Time to Talk about Data Mesh
Time to Talk about Data MeshTime to Talk about Data Mesh
Time to Talk about Data Mesh
 
Introducing the Snowflake Computing Cloud Data Warehouse
Introducing the Snowflake Computing Cloud Data WarehouseIntroducing the Snowflake Computing Cloud Data Warehouse
Introducing the Snowflake Computing Cloud Data Warehouse
 
Data Lakehouse Symposium | Day 1 | Part 2
Data Lakehouse Symposium | Day 1 | Part 2Data Lakehouse Symposium | Day 1 | Part 2
Data Lakehouse Symposium | Day 1 | Part 2
 
Snowflake Overview
Snowflake OverviewSnowflake Overview
Snowflake Overview
 
Modern Data architecture Design
Modern Data architecture DesignModern Data architecture Design
Modern Data architecture Design
 
Data Warehousing Trends, Best Practices, and Future Outlook
Data Warehousing Trends, Best Practices, and Future OutlookData Warehousing Trends, Best Practices, and Future Outlook
Data Warehousing Trends, Best Practices, and Future Outlook
 
Modern Data Warehousing with the Microsoft Analytics Platform System
Modern Data Warehousing with the Microsoft Analytics Platform SystemModern Data Warehousing with the Microsoft Analytics Platform System
Modern Data Warehousing with the Microsoft Analytics Platform System
 
Snowflake Data Science and AI/ML at Scale
Snowflake Data Science and AI/ML at ScaleSnowflake Data Science and AI/ML at Scale
Snowflake Data Science and AI/ML at Scale
 
Data Driven Advanced Analytics using Denodo Platform on AWS
Data Driven Advanced Analytics using Denodo Platform on AWSData Driven Advanced Analytics using Denodo Platform on AWS
Data Driven Advanced Analytics using Denodo Platform on AWS
 
Intro to Delta Lake
Intro to Delta LakeIntro to Delta Lake
Intro to Delta Lake
 
Data lineage and observability with Marquez - subsurface 2020
Data lineage and observability with Marquez - subsurface 2020Data lineage and observability with Marquez - subsurface 2020
Data lineage and observability with Marquez - subsurface 2020
 
Data Lakehouse, Data Mesh, and Data Fabric (r2)
Data Lakehouse, Data Mesh, and Data Fabric (r2)Data Lakehouse, Data Mesh, and Data Fabric (r2)
Data Lakehouse, Data Mesh, and Data Fabric (r2)
 
Data Management vs Data Strategy
Data Management vs Data StrategyData Management vs Data Strategy
Data Management vs Data Strategy
 
Learn to Use Databricks for Data Science
Learn to Use Databricks for Data ScienceLearn to Use Databricks for Data Science
Learn to Use Databricks for Data Science
 
Demystifying Data Warehousing as a Service - DFW
Demystifying Data Warehousing as a Service - DFWDemystifying Data Warehousing as a Service - DFW
Demystifying Data Warehousing as a Service - DFW
 
Changing the game with cloud dw
Changing the game with cloud dwChanging the game with cloud dw
Changing the game with cloud dw
 
Zero to Snowflake Presentation
Zero to Snowflake Presentation Zero to Snowflake Presentation
Zero to Snowflake Presentation
 
Informatica Cloud Overview
Informatica Cloud OverviewInformatica Cloud Overview
Informatica Cloud Overview
 

Ähnlich wie Which Change Data Capture Strategy is Right for You?

Hadoop in 2015: Keys to Achieving Operational Excellence for the Real-Time En...
Hadoop in 2015: Keys to Achieving Operational Excellence for the Real-Time En...Hadoop in 2015: Keys to Achieving Operational Excellence for the Real-Time En...
Hadoop in 2015: Keys to Achieving Operational Excellence for the Real-Time En...MapR Technologies
 
ADV Slides: Platforming Your Data for Success – Databases, Hadoop, Managed Ha...
ADV Slides: Platforming Your Data for Success – Databases, Hadoop, Managed Ha...ADV Slides: Platforming Your Data for Success – Databases, Hadoop, Managed Ha...
ADV Slides: Platforming Your Data for Success – Databases, Hadoop, Managed Ha...DATAVERSITY
 
Data Warehouse Optimization
Data Warehouse OptimizationData Warehouse Optimization
Data Warehouse OptimizationCloudera, Inc.
 
Engineering Machine Learning Data Pipelines Series: Streaming New Data as It ...
Engineering Machine Learning Data Pipelines Series: Streaming New Data as It ...Engineering Machine Learning Data Pipelines Series: Streaming New Data as It ...
Engineering Machine Learning Data Pipelines Series: Streaming New Data as It ...Precisely
 
ADV Slides: When and How Data Lakes Fit into a Modern Data Architecture
ADV Slides: When and How Data Lakes Fit into a Modern Data ArchitectureADV Slides: When and How Data Lakes Fit into a Modern Data Architecture
ADV Slides: When and How Data Lakes Fit into a Modern Data ArchitectureDATAVERSITY
 
Skillwise Big Data part 2
Skillwise Big Data part 2Skillwise Big Data part 2
Skillwise Big Data part 2Skillwise Group
 
Bring Your SAP and Enterprise Data to Hadoop, Kafka, and the Cloud
Bring Your SAP and Enterprise Data to Hadoop, Kafka, and the CloudBring Your SAP and Enterprise Data to Hadoop, Kafka, and the Cloud
Bring Your SAP and Enterprise Data to Hadoop, Kafka, and the CloudDataWorks Summit
 
The Future of Data Warehousing: ETL Will Never be the Same
The Future of Data Warehousing: ETL Will Never be the SameThe Future of Data Warehousing: ETL Will Never be the Same
The Future of Data Warehousing: ETL Will Never be the SameCloudera, Inc.
 
The Shifting Landscape of Data Integration
The Shifting Landscape of Data IntegrationThe Shifting Landscape of Data Integration
The Shifting Landscape of Data IntegrationDATAVERSITY
 
DAMA & Denodo Webinar: Modernizing Data Architecture Using Data Virtualization
DAMA & Denodo Webinar: Modernizing Data Architecture Using Data Virtualization DAMA & Denodo Webinar: Modernizing Data Architecture Using Data Virtualization
DAMA & Denodo Webinar: Modernizing Data Architecture Using Data Virtualization Denodo
 
Hadoop and SQL: Delivery Analytics Across the Organization
Hadoop and SQL:  Delivery Analytics Across the OrganizationHadoop and SQL:  Delivery Analytics Across the Organization
Hadoop and SQL: Delivery Analytics Across the OrganizationSeeling Cheung
 
Horses for Courses: Database Roundtable
Horses for Courses: Database RoundtableHorses for Courses: Database Roundtable
Horses for Courses: Database RoundtableEric Kavanagh
 
Data Con LA 2018 - Populating your Enterprise Data Hub for Next Gen Analytics...
Data Con LA 2018 - Populating your Enterprise Data Hub for Next Gen Analytics...Data Con LA 2018 - Populating your Enterprise Data Hub for Next Gen Analytics...
Data Con LA 2018 - Populating your Enterprise Data Hub for Next Gen Analytics...Data Con LA
 
Fueling AI & Machine Learning: Legacy Data as a Competitive Advantage
Fueling AI & Machine Learning: Legacy Data as a Competitive AdvantageFueling AI & Machine Learning: Legacy Data as a Competitive Advantage
Fueling AI & Machine Learning: Legacy Data as a Competitive AdvantagePrecisely
 
Traditional BI vs. Business Data Lake – A Comparison
Traditional BI vs. Business Data Lake – A ComparisonTraditional BI vs. Business Data Lake – A Comparison
Traditional BI vs. Business Data Lake – A ComparisonCapgemini
 
Iod session 3423 analytics patterns of expertise, the fast path to amazing ...
Iod session 3423   analytics patterns of expertise, the fast path to amazing ...Iod session 3423   analytics patterns of expertise, the fast path to amazing ...
Iod session 3423 analytics patterns of expertise, the fast path to amazing ...Rachel Bland
 
WP_Impetus_2016_Guide_to_Modernize_Your_Enterprise_Data_Warehouse_JRoberts
WP_Impetus_2016_Guide_to_Modernize_Your_Enterprise_Data_Warehouse_JRobertsWP_Impetus_2016_Guide_to_Modernize_Your_Enterprise_Data_Warehouse_JRoberts
WP_Impetus_2016_Guide_to_Modernize_Your_Enterprise_Data_Warehouse_JRobertsJane Roberts
 
Using the Power of Big SQL 3.0 to Build a Big Data-Ready Hybrid Warehouse
Using the Power of Big SQL 3.0 to Build a Big Data-Ready Hybrid WarehouseUsing the Power of Big SQL 3.0 to Build a Big Data-Ready Hybrid Warehouse
Using the Power of Big SQL 3.0 to Build a Big Data-Ready Hybrid WarehouseRizaldy Ignacio
 

Ähnlich wie Which Change Data Capture Strategy is Right for You? (20)

Hadoop in 2015: Keys to Achieving Operational Excellence for the Real-Time En...
Hadoop in 2015: Keys to Achieving Operational Excellence for the Real-Time En...Hadoop in 2015: Keys to Achieving Operational Excellence for the Real-Time En...
Hadoop in 2015: Keys to Achieving Operational Excellence for the Real-Time En...
 
ADV Slides: Platforming Your Data for Success – Databases, Hadoop, Managed Ha...
ADV Slides: Platforming Your Data for Success – Databases, Hadoop, Managed Ha...ADV Slides: Platforming Your Data for Success – Databases, Hadoop, Managed Ha...
ADV Slides: Platforming Your Data for Success – Databases, Hadoop, Managed Ha...
 
Data Warehouse Optimization
Data Warehouse OptimizationData Warehouse Optimization
Data Warehouse Optimization
 
Hadoop and Your Enterprise Data Warehouse
Hadoop and Your Enterprise Data WarehouseHadoop and Your Enterprise Data Warehouse
Hadoop and Your Enterprise Data Warehouse
 
Engineering Machine Learning Data Pipelines Series: Streaming New Data as It ...
Engineering Machine Learning Data Pipelines Series: Streaming New Data as It ...Engineering Machine Learning Data Pipelines Series: Streaming New Data as It ...
Engineering Machine Learning Data Pipelines Series: Streaming New Data as It ...
 
ADV Slides: When and How Data Lakes Fit into a Modern Data Architecture
ADV Slides: When and How Data Lakes Fit into a Modern Data ArchitectureADV Slides: When and How Data Lakes Fit into a Modern Data Architecture
ADV Slides: When and How Data Lakes Fit into a Modern Data Architecture
 
Skillwise Big Data part 2
Skillwise Big Data part 2Skillwise Big Data part 2
Skillwise Big Data part 2
 
Skilwise Big data
Skilwise Big dataSkilwise Big data
Skilwise Big data
 
Bring Your SAP and Enterprise Data to Hadoop, Kafka, and the Cloud
Bring Your SAP and Enterprise Data to Hadoop, Kafka, and the CloudBring Your SAP and Enterprise Data to Hadoop, Kafka, and the Cloud
Bring Your SAP and Enterprise Data to Hadoop, Kafka, and the Cloud
 
The Future of Data Warehousing: ETL Will Never be the Same
The Future of Data Warehousing: ETL Will Never be the SameThe Future of Data Warehousing: ETL Will Never be the Same
The Future of Data Warehousing: ETL Will Never be the Same
 
The Shifting Landscape of Data Integration
The Shifting Landscape of Data IntegrationThe Shifting Landscape of Data Integration
The Shifting Landscape of Data Integration
 
DAMA & Denodo Webinar: Modernizing Data Architecture Using Data Virtualization
DAMA & Denodo Webinar: Modernizing Data Architecture Using Data Virtualization DAMA & Denodo Webinar: Modernizing Data Architecture Using Data Virtualization
DAMA & Denodo Webinar: Modernizing Data Architecture Using Data Virtualization
 
Hadoop and SQL: Delivery Analytics Across the Organization
Hadoop and SQL:  Delivery Analytics Across the OrganizationHadoop and SQL:  Delivery Analytics Across the Organization
Hadoop and SQL: Delivery Analytics Across the Organization
 
Horses for Courses: Database Roundtable
Horses for Courses: Database RoundtableHorses for Courses: Database Roundtable
Horses for Courses: Database Roundtable
 
Data Con LA 2018 - Populating your Enterprise Data Hub for Next Gen Analytics...
Data Con LA 2018 - Populating your Enterprise Data Hub for Next Gen Analytics...Data Con LA 2018 - Populating your Enterprise Data Hub for Next Gen Analytics...
Data Con LA 2018 - Populating your Enterprise Data Hub for Next Gen Analytics...
 
Fueling AI & Machine Learning: Legacy Data as a Competitive Advantage
Fueling AI & Machine Learning: Legacy Data as a Competitive AdvantageFueling AI & Machine Learning: Legacy Data as a Competitive Advantage
Fueling AI & Machine Learning: Legacy Data as a Competitive Advantage
 
Traditional BI vs. Business Data Lake – A Comparison
Traditional BI vs. Business Data Lake – A ComparisonTraditional BI vs. Business Data Lake – A Comparison
Traditional BI vs. Business Data Lake – A Comparison
 
Iod session 3423 analytics patterns of expertise, the fast path to amazing ...
Iod session 3423   analytics patterns of expertise, the fast path to amazing ...Iod session 3423   analytics patterns of expertise, the fast path to amazing ...
Iod session 3423 analytics patterns of expertise, the fast path to amazing ...
 
WP_Impetus_2016_Guide_to_Modernize_Your_Enterprise_Data_Warehouse_JRoberts
WP_Impetus_2016_Guide_to_Modernize_Your_Enterprise_Data_Warehouse_JRobertsWP_Impetus_2016_Guide_to_Modernize_Your_Enterprise_Data_Warehouse_JRoberts
WP_Impetus_2016_Guide_to_Modernize_Your_Enterprise_Data_Warehouse_JRoberts
 
Using the Power of Big SQL 3.0 to Build a Big Data-Ready Hybrid Warehouse
Using the Power of Big SQL 3.0 to Build a Big Data-Ready Hybrid WarehouseUsing the Power of Big SQL 3.0 to Build a Big Data-Ready Hybrid Warehouse
Using the Power of Big SQL 3.0 to Build a Big Data-Ready Hybrid Warehouse
 

Mehr von Precisely

Justifying Capacity Managment Webinar 4/10
Justifying Capacity Managment Webinar 4/10Justifying Capacity Managment Webinar 4/10
Justifying Capacity Managment Webinar 4/10Precisely
 
Automate Studio Training: Materials Maintenance Tips for Efficiency and Ease ...
Automate Studio Training: Materials Maintenance Tips for Efficiency and Ease ...Automate Studio Training: Materials Maintenance Tips for Efficiency and Ease ...
Automate Studio Training: Materials Maintenance Tips for Efficiency and Ease ...Precisely
 
Leveraging Mainframe Data in Near Real Time to Unleash Innovation With Cloud:...
Leveraging Mainframe Data in Near Real Time to Unleash Innovation With Cloud:...Leveraging Mainframe Data in Near Real Time to Unleash Innovation With Cloud:...
Leveraging Mainframe Data in Near Real Time to Unleash Innovation With Cloud:...Precisely
 
Testjrjnejrvnorno4rno3nrfnfjnrfnournfou3nfou3f
Testjrjnejrvnorno4rno3nrfnfjnrfnournfou3nfou3fTestjrjnejrvnorno4rno3nrfnfjnrfnournfou3nfou3f
Testjrjnejrvnorno4rno3nrfnfjnrfnournfou3nfou3fPrecisely
 
Data Innovation Summit: Data Integrity Trends
Data Innovation Summit: Data Integrity TrendsData Innovation Summit: Data Integrity Trends
Data Innovation Summit: Data Integrity TrendsPrecisely
 
AI You Can Trust - Ensuring Success with Data Integrity Webinar
AI You Can Trust - Ensuring Success with Data Integrity WebinarAI You Can Trust - Ensuring Success with Data Integrity Webinar
AI You Can Trust - Ensuring Success with Data Integrity WebinarPrecisely
 
Optimisez la fonction financière en automatisant vos processus SAP
Optimisez la fonction financière en automatisant vos processus SAPOptimisez la fonction financière en automatisant vos processus SAP
Optimisez la fonction financière en automatisant vos processus SAPPrecisely
 
SAPS/4HANA Migration - Transformation-Management + nachhaltige Investitionen
SAPS/4HANA Migration - Transformation-Management + nachhaltige InvestitionenSAPS/4HANA Migration - Transformation-Management + nachhaltige Investitionen
SAPS/4HANA Migration - Transformation-Management + nachhaltige InvestitionenPrecisely
 
Automatisierte SAP Prozesse mit Hilfe von APIs
Automatisierte SAP Prozesse mit Hilfe von APIsAutomatisierte SAP Prozesse mit Hilfe von APIs
Automatisierte SAP Prozesse mit Hilfe von APIsPrecisely
 
Moving IBM i Applications to the Cloud with AWS and Precisely
Moving IBM i Applications to the Cloud with AWS and PreciselyMoving IBM i Applications to the Cloud with AWS and Precisely
Moving IBM i Applications to the Cloud with AWS and PreciselyPrecisely
 
Effective Security Monitoring for IBM i: What You Need to Know
Effective Security Monitoring for IBM i: What You Need to KnowEffective Security Monitoring for IBM i: What You Need to Know
Effective Security Monitoring for IBM i: What You Need to KnowPrecisely
 
Automate Your Master Data Processes for Shared Service Center Excellence
Automate Your Master Data Processes for Shared Service Center ExcellenceAutomate Your Master Data Processes for Shared Service Center Excellence
Automate Your Master Data Processes for Shared Service Center ExcellencePrecisely
 
5 Keys to Improved IT Operation Management
5 Keys to Improved IT Operation Management5 Keys to Improved IT Operation Management
5 Keys to Improved IT Operation ManagementPrecisely
 
Unlock Efficiency With Your Address Data Today For a Smarter Tomorrow
Unlock Efficiency With Your Address Data Today For a Smarter TomorrowUnlock Efficiency With Your Address Data Today For a Smarter Tomorrow
Unlock Efficiency With Your Address Data Today For a Smarter TomorrowPrecisely
 
Navigating Cloud Trends in 2024 Webinar Deck
Navigating Cloud Trends in 2024 Webinar DeckNavigating Cloud Trends in 2024 Webinar Deck
Navigating Cloud Trends in 2024 Webinar DeckPrecisely
 
Mainframe Sort Operations: Gaining the Insights You Need for Peak Performance
Mainframe Sort Operations: Gaining the Insights You Need for Peak PerformanceMainframe Sort Operations: Gaining the Insights You Need for Peak Performance
Mainframe Sort Operations: Gaining the Insights You Need for Peak PerformancePrecisely
 
Preventing Downtime with Better IT Operations Management
Preventing Downtime with Better IT Operations ManagementPreventing Downtime with Better IT Operations Management
Preventing Downtime with Better IT Operations ManagementPrecisely
 
Migrating IBM i Systems to the Cloud: Exploring the Pros and Cons
Migrating IBM i Systems to the Cloud: Exploring the Pros and ConsMigrating IBM i Systems to the Cloud: Exploring the Pros and Cons
Migrating IBM i Systems to the Cloud: Exploring the Pros and ConsPrecisely
 
Americas Marketing Campaigns Tracker1234
Americas Marketing Campaigns Tracker1234Americas Marketing Campaigns Tracker1234
Americas Marketing Campaigns Tracker1234Precisely
 
9,000 Ways to Optimize Outcomes in Financial Services
9,000 Ways to Optimize Outcomes in Financial Services9,000 Ways to Optimize Outcomes in Financial Services
9,000 Ways to Optimize Outcomes in Financial ServicesPrecisely
 

Mehr von Precisely (20)

Justifying Capacity Managment Webinar 4/10
Justifying Capacity Managment Webinar 4/10Justifying Capacity Managment Webinar 4/10
Justifying Capacity Managment Webinar 4/10
 
Automate Studio Training: Materials Maintenance Tips for Efficiency and Ease ...
Automate Studio Training: Materials Maintenance Tips for Efficiency and Ease ...Automate Studio Training: Materials Maintenance Tips for Efficiency and Ease ...
Automate Studio Training: Materials Maintenance Tips for Efficiency and Ease ...
 
Leveraging Mainframe Data in Near Real Time to Unleash Innovation With Cloud:...
Leveraging Mainframe Data in Near Real Time to Unleash Innovation With Cloud:...Leveraging Mainframe Data in Near Real Time to Unleash Innovation With Cloud:...
Leveraging Mainframe Data in Near Real Time to Unleash Innovation With Cloud:...
 
Testjrjnejrvnorno4rno3nrfnfjnrfnournfou3nfou3f
Testjrjnejrvnorno4rno3nrfnfjnrfnournfou3nfou3fTestjrjnejrvnorno4rno3nrfnfjnrfnournfou3nfou3f
Testjrjnejrvnorno4rno3nrfnfjnrfnournfou3nfou3f
 
Data Innovation Summit: Data Integrity Trends
Data Innovation Summit: Data Integrity TrendsData Innovation Summit: Data Integrity Trends
Data Innovation Summit: Data Integrity Trends
 
AI You Can Trust - Ensuring Success with Data Integrity Webinar
AI You Can Trust - Ensuring Success with Data Integrity WebinarAI You Can Trust - Ensuring Success with Data Integrity Webinar
AI You Can Trust - Ensuring Success with Data Integrity Webinar
 
Optimisez la fonction financière en automatisant vos processus SAP
Optimisez la fonction financière en automatisant vos processus SAPOptimisez la fonction financière en automatisant vos processus SAP
Optimisez la fonction financière en automatisant vos processus SAP
 
SAPS/4HANA Migration - Transformation-Management + nachhaltige Investitionen
SAPS/4HANA Migration - Transformation-Management + nachhaltige InvestitionenSAPS/4HANA Migration - Transformation-Management + nachhaltige Investitionen
SAPS/4HANA Migration - Transformation-Management + nachhaltige Investitionen
 
Automatisierte SAP Prozesse mit Hilfe von APIs
Automatisierte SAP Prozesse mit Hilfe von APIsAutomatisierte SAP Prozesse mit Hilfe von APIs
Automatisierte SAP Prozesse mit Hilfe von APIs
 
Moving IBM i Applications to the Cloud with AWS and Precisely
Moving IBM i Applications to the Cloud with AWS and PreciselyMoving IBM i Applications to the Cloud with AWS and Precisely
Moving IBM i Applications to the Cloud with AWS and Precisely
 
Effective Security Monitoring for IBM i: What You Need to Know
Effective Security Monitoring for IBM i: What You Need to KnowEffective Security Monitoring for IBM i: What You Need to Know
Effective Security Monitoring for IBM i: What You Need to Know
 
Automate Your Master Data Processes for Shared Service Center Excellence
Automate Your Master Data Processes for Shared Service Center ExcellenceAutomate Your Master Data Processes for Shared Service Center Excellence
Automate Your Master Data Processes for Shared Service Center Excellence
 
5 Keys to Improved IT Operation Management
5 Keys to Improved IT Operation Management5 Keys to Improved IT Operation Management
5 Keys to Improved IT Operation Management
 
Unlock Efficiency With Your Address Data Today For a Smarter Tomorrow
Unlock Efficiency With Your Address Data Today For a Smarter TomorrowUnlock Efficiency With Your Address Data Today For a Smarter Tomorrow
Unlock Efficiency With Your Address Data Today For a Smarter Tomorrow
 
Navigating Cloud Trends in 2024 Webinar Deck
Navigating Cloud Trends in 2024 Webinar DeckNavigating Cloud Trends in 2024 Webinar Deck
Navigating Cloud Trends in 2024 Webinar Deck
 
Mainframe Sort Operations: Gaining the Insights You Need for Peak Performance
Mainframe Sort Operations: Gaining the Insights You Need for Peak PerformanceMainframe Sort Operations: Gaining the Insights You Need for Peak Performance
Mainframe Sort Operations: Gaining the Insights You Need for Peak Performance
 
Preventing Downtime with Better IT Operations Management
Preventing Downtime with Better IT Operations ManagementPreventing Downtime with Better IT Operations Management
Preventing Downtime with Better IT Operations Management
 
Migrating IBM i Systems to the Cloud: Exploring the Pros and Cons
Migrating IBM i Systems to the Cloud: Exploring the Pros and ConsMigrating IBM i Systems to the Cloud: Exploring the Pros and Cons
Migrating IBM i Systems to the Cloud: Exploring the Pros and Cons
 
Americas Marketing Campaigns Tracker1234
Americas Marketing Campaigns Tracker1234Americas Marketing Campaigns Tracker1234
Americas Marketing Campaigns Tracker1234
 
9,000 Ways to Optimize Outcomes in Financial Services
9,000 Ways to Optimize Outcomes in Financial Services9,000 Ways to Optimize Outcomes in Financial Services
9,000 Ways to Optimize Outcomes in Financial Services
 

Kürzlich hochgeladen

Glenn Lazarus- Why Your Observability Strategy Needs Security Observability
Glenn Lazarus- Why Your Observability Strategy Needs Security ObservabilityGlenn Lazarus- Why Your Observability Strategy Needs Security Observability
Glenn Lazarus- Why Your Observability Strategy Needs Security Observabilityitnewsafrica
 
Transcript: Green paths: Learning from publishers’ sustainability journeys - ...
Transcript: Green paths: Learning from publishers’ sustainability journeys - ...Transcript: Green paths: Learning from publishers’ sustainability journeys - ...
Transcript: Green paths: Learning from publishers’ sustainability journeys - ...BookNet Canada
 
Genislab builds better products and faster go-to-market with Lean project man...
Genislab builds better products and faster go-to-market with Lean project man...Genislab builds better products and faster go-to-market with Lean project man...
Genislab builds better products and faster go-to-market with Lean project man...Farhan Tariq
 
Laying the Data Foundations for Artificial Intelligence!
Laying the Data Foundations for Artificial Intelligence!Laying the Data Foundations for Artificial Intelligence!
Laying the Data Foundations for Artificial Intelligence!Memoori
 
Email Marketing Automation for Bonterra Impact Management (fka Social Solutio...
Email Marketing Automation for Bonterra Impact Management (fka Social Solutio...Email Marketing Automation for Bonterra Impact Management (fka Social Solutio...
Email Marketing Automation for Bonterra Impact Management (fka Social Solutio...Jeffrey Haguewood
 
Abdul Kader Baba- Managing Cybersecurity Risks and Compliance Requirements i...
Abdul Kader Baba- Managing Cybersecurity Risks  and Compliance Requirements i...Abdul Kader Baba- Managing Cybersecurity Risks  and Compliance Requirements i...
Abdul Kader Baba- Managing Cybersecurity Risks and Compliance Requirements i...itnewsafrica
 
Landscape Catalogue 2024 Australia-1.pdf
Landscape Catalogue 2024 Australia-1.pdfLandscape Catalogue 2024 Australia-1.pdf
Landscape Catalogue 2024 Australia-1.pdfAarwolf Industries LLC
 
2024 April Patch Tuesday
2024 April Patch Tuesday2024 April Patch Tuesday
2024 April Patch TuesdayIvanti
 
Unleashing Real-time Insights with ClickHouse_ Navigating the Landscape in 20...
Unleashing Real-time Insights with ClickHouse_ Navigating the Landscape in 20...Unleashing Real-time Insights with ClickHouse_ Navigating the Landscape in 20...
Unleashing Real-time Insights with ClickHouse_ Navigating the Landscape in 20...Alkin Tezuysal
 
Design pattern talk by Kaya Weers - 2024 (v2)
Design pattern talk by Kaya Weers - 2024 (v2)Design pattern talk by Kaya Weers - 2024 (v2)
Design pattern talk by Kaya Weers - 2024 (v2)Kaya Weers
 
Deliver Latency Free Customer Experience
Deliver Latency Free Customer ExperienceDeliver Latency Free Customer Experience
Deliver Latency Free Customer ExperienceOpsTree solutions
 
Transcript: New from BookNet Canada for 2024: BNC SalesData and LibraryData -...
Transcript: New from BookNet Canada for 2024: BNC SalesData and LibraryData -...Transcript: New from BookNet Canada for 2024: BNC SalesData and LibraryData -...
Transcript: New from BookNet Canada for 2024: BNC SalesData and LibraryData -...BookNet Canada
 
Introduction-to-Wazuh-and-its-integration.pptx
Introduction-to-Wazuh-and-its-integration.pptxIntroduction-to-Wazuh-and-its-integration.pptx
Introduction-to-Wazuh-and-its-integration.pptxmprakaash5
 
Dublin_mulesoft_meetup_API_specifications.pptx
Dublin_mulesoft_meetup_API_specifications.pptxDublin_mulesoft_meetup_API_specifications.pptx
Dublin_mulesoft_meetup_API_specifications.pptxKunal Gupta
 
QCon London: Mastering long-running processes in modern architectures
QCon London: Mastering long-running processes in modern architecturesQCon London: Mastering long-running processes in modern architectures
QCon London: Mastering long-running processes in modern architecturesBernd Ruecker
 
Potential of AI (Generative AI) in Business: Learnings and Insights
Potential of AI (Generative AI) in Business: Learnings and InsightsPotential of AI (Generative AI) in Business: Learnings and Insights
Potential of AI (Generative AI) in Business: Learnings and InsightsRavi Sanghani
 
The Future Roadmap for the Composable Data Stack - Wes McKinney - Data Counci...
The Future Roadmap for the Composable Data Stack - Wes McKinney - Data Counci...The Future Roadmap for the Composable Data Stack - Wes McKinney - Data Counci...
The Future Roadmap for the Composable Data Stack - Wes McKinney - Data Counci...Wes McKinney
 
Women in Automation 2024: Career session - explore career paths in automation
Women in Automation 2024: Career session - explore career paths in automationWomen in Automation 2024: Career session - explore career paths in automation
Women in Automation 2024: Career session - explore career paths in automationDianaGray10
 
Accelerating Enterprise Software Engineering with Platformless
Accelerating Enterprise Software Engineering with PlatformlessAccelerating Enterprise Software Engineering with Platformless
Accelerating Enterprise Software Engineering with PlatformlessWSO2
 
Generative Artificial Intelligence: How generative AI works.pdf
Generative Artificial Intelligence: How generative AI works.pdfGenerative Artificial Intelligence: How generative AI works.pdf
Generative Artificial Intelligence: How generative AI works.pdfIngrid Airi González
 

Kürzlich hochgeladen (20)

Glenn Lazarus- Why Your Observability Strategy Needs Security Observability
Glenn Lazarus- Why Your Observability Strategy Needs Security ObservabilityGlenn Lazarus- Why Your Observability Strategy Needs Security Observability
Glenn Lazarus- Why Your Observability Strategy Needs Security Observability
 
Transcript: Green paths: Learning from publishers’ sustainability journeys - ...
Transcript: Green paths: Learning from publishers’ sustainability journeys - ...Transcript: Green paths: Learning from publishers’ sustainability journeys - ...
Transcript: Green paths: Learning from publishers’ sustainability journeys - ...
 
Genislab builds better products and faster go-to-market with Lean project man...
Genislab builds better products and faster go-to-market with Lean project man...Genislab builds better products and faster go-to-market with Lean project man...
Genislab builds better products and faster go-to-market with Lean project man...
 
Laying the Data Foundations for Artificial Intelligence!
Laying the Data Foundations for Artificial Intelligence!Laying the Data Foundations for Artificial Intelligence!
Laying the Data Foundations for Artificial Intelligence!
 
Email Marketing Automation for Bonterra Impact Management (fka Social Solutio...
Email Marketing Automation for Bonterra Impact Management (fka Social Solutio...Email Marketing Automation for Bonterra Impact Management (fka Social Solutio...
Email Marketing Automation for Bonterra Impact Management (fka Social Solutio...
 
Abdul Kader Baba- Managing Cybersecurity Risks and Compliance Requirements i...
Abdul Kader Baba- Managing Cybersecurity Risks  and Compliance Requirements i...Abdul Kader Baba- Managing Cybersecurity Risks  and Compliance Requirements i...
Abdul Kader Baba- Managing Cybersecurity Risks and Compliance Requirements i...
 
Landscape Catalogue 2024 Australia-1.pdf
Landscape Catalogue 2024 Australia-1.pdfLandscape Catalogue 2024 Australia-1.pdf
Landscape Catalogue 2024 Australia-1.pdf
 
2024 April Patch Tuesday
2024 April Patch Tuesday2024 April Patch Tuesday
2024 April Patch Tuesday
 
Unleashing Real-time Insights with ClickHouse_ Navigating the Landscape in 20...
Unleashing Real-time Insights with ClickHouse_ Navigating the Landscape in 20...Unleashing Real-time Insights with ClickHouse_ Navigating the Landscape in 20...
Unleashing Real-time Insights with ClickHouse_ Navigating the Landscape in 20...
 
Design pattern talk by Kaya Weers - 2024 (v2)
Design pattern talk by Kaya Weers - 2024 (v2)Design pattern talk by Kaya Weers - 2024 (v2)
Design pattern talk by Kaya Weers - 2024 (v2)
 
Deliver Latency Free Customer Experience
Deliver Latency Free Customer ExperienceDeliver Latency Free Customer Experience
Deliver Latency Free Customer Experience
 
Transcript: New from BookNet Canada for 2024: BNC SalesData and LibraryData -...
Transcript: New from BookNet Canada for 2024: BNC SalesData and LibraryData -...Transcript: New from BookNet Canada for 2024: BNC SalesData and LibraryData -...
Transcript: New from BookNet Canada for 2024: BNC SalesData and LibraryData -...
 
Introduction-to-Wazuh-and-its-integration.pptx
Introduction-to-Wazuh-and-its-integration.pptxIntroduction-to-Wazuh-and-its-integration.pptx
Introduction-to-Wazuh-and-its-integration.pptx
 
Dublin_mulesoft_meetup_API_specifications.pptx
Dublin_mulesoft_meetup_API_specifications.pptxDublin_mulesoft_meetup_API_specifications.pptx
Dublin_mulesoft_meetup_API_specifications.pptx
 
QCon London: Mastering long-running processes in modern architectures
QCon London: Mastering long-running processes in modern architecturesQCon London: Mastering long-running processes in modern architectures
QCon London: Mastering long-running processes in modern architectures
 
Potential of AI (Generative AI) in Business: Learnings and Insights
Potential of AI (Generative AI) in Business: Learnings and InsightsPotential of AI (Generative AI) in Business: Learnings and Insights
Potential of AI (Generative AI) in Business: Learnings and Insights
 
The Future Roadmap for the Composable Data Stack - Wes McKinney - Data Counci...
The Future Roadmap for the Composable Data Stack - Wes McKinney - Data Counci...The Future Roadmap for the Composable Data Stack - Wes McKinney - Data Counci...
The Future Roadmap for the Composable Data Stack - Wes McKinney - Data Counci...
 
Women in Automation 2024: Career session - explore career paths in automation
Women in Automation 2024: Career session - explore career paths in automationWomen in Automation 2024: Career session - explore career paths in automation
Women in Automation 2024: Career session - explore career paths in automation
 
Accelerating Enterprise Software Engineering with Platformless
Accelerating Enterprise Software Engineering with PlatformlessAccelerating Enterprise Software Engineering with Platformless
Accelerating Enterprise Software Engineering with Platformless
 
Generative Artificial Intelligence: How generative AI works.pdf
Generative Artificial Intelligence: How generative AI works.pdfGenerative Artificial Intelligence: How generative AI works.pdf
Generative Artificial Intelligence: How generative AI works.pdf
 

Which Change Data Capture Strategy is Right for You?

  • 1. Which Change Data Capture Strategy Is Right for You? Presented by Paige Roberts Sr. Product Marketing Manager Data Integration, Data Quality 1
  • 2. Choosing a Change Data Capture Strategy 1 What is Change Data Capture? 2 Why Do Change Data Capture? 3 Strategies for Change Data Capture 4 Examples of Change Data Capture 5 Q and A
  • 3. CDC is the process that ensures that changes made over time in one dataset are automatically transferred to another dataset. Change Data Capture or CDC is most often used with databases that hold important transactional data to make sure that organizations are working with up-to-date information across the enterprise. Source - often used to record transactions or other business occurrences as they happen. Target - often used to create a report or do analysis to determine a course of action. Sometimes, data is replicated bi-directionally so that a source is also a target and vice versa. Which Change Data Capture Strategy Is Right for You?3 What is Change Data Capture?
  • 4. Replication Options One Way Two Way Cascade Bi-Directional Distribute Consolidate Choose a topology or combine them to meet your data sharing needs
  • 5. 5 Integrated Architecture Use Case ERP SYSTEM Customer Orders Payment Details Product Catalogue Price List eCOMMERCE & WEB PORTALS TEST & AUDIT ENVIRONMENT DATA EXCHANGE WITH OUTSIDE VENDOR (FLAT FILE) DR / BACKUP
  • 6. 6 Customer Example Architecture EDGE NODE CLUSTER DATA NODES DATABASE SOURCES MAINFRAME SOURCES VSAM Db2 CAPTURE AGENT
  • 8. 1. Businesses have Multiple Databases Multiple databases are the norm • Merger or acquisition • Choice of multiple apps or databases for best of breed solutions • Combination of legacy and new databases • Multi-organization supply chain IT infrastructures are heterogeneous • Database platforms • Operating systems • Hardware 8 Drivers Behind Change Data Capture 83% 10% 8% Does your organization rely on multiple databases? Yes No I don't know. 73%of those with multiple databases share data among them Does your organization share data between multiple databases? Source: Vision Solutions ‘ 2017 State of Resilience Report
  • 9. 2. Enabling Analytics, Reporting and BI • Protecting performance of production database by offloading data to a reporting system for queries, reports, business intelligence or analytics • Consolidating data into centralized databases, data marts or data warehouses for decision making or business processing Which Change Data Capture Strategy Is Right for You?9 Drivers Behind Change Data Capture
  • 10. 3. Enabling Machine Learning, Advanced Analytics and AI • Growing data volumes lead to new architectures for data consolidation – data lakes and enterprise data hubs based on Hadoop or Spark. • New types of data and larger amounts of data from multiple sources combined together create an ideal environment for training and employing machine learning and artificial intelligence. • Businesses across many industries seek competitive edge from these new technologies in use cases from fraud detection to targeted marketing. • ML and AI systems have a constant, voracious need for more data, and must constantly have the latest, most current data available to provide the promised insights. Which Change Data Capture Strategy Is Right for You?10 Drivers Behind Change Data Capture
  • 11. 4. Varied Business and IT Goals • Offloading data for maintenance, backup, or testing on a secondary system without production impact • Maintaining synchronization between siloed databases or branch offices • Feeding segmented data to customer or partner applications • Migrating data to new databases • Re-platforming databases to new database or operating system platforms 11 Drivers Behind Change Data Capture Source: Vision Solutions ‘ 2017 State of Resilience Report For what business purpose does your organization share data between databases? Consolidating data from multiple sources into… Reporting on data offloaded from the… Synchronizing data between distributed… Testing on offloaded data Running business processes on offloaded data I don’t know 0% 10% 20% 30% 40% 50% 60% 70%
  • 12. Why do you need to capture and move the changes in your data? • Populating centralized databases, data marts, data warehouses, or data lakes • Enabling machine learning, advanced analytics and AI on modern data architectures like Hadoop and Spark • Enabling queries, reports, business intelligence or analytics without production impact • Feeding real-time data to employee, customer or partner applications • Keeping data from siloed databases in sync • Reducing the impact of database maintenance, backup or testing • Re-platforming to new database or operating systems • Consolidating databases 12 Goals for Change Data Capture
  • 14. Which Change Data Capture Strategy Is Right for You?14 Timestamps or Version Numbers Advantages • Simple • Nearly every database can query with a where clause. Disadvantages • Must be built into database • Bloats database size • Query requires considerable compute resources in source database • Not always reliable
  • 15. Which Change Data Capture Strategy Is Right for You?15 Table Triggers Advantages • Very reliable and detailed • Changes can be captured, almost as fast as they are made – real-time CDC. Disadvantages • Significant drag on database resources, both compute and storage. • Requires that the database have the capability. • Negative impact on performance of applications that depend on the source database.
  • 16. Which Change Data Capture Strategy Is Right for You?16 Snapshot or Table Comparison Advantages • Relatively easy to implement with good ETL software. • Requires no specialized knowledge of the source database. • Very dependable and accurate. Disadvantages • Requires repeatedly moving all data in monitored tables. May impact target or staging system resources and network bandwidth. • Moving lots of data can be slow, may not meet SLA’s. • Joining, comparing, and finding changes may also take time. Even slower. • Not a complete record of intermediate changes between snapshot captures.
  • 17. Which Change Data Capture Strategy Is Right for You?17 Log Scraping Advantages • Very reliable and detailed. • Virtually no impact on database or application performance. • Changes captured in real-time. • No database bloat. Disadvantages • Every RDMS has a different log format, often not documented. • Log formats often change between RDBMS versions. • Log files are frequently archived by the database. CDC software must read them before they’re archived, or be able to go read the archived logs. • Requires specialized CDC software. Cannot be easily accomplished with ETL software. • Can fail if connectivity is lost on source or target, causing lost data, duplicated data, or need to restart from initial data load.
  • 19. 19 Syncsort DMX & DMX-h: Simple and Powerful Big Data Integration Software Syncsort Data Integration and Data Quality for the Cloud DMX • GUI for developing MapReduce & Spark jobs • Test & debug locally in Windows; deploy on Hadoop • Use-case Accelerators to fast-track development • Broad based connectivity with automated parallelism • Simply the best mainframe access and integration with Hadoop • Improved per node scalability and throughput High Performance ETL Software • Template driven design for: o High performance ETL o SQL migration/DB offload o Mainframe data movement • Light weight footprint on commodity hardware • High speed flat file processing • Self tuning engine High Performance Hadoop ETL SoftwareDMX-h
  • 20. DMX Change Data Capture Keep data in sync in real-time • Without overloading networks. • Without affecting source database performance. • Without coding or tuning. Reliable transfer of data you can trust even if connectivity fails on either side. • Auto restart. • No data loss. Real-Time Replication with Transformation Conflict Resolution, Collision Monitoring, Tracking and Auditing Files RDBMS Streams Streams RDBMS Data Lake Mainframe Cloud OLAP
  • 21. DMX Change Data Capture Sources and Targets SOURCES • IBM Db2/z • IBM Db2/i • IBM Db2/LUW • VSAM • Kafka • Oracle • Oracle RAC Real Application Clusters • MS SQL Server • IBM Informix • Sybase TARGETS • Kafka • Amazon Kinesis • Teradata • HDFS • Hive (HDFS, ORC, Avro, Parquet) • Impala (Parquet, Kudu) • IBM Db2 • SQL Server • MS Azure SQL • PostgreSQL • MySQL • Oracle • Oracle RAC • Sybase • And more … Real-Time Replication with Transformation Conflict Resolution, Collision Monitoring, Tracking and Auditing Files RDBMS Streams Streams RDBMS Data Hub Mainframe Cloud OLAP
  • 22. 22 Design Once, Deploy Anywhere Syncsort Data Integration and Data Quality for the Cloud Intelligent Execution - Insulate your organization from underlying complexities of Hadoop. Get excellent performance every time without tuning, load balancing, etc. No re-design, re-compile, no re-work ever • Future-proof job designs for emerging compute frameworks, e.g. Spark 2.x • Move from development to test to production • Move from on-premise to Cloud • Move from one Cloud to another Use existing ETL and data quality skills No parallel programming – Java, MapReduce, Spark … No worries about: • Mappers, Reducers • Big side or small side of joins … Design Once in visual GUI Deploy Anywhere! On-Premise, Cloud Mapreduce, Spark, Future Platforms Windows, Unix, Linux Batch, Streaming Single Node, Cluster
  • 23. Which Change Data Capture Strategy Is Right for You?23 Snapshot CDC with DMX/DMX-h • Captures database changes on a scheduled basis • High speed sort and join • Transforms and enhances data during replication • Supplies end-to-end lineage of data for compliance, auditing • Any source, any target, not limited to sources with logging • Fast development in template- based GUI • Latency – Usually hourly to weekly
  • 24. Integration in the Cloud with DMX ETL “DMX allows Dickey’s to rapidly collect, transform and load thousands of very large files, with diverse data types from multiple servers across all of Dickey’s locations, without performance bottlenecks.” Laura Rea, Dickey’s, CIO 24 Modernize antiquated, Excel-based Point of Sales system analytics. Must function with minimal on-site infrastructure and support personnel. • Standardize software across 500+ stores. • 1000’sof large files • Diverse data types – financial, operations, inventory, purchasing • DMX ETL • AWS cloud-based architecture designed and implemented by iOLAP. • Rapid job development in visual interface – no hand coding or scripts to maintain. • Everyday operations data available to non- technical business users. AWS Cloud scales with project needs – Dickeys pays for only what they use Redshift updated every 15-20 minutes for quick, easy, current data- driven business insights. Better reporting and analytics = more dollars saved and earned. SOLUTION:
  • 25. 25 Log-Based Anything to Hadoop • Real-time capture • Minimizes bandwidth usage with LAN/WAN friendly replication • Parallel load on cluster • Updates HDFS, Hive or Impala, backed by HDFS, Parquet, ORC, or Kudu. • Updates even versions of Hive that did not support updating • Latency – Minutes (less than 5) Real-Time Replication with Transformation Conflict Resolution, Collision Monitoring, Tracking and Auditing Data Lake Cloud Files RDBMS Streams Mainframe
  • 26. Case Study: Guardian Life Insurance "We found DMX-h to be very usable and easy to ramp up in terms of skills. Most of all, Syncsort has been a very good partner in terms of support and listening to our needs.“ – Alex Rosenthal, Enterprise Data Office CHALLENGE • Enable visualization and BI on broad range of data sets. • Reduce data preparation, transformation times • Reduce time-to-market for analytics projects. • Make data assets available to whole enterprise – including Mainframe. SOLUTION • Created Amazon-style data marketplace, supported by data lake, Hadoop, NoSQL. New projects reuse and build upon existing data assets. DMX-h adds new data to the Data Lake with each new project. • DMX DataFunnel quickly ingested hundreds of database tables at push of a button • DMX Change Data Capture pushes changes from DB2 to the data lake in real-time. Current data up-to-the minute. BENEFITS • Centralized standardized reusable data assets – searchable, accessible and managed. • DMX-h and DataFunnel accelerated data acquisition, reduced time to market for analytics and reporting.
  • 27. 27 Anything to Stream, or Stream to Anything • Real-time capture • Minimizes bandwidth usage with LAN/WAN friendly replication • Parallel load on cluster • Updates HDFS, Hive or Impala, backed by HDFS, Parquet, ORC, or Kudu. • Updates even versions of Hive that did not support updating • Latency – Real-time, actual SLA varies depending on update speed of target, stream settings, etc. Usually, seconds. Real-Time Replication with Transformation Conflict Resolution, Collision Monitoring, Tracking and Auditing Files RDBMS Streams Streams RDBMS Data Lake Mainframe Cloud OLAP
  • 28. Case Study: Global Hotel Data Kept Current On the Cloud Syncsort Data Integration and Data Quality for the Cloud28 C H A L L E N G E • More timely collection & reporting on room availability, event bookings, inventory and other hotel data from 4,000+ properties globally S O LU T I O N • Near real-time reporting - DMX-h consumes property updates from Kafka every 10 seconds • DMX-h processes data on HDP, loading to Teradata every 30 minutes • Deployed on Google Cloud Platform • Productivity: Leveraging ETL team for Hadoop (Spark), visual understanding of data pipeline • Insight: Up-to-date data = better business decisions = happier customers B E N E F I T S • Time to Value: DMX-h ease of use drastically cut development time • Agility: Global reports updated every 30 min – before 24 hours
  • 29. 29 Log-Based Database to Database • Captures database changes as they happen • Transforms and enhances data during replication • Minimizes bandwidth usage with LAN/WAN friendly replication • Ensures data integrity with conflict resolution and collision monitoring • Enables tracking and auditing of transactions for compliance • Latency – sub-second Real-Time Replication with Transformation Conflict Resolution, Collision Monitoring, Tracking and Auditing RDBMS RDBMS OLAP
  • 30. Centralized Reporting Use Case Casino 1 IBM i Db2 Casino 2 Casino 3 Casino 4 Casino 5 Casino 6 Single Data Warehouse Database Windows Cluster MS SQL Server Business intelligence Real time CDC replication with transformation • Customer loyalty • Amounts paid • Amounts won • Time at the table • Time at the machine IBM i Db2 IBM i Db2 IBM i Db2 IBM i Db2 IBM i Db2
  • 31. Gradual Database Re-Platforming Use Case IBM i Db2 Old System Windows SQL Server New System America II Corp Active-Active replication eliminated need for hard cutover and enabled partners to move back and forth between systems True zero downtime for migration to new systems Transformation between different OS and database platforms with completely different schemas 100’s of partners moved to new server after training at their own pace
  • 32. Syncsort Addresses All Your Data Sharing Needs ✓ Enables centralization or consolidation of data ✓ Facilitates machine learning, advanced analytics and AI ✓ Facilitates real-time queries, reporting and business intelligence ✓ Transforms data for smooth data flow between databases ✓ Keeps distributed applications and data in sync ✓ Feeds real-time data to mission critical applications ✓ Offloads data for maintenance, testing and backup ✓ Migrates legacy data to new platforms ✓ And more!
  • 33. 33