This session will describe and demonstrate the longstanding integration between Couchbase Server and Apache Kafka and will include descriptions of both the mechanics of the integration and practical situations when combining these products is appropriate.

1. Confidential and Proprietary. Do not distribute without Couchbase consent. © Couchbase 2020. All rights reserved.
THE COUCHBASE &
CONFLUENT BRIDGE
FROM RELATIONAL TO NOSQL
July 14, 2020
Jeff Morris, VP Product & Solutions Marketing

2. BIG DATA
ARCHITECTURE
EVOLUTION1

3. Confidential and Proprietary. Do not distribute without
Couchbase consent. © Couchbase 2020. All rights reserved. 3
Big Data
Architecture
Evolution
Big Data version 1 focused on
• ingest
• archive

4. Confidential and Proprietary. Do not distribute without
Couchbase consent. © Couchbase 2020. All rights reserved. 4
Big Data
Architecture
Evolution
Big Data version 2 focused on
• collate
• analyze

5. Confidential and Proprietary. Do not distribute without
Couchbase consent. © Couchbase 2020. All rights reserved. 5
Big Data
Architecture
Evolution
Big Data version 3 ...
• stream
• remix

6. Confidential and Proprietary. Do not distribute without
Couchbase consent. © Couchbase 2020. All rights reserved. 6
Big Data
Architecture
Evolution
Big Data version 3 ...
• stream
• remix
• ... engage

7. Confidential and Proprietary. Do not distribute without
Couchbase consent. © Couchbase 2020. All rights reserved. 7
Big Data
Architecture
Evolution
Big Data version 3 ...
• stream
• remix
• ... engage

8. Confidential and Proprietary. Do not distribute without Couchbase consent. © Couchbase 2020. All rights reserved. 8

9. Confidential and Proprietary. Do not distribute without
Couchbase consent. © Couchbase 2017. All rights
reserved. 9
Data
Processing
Platform
• Critical connectivity
• Analysis
• Streaming
HDFS
DBMS
Mobile
Other
Platforms
Data
Processing
Platform

10. Confidential and Proprietary. Do not distribute without
Couchbase consent. © Couchbase 2017. All rights
reserved. 10
Stream
Data
Platform
• Critical connectivity
• Analysis
• Streaming
• Distributed
• High throughput
• Wider engagement
HDFS
DBMS
Mobile
Other
Platforms
Stream
Data
Platform

11. KAFKA AS A
STREAMING BUS
2

12. Confidential and Proprietary. Do not distribute without
Couchbase consent. © Couchbase 2017. All rights
reserved. 12
Couchbase
& Kafka
• Source
• Sink
• Custom Filter
• Apache Kafka
• Confluent Platform
Kafka
?

13. Confidential and Proprietary. Do not distribute without Couchbase consent. © Couchbase 2020. All rights reserved. 13
How Organizations Handle Data Flow: A Giant Mess
Data
Warehouse
Hadoop
NoSQL
Oracle
SFDC
Logging
Bloomberg
… any sink/source
… and more
OLTP
ActiveMQ
App App
Caches
OLTP OLTPAppAppApp
Web Custom Apps Microservices Monitoring Analytics

14. Confidential and Proprietary. Do not distribute without Couchbase consent. © Couchbase 2020. All rights reserved. 14
Kafka Connect

15. Confidential and Proprietary. Do not distribute without Couchbase consent. © Couchbase 2020. All rights reserved. 15
The Couchbase Kafka Connect Plug-In
Reads Data
Connect Protocol
stream from
Couchbase and
publishes to
Kafka topic
Consumes Kafka topic
and writes records to
Couchbase

16. Confidential and Proprietary. Do not distribute without Couchbase consent. © Couchbase 2020. All rights reserved. 16
Apache Kafka™: A Distributed Streaming Platform
Apache Kafka
Data
Warehouse
HadoopOracle
SFDC
Twitter
Bloomberg
… any sink/source … any sink/source
… and more
Web Custom Apps Microservices Monitoring Analytics

17. REAL WORLD
USE CASE
3

18. Confidential and Proprietary. Do not distribute without Couchbase consent. © Couchbase 2020. All rights reserved. 18
Applications Across Industries
Healthcare & Pharma
Patient Monitoring, Pharma
Substance control, Patient
Relapse, Lab Results Alerts
Banking & Capital Markets
Fraud Detection, Trade Data Capture,
Customer 360
Retail
Inventory Management, Product
Catalog, A/B Testing, Proactive
Alerts
Telecommunications
Personalized Ads, Customer 360,
Network Integrity
Automotive
Connected Car, Manufacturing
Data Processing
Travel & Leisure
Visitor Segmentation,
Fraud Detection

19. Confidential and Proprietary. Do not distribute without Couchbase consent. © Couchbase 2020. All rights reserved. 19
Couchbase Data Sharing Use Cases
Catalog and
Inventory Management
Catalogs
• Deliver relevant product
content and a real-time
view of inventory
• Scale to millions of
products and requests
for the latest information
• Serves highly engaged
online audiences
Profile and
Session Management
Personalization
• Create custom experiences in
real time based on aggregate
data from multiple
• Aggregate customer data,
recommendations, user
profiles, session, history data
Customer 360
Single View
• Deliver a consistent, single
view of your data with one
platform
• Improve customer experience
and operational visibility
Digital
Transformations
Offload
• Create transformative digital
experiences by offloading
mainframe, RDBMS systems
• Reduce costs and improve
productivity and agility

20. Confidential and Proprietary. Do not distribute without
Couchbase consent. © Couchbase 2020. All rights reserved. 20
Confluent
Platform
Enterprise
Streaming based
on Apache
Kafka™
Database
Changes
Log
Events
loT Data
Web
Events
…
CRM
Data
Warehouse
Database
Hadoop
Data
Integration
…
Monitoring
Analytics
Custom
Apps
Transformations
Real-time
Applications
…
Apache Open
Source
Confluent Open
Source
Confluent
Enterprise
Confluent Platform
Apache Kafka™
Data Compatibility
Monitoring & Administration
Operations
Clients Connectors
Complete Open Trusted Enterprise Grade

21. Confidential and Proprietary. Do not distribute without Couchbase consent. © Couchbase 2020. All rights reserved. 21
Bank Reduces OpEx by $25M/year via Mainframe Offload
Date Amount
1/27/2017 $4.56
1/22/2017 $32.14
Transaction Data
Vendor Description
Starbucks Coffee
Walmart Blu-Ray
Transaction Description
Schema
Website
Microservices
Match data and
description
Client profiles
Lookup client
profiles
Mainframe MIPS = $$

22. Confidential and Proprietary. Do not distribute without Couchbase consent. © Couchbase 2020. All rights reserved. 22
Ingest, Process, Load, and Serve Data at a Global Scale
Couchbase
…
Couchbase
…
Kafka cluster
Applications
Other data
stores
Kafka cluster
Kafka Streams
(Data Enrichment and Transformation)
Kafka Connect
(Connectors to Extract and Load data)
Confluent
Replicator
Confluent
Replicator
Custom
Replication
Custom
Replication
Raw event
data

23. Confidential and Proprietary. Do not distribute without Couchbase consent. © Couchbase 2020. All rights reserved. 23
Couchbase’s NoEQUAL Architectural Differentiation
Developer agility & versatility
• Multi-model: Key Value & JSON documents
• Multi-mode: Memory-first, ACID, operational & analytic workloads
• Programmable: schema flexibility + SQL in N1QL + stack-based SDKs
Performance at any scale
• No hassle scale out – shared-nothing, asynchronous, elastic architecture
• Built-in replication (XDCR)
• Always-on, globally distributed, edge-to-cloud
Easy to manage
• Workload isolation with multi-dimensional scaling
• Automatic cluster rebalancing
• Location and deployment agnostic
• Kubernetes & microservices-friendly

24. Confidential and Proprietary. Do not distribute without Couchbase consent. © Couchbase 2020. All rights reserved. 24
Sample QA SetupSample Dev Setup
Elastic Scaling Architecture
Sample Production DeploymentNODE 1
Query
Global Index
Data
Analytics
Full Text
Cluster Manager
NODE 2
Eventing
NODE 1 NODE 13
Cluster Manager
Data Full Text AnalyticsGlobal Index Query Eventing
NODE 1
Global IndexQuery
Full Text
Analytics
Data
Cluster Manager
NODE 4
Eventing
Flexible cluster topology adjusts with growing demand

25. DEMO
4

26. Confidential and Proprietary. Do not distribute without Couchbase consent. © Couchbase 2020. All rights reserved. 26 26
Customers spend
more time
interacting than
transacting…

27. Confidential and Proprietary. Do not distribute without Couchbase consent. © Couchbase 2020. All rights reserved. 27
Couchbase Behind Today’s Business-Critical Applications
Application
Customers
Infrastructure
ManageabilityPerformance at Scale
Performance
Flight availability,
booking, pricing
analytics, etc.
15M
ops / second
<2.5ms
response time
Open source identity
management
1B
full-trip, web
authentications
per day
13.6k transactions
per second
Customer 360 single
view, unified notes
210M
documents
100K
users
Developer Agility
Caching & session
store for single view
2M+
reads/sec.
10M
queries/sec.
Real-time pricing,
product catalog,
inventory management
10M+
unique SKUs
35K
requests/sec.
Security + Availability

28. Confidential and Proprietary. Do not distribute without Couchbase consent. © Couchbase 2020. All rights reserved. 28
A Proven Enterprise Solution Chosen by Industry Leaders
3 of the Top 10
eCommerce
Companies
6 of the Top 10
Broadcast
Companies
Retail & E-
Commerce
3 of the Top 3
GDS
Companies
Travel &
Hospitality
Telecom
6 of the Top 10
Online Casino
Gaming Companies
Gaming
3 of the Top 3
Credit Reporting
Companies
Financial
Services
3 Fortune 500
Healthcare
Companies
Healthcare
Media &
Entertainment
2 of the Top 2
IoT
Platforms
Industrial IoT
Confidential and Proprietary. Do not distribute without Couchbase consent. © Couchbase 2019. All rights reserved.

29. Confidential and Proprietary. Do not distribute without Couchbase consent. © Couchbase 2020. All rights reserved.
Thank You