SlideShare ist ein Scribd-Unternehmen logo
1 von 36
Downloaden Sie, um offline zu lesen
1
Why Build an Apache Kafka® Connector
Series - Building Kafka Connectors - the Why and How
Part 1 -Why Build a Kafka Connector
Part 2 -How to Build a Kafka Connector
2
Speakers
Sree Karuthody, Sr. Manager,
Technology Partnerships
Jeff Bean,
Partner Solution Architect
Sid Rabindran,
Dir. Partner Programs and
Tech Ecosystem Partners
Lisa Sensmeier,
Partner Marketing
3
What is Event Streaming
Kafka and Connecting to Kafka
Value of building a connector
Verified Integrations Program
Q and A
Agenda
Key business drivers
behind connecting to
Kafka
The Verified Integrations
Program, benefits, how
to participate, process
44
What is Event Streaming
55
The New Enterprise Reality
Innovate or be
disrupted
● Deliver new, real-time
customer experiences
● Create new business
models
● Deliver massive internal
efficiencies & risk reduction
Every company is a
software company
● Capital One
10,000 of 40,000 employees
are software engineers
● Goldman Sachs
1.5B lines of code across
7,000+ applications
Innovation is about
events, not “data”
● Event = something happened
● Your business = a continually
updating stream of events
● Your success = your ability to
respond to these events
66
The Rise of Event Streaming
Data as a continuous stream of events
60%
Fortune 100 Companies
Using Apache Kafka
77
Auto / Transport
Without Event Streaming With Event Streaming
Event Streaming Enables New Outcomes
Call for driver availability
No knowledge of driver arrival
No data on feature usage
Real-time driver-rider match
Real-time ETA
Real-time sensor diagnostics
Banking Nightly updated account balance
Batch fraud checks
Batch regulatory reporting
Real-time account updates
Real-time credit card fraud alerts
Real-time regulatory reporting
Retail Post-order “out of stock” emails
No upsell through personalization
Batch point-of sale reports
Real-time inventory
Real-time recommendations
Real-time sales reporting
88
Confluent is Enabling Event-driven
Transformation across Industries
Healthcare
& Pharma
● Patient monitoring
● Prescription control
● Lab alerts
● Medication tracking
Banking &
Capital Markets
● Customer 360
● Fraud detection
● Mainframe offload
● Trade data capture
Retail
● Inventory management
● Product catalog
● A/B testing
● Customized experiences
Telecommunications
● Personalized ads
● Customer 360
● Network integrity
Automotive &
Transportation
● Connected car
● Fleet management
● Manufacturing data
processing
Travel & Leisure
● Visitor segmentation
● Booking systems
● Pricing services
● Fraud detection
99
Why Confluent
1010
Confluent Enables Your Event Streaming Success
Confluent founders are
original creators of
Kafka
Confluent team wrote 80% of
Kafka commits
Confluent Platform
extends Apache Kafka to
be a secure,
enterprise-ready platform
Confluent helps enterprises
successfully deploy event
streaming at scale and
accelerate time to market
Hall of Innovation
CTO Innovation
Award Winner
2019
11
Confluent Platform
Operations and Security
Development & Stream Processing
Support,services,training&partners
Apache Kafka
Security plugins | Role-Based Access Control
Control Center | Replicator | Auto Data Balancer | Operator
Connectors
Clients | REST Proxy
MQTT Proxy | Schema Registry
KSQL
Connect Continuous Commit Log Streams
Complete Event
Streaming Platform
Mission-critical
Reliability
Freedom of Choice
Datacenter Public Cloud Confluent Cloud
Self-Managed Software Fully-Managed Service
12
Real-Time
Inventory
Real-Time
Fraud
Detection
Real-Time
Customer 360
Machine
Learning
Models
Real-Time
Data
Transformation
...
Contextual Event-Driven Applications
Universal Event Pipeline
Data Stores Logs 3rd Party Apps Custom Apps/Microservices
TREAMSSTREAMS
CONNECT CLIENTS
With
Confluent
1414
Kafka and
Connecting to Kafka
15
Without Kafka and Confluent
16
● End to end monitoring and analytics
● Enterprise-grade security
● Multi-datacenter replication
● SQL-based stream processing
● Cloud-native deployment with K8s operator
● Run as self-managed software or as fully
managed service with Confluent Cloud
Enterprise streaming platform built by the original creators of Apache Kafka
Edge
Cloud
Data LakeDatabases
Datacenter
IoT
SaaS AppsMobile
Microservices Machine
Learning
Confluent Platform
17
Deploy and Stream with Confluent on Any Cloud
Confluent Platform
The Enterprise Distribution of Apache Kafka
Deploy on any platform on-premises or in public clouds
Fully-Managed Service
Confluent Cloud
Apache Kafka Re-engineered for the Cloud
Available on the leading public clouds
VM
Self-Managed Software
1818
Apache Kafka™ Connect API – Streaming Data Capture
JDBC
Mongo
MySQL
Elastic
Cassandra
HDFS
Kafka Connect API
Kafka Pipeline
Connector
Connector
Connector
Connector
Connector
Connector
Sources Sinks
Fault tolerant
Manage hundreds of
data sources and sinks
Preserves data schema
Part of Apache Kafka
project
Integrated within
Confluent Platform’s
Control Center
19
Confluent Hub
Connectors are posted
- Verified Partner
Integrations
- Confluent provided
and supported
confluent.io/hub
20
2121
Value of building a connector
22
Leverage the
Ecosystem
Search/Log Alternative Processing
Databases/Data Stores
JDBC
Data Lake/Data Warehouse
File/Messaging/Custom
MQTT
FTP
Container/Deployment Platforms
Kafka and
Confluent
Ecosystem
JMS
SQS
23
New
Opportunities
Engage with Early Adopters and Early Majority Kafka
users
Help customers access new, event driven
applications
Expand your reach into new markets, new use
cases and new opportunities
Extend the use of your product - a more complete
solution offering
Kafka adoption
continues to grow
24
Visibility and access on Confluent Hub
Joint go-to-market activities including
Logo exchange
Blog opportunity
Online talk, field events - and lead sharing
Marketing development fund availability
Joint collateral
Confluent sales enablement activities
Joint tutorials
Marketing
Benefits
Go To Market with
Confluent
25
"Customers want to ingest data streams in real
time from Apache Kafka into Kinetica for
immediate action and analysis. Because the
Confluent Platform adds significant value for
enterprises, we built out the Kinetica connector
using Connect APIs, offering a deeper level of
integration with the Confluent Platform."
-- Irina Farooq, CPO, Kinetica
“Neo4j and Confluent share a customer base that
is determined to push the boundaries of detecting
data connections as interactions and events
occur. Driven by customer need to realize more
value from their streaming data, we have
integrated Neo4j and Kafka both as a sink or
source in a Confluent setup. As a result, Confluent
and Neo4j customers will be able to power their
financial fraud investigations, social media
analyses, network & IT management use cases,
and more with real-time graph analysis.”
-- Philip Rathle, VP of Products, Neo4j
"A Verified Gold Connector with Confluent
Platform is important to our customers who want
a validated and optimized way to enable
operational data flows to and from Couchbase, an
enterprise-class NoSQL database, and Kafka.
With the Kafka Connect API, we have a deeper
level of integration with the Confluent Platform so
together, our joint solution is truly enterprise ready
for any application modernization or cloud native
application initiatives."
-- Anthony Farinha, Senior Director, Business
Development, Couchbase
“In collaboration with Confluent we developed a
Verified Gold connector that enables our
customers to achieve the highest throughput
rates possible. It also enables highly secure,
resilient, and flexible connections between
DataStax database products built on Apache
Cassandra™ and Confluent’s event streaming
platform. We promised our joint enterprise
customers a fully supported microservices-based
application stack and this partnership delivers on
that promise.“
—Kathryn Erickson, Senior Director of Strategic
Partnerships, DataStax
"Imply is a real-time analytics solution, built on
Apache Druid, to store, query, and visualize
event-driven data. Connecting Imply to Kafka
and Confluent Platform enables high-throughput
streaming and sub-second interactive queries at
scale. Together they enable enterprise data
applications to analyze clickstreams, use
behavior, network telemetry and more."
-- Gian Merlino, Chief Technology Officer, Imply
"Attunity, a division of Qlik, partners with
Confluent to provide technologies that solve the
very real problem of streaming data
continuously so you can run your business in
real time.
-- Itamar Ankorion, Managing Director, Data
Integration and SVP, Technology Alliances, Qlik
What our Partners are saying
Driven by customer need to realize
more value from their streaming data
..we developed a Verified Gold
Connector to enable the highest
throughput rates possible.
With the Kakfa Connect API, we
have a deeper level of integration
with Confluent Platform so our
solution is truly enterprise ready...
2626
Verified Integration Program
https://www.confluent.io/verified-integrations-program/
27
Helps you
Build and verify your
Connector or integration
At no cost to Partner
What is the
Verified
Integrations
Program
28
GOLD: Kafka Connect API
Standard: Other integrations, such as:
● Consumer / producer
● Stream processor
● Platforms
● Complimentary
Verification
Levels
29
Best practices documentation
Dev/test assistance
Verification service and support
How We Help Support
30
TSANet
Multi vendor case management
process and tool
Solves multi vendor problems faster
and easier
Alleviates customer concerns about
finger pointing
Program
Benefits
31
Kafka Connect API
Confluent Platform: Commercial standard for
Apache Kafka
Standardized source and sink
Schema registry integration
Control center
Better user experience
Better scalability
Free, easy to use, fault tolerant framework
Why Gold?
Best
Integration for
Confluent
Platform
32
Process
1. Initiated
Discussions to start the
process
2. Guidance
Building the integration
or connector
3. Submitted
Start the testing with
Confluent
4. Verified
Published on
Confluent.io/Hub, start
demand gen activities
33
Requirements
Gold
Verification Guide
● Criteria detail
● Best practices
● Testing
● Quickstart Guide /
Tutorial
Standard
Verification Guide
● Criteria detail
● Integration points
● Quickstart /
Tutorial
Documentation
34
How to Submit
Gold
● Contact Info
● Connector
● Documentation
○ Product Brief
○ End-to-end usage
○ Version info
○ Config info
○ Testing and results
Standard
● Contact Info
● Software Package
● Documentation
○ Product Brief
○ Evaluation Guide
○ Demonstration
Checklist
35
Building the
connector
Have a
connector?
Review the Checklist
Contact us to discuss
Submit your connector
New Connector
Review the Connector
Verification Guide
Contact us to start the
process
Sign Up or Contact Us
confluent.io/verified-integrations-program
37
Sign up - questions or to start the process:
Verified Integrations Program
confluent.io/verified-integrations-program/
Attend the online talk part 2:
How to build a Kafka connector
confluent.io/online-talks/
Kafka Summit San Francisco:
kafka-summit.org/
code KS19Online25 for 25% off
Q&A
38

Weitere ähnliche Inhalte

Was ist angesagt?

Building an Enterprise Eventing Framework (Bryan Zelle, Centene; Neil Buesing...
Building an Enterprise Eventing Framework (Bryan Zelle, Centene; Neil Buesing...Building an Enterprise Eventing Framework (Bryan Zelle, Centene; Neil Buesing...
Building an Enterprise Eventing Framework (Bryan Zelle, Centene; Neil Buesing...
confluent
 
Introducing Events and Stream Processing into Nationwide Building Society
Introducing Events and Stream Processing into Nationwide Building SocietyIntroducing Events and Stream Processing into Nationwide Building Society
Introducing Events and Stream Processing into Nationwide Building Society
confluent
 
Technical Deep Dive: Using Apache Kafka to Optimize Real-Time Analytics in Fi...
Technical Deep Dive: Using Apache Kafka to Optimize Real-Time Analytics in Fi...Technical Deep Dive: Using Apache Kafka to Optimize Real-Time Analytics in Fi...
Technical Deep Dive: Using Apache Kafka to Optimize Real-Time Analytics in Fi...
confluent
 

Was ist angesagt? (20)

Top use cases for 2022 with Data in Motion and Apache Kafka
Top use cases for 2022 with Data in Motion and Apache KafkaTop use cases for 2022 with Data in Motion and Apache Kafka
Top use cases for 2022 with Data in Motion and Apache Kafka
 
Core Banking System on Apache Kafka
Core Banking System on Apache KafkaCore Banking System on Apache Kafka
Core Banking System on Apache Kafka
 
Bridge Your Kafka Streams to Azure Webinar
Bridge Your Kafka Streams to Azure WebinarBridge Your Kafka Streams to Azure Webinar
Bridge Your Kafka Streams to Azure Webinar
 
Apache Kafka vs. Integration Middleware (MQ, ETL, ESB) - Friends, Enemies or ...
Apache Kafka vs. Integration Middleware (MQ, ETL, ESB) - Friends, Enemies or ...Apache Kafka vs. Integration Middleware (MQ, ETL, ESB) - Friends, Enemies or ...
Apache Kafka vs. Integration Middleware (MQ, ETL, ESB) - Friends, Enemies or ...
 
Events Everywhere: Enabling Digital Transformation in the Public Sector
Events Everywhere: Enabling Digital Transformation in the Public SectorEvents Everywhere: Enabling Digital Transformation in the Public Sector
Events Everywhere: Enabling Digital Transformation in the Public Sector
 
Should we manage events like APIs? | Kim Clark, IBM
Should we manage events like APIs? | Kim Clark, IBMShould we manage events like APIs? | Kim Clark, IBM
Should we manage events like APIs? | Kim Clark, IBM
 
Building an Enterprise Eventing Framework (Bryan Zelle, Centene; Neil Buesing...
Building an Enterprise Eventing Framework (Bryan Zelle, Centene; Neil Buesing...Building an Enterprise Eventing Framework (Bryan Zelle, Centene; Neil Buesing...
Building an Enterprise Eventing Framework (Bryan Zelle, Centene; Neil Buesing...
 
Transform Your Mainframe and IBM i Data for the Cloud with Precisely and Apac...
Transform Your Mainframe and IBM i Data for the Cloud with Precisely and Apac...Transform Your Mainframe and IBM i Data for the Cloud with Precisely and Apac...
Transform Your Mainframe and IBM i Data for the Cloud with Precisely and Apac...
 
Apache Kafka® Use Cases for Financial Services
Apache Kafka® Use Cases for Financial ServicesApache Kafka® Use Cases for Financial Services
Apache Kafka® Use Cases for Financial Services
 
IIoT with Kafka and Machine Learning for Supply Chain Optimization In Real Ti...
IIoT with Kafka and Machine Learning for Supply Chain Optimization In Real Ti...IIoT with Kafka and Machine Learning for Supply Chain Optimization In Real Ti...
IIoT with Kafka and Machine Learning for Supply Chain Optimization In Real Ti...
 
Introducing Events and Stream Processing into Nationwide Building Society
Introducing Events and Stream Processing into Nationwide Building SocietyIntroducing Events and Stream Processing into Nationwide Building Society
Introducing Events and Stream Processing into Nationwide Building Society
 
Technical Deep Dive: Using Apache Kafka to Optimize Real-Time Analytics in Fi...
Technical Deep Dive: Using Apache Kafka to Optimize Real-Time Analytics in Fi...Technical Deep Dive: Using Apache Kafka to Optimize Real-Time Analytics in Fi...
Technical Deep Dive: Using Apache Kafka to Optimize Real-Time Analytics in Fi...
 
Event streaming: A paradigm shift in enterprise software architecture
Event streaming: A paradigm shift in enterprise software architectureEvent streaming: A paradigm shift in enterprise software architecture
Event streaming: A paradigm shift in enterprise software architecture
 
GCP for Apache Kafka® Users: Stream Ingestion and Processing
GCP for Apache Kafka® Users: Stream Ingestion and ProcessingGCP for Apache Kafka® Users: Stream Ingestion and Processing
GCP for Apache Kafka® Users: Stream Ingestion and Processing
 
Building an Enterprise Eventing Framework
Building an Enterprise Eventing FrameworkBuilding an Enterprise Eventing Framework
Building an Enterprise Eventing Framework
 
Modernizing your Application Architecture with Microservices
Modernizing your Application Architecture with MicroservicesModernizing your Application Architecture with Microservices
Modernizing your Application Architecture with Microservices
 
Redis and Kafka - Advanced Microservices Design Patterns Simplified
Redis and Kafka - Advanced Microservices Design Patterns SimplifiedRedis and Kafka - Advanced Microservices Design Patterns Simplified
Redis and Kafka - Advanced Microservices Design Patterns Simplified
 
Event-Streaming verstehen in unter 10 Min
Event-Streaming verstehen in unter 10 MinEvent-Streaming verstehen in unter 10 Min
Event-Streaming verstehen in unter 10 Min
 
Confluent Platform 5.4 + Apache Kafka 2.4 Overview (RBAC, Tiered Storage, Mul...
Confluent Platform 5.4 + Apache Kafka 2.4 Overview (RBAC, Tiered Storage, Mul...Confluent Platform 5.4 + Apache Kafka 2.4 Overview (RBAC, Tiered Storage, Mul...
Confluent Platform 5.4 + Apache Kafka 2.4 Overview (RBAC, Tiered Storage, Mul...
 
Bridge to Cloud: Using Apache Kafka to Migrate to GCP
Bridge to Cloud: Using Apache Kafka to Migrate to GCPBridge to Cloud: Using Apache Kafka to Migrate to GCP
Bridge to Cloud: Using Apache Kafka to Migrate to GCP
 

Ähnlich wie Why Build an Apache Kafka® Connector

Ähnlich wie Why Build an Apache Kafka® Connector (20)

Apache Kafka as Event Streaming Platform for Microservice Architectures
Apache Kafka as Event Streaming Platform for Microservice ArchitecturesApache Kafka as Event Streaming Platform for Microservice Architectures
Apache Kafka as Event Streaming Platform for Microservice Architectures
 
Confluent & GSI Webinars series - Session 3
Confluent & GSI Webinars series - Session 3Confluent & GSI Webinars series - Session 3
Confluent & GSI Webinars series - Session 3
 
Fast Data – Fast Cars: Wie Apache Kafka die Datenwelt revolutioniert
Fast Data – Fast Cars: Wie Apache Kafka die Datenwelt revolutioniertFast Data – Fast Cars: Wie Apache Kafka die Datenwelt revolutioniert
Fast Data – Fast Cars: Wie Apache Kafka die Datenwelt revolutioniert
 
Catch the Wave: SAP Event-Driven and Data Streaming for the Intelligence Ente...
Catch the Wave: SAP Event-Driven and Data Streaming for the Intelligence Ente...Catch the Wave: SAP Event-Driven and Data Streaming for the Intelligence Ente...
Catch the Wave: SAP Event-Driven and Data Streaming for the Intelligence Ente...
 
Confluent Partner Tech Talk with Synthesis
Confluent Partner Tech Talk with SynthesisConfluent Partner Tech Talk with Synthesis
Confluent Partner Tech Talk with Synthesis
 
Benefits of Stream Processing and Apache Kafka Use Cases
Benefits of Stream Processing and Apache Kafka Use CasesBenefits of Stream Processing and Apache Kafka Use Cases
Benefits of Stream Processing and Apache Kafka Use Cases
 
Confluent Partner Tech Talk with BearingPoint
Confluent Partner Tech Talk with BearingPointConfluent Partner Tech Talk with BearingPoint
Confluent Partner Tech Talk with BearingPoint
 
Apache Kafka in Financial Services - Use Cases and Architectures
Apache Kafka in Financial Services - Use Cases and ArchitecturesApache Kafka in Financial Services - Use Cases and Architectures
Apache Kafka in Financial Services - Use Cases and Architectures
 
Santander Stream Processing with Apache Flink
Santander Stream Processing with Apache FlinkSantander Stream Processing with Apache Flink
Santander Stream Processing with Apache Flink
 
Message Driven and Event Sourcing
Message Driven and Event SourcingMessage Driven and Event Sourcing
Message Driven and Event Sourcing
 
Kochi mulesoft meetup 02
Kochi mulesoft meetup 02Kochi mulesoft meetup 02
Kochi mulesoft meetup 02
 
Introduction to Apache Kafka and why it matters - Madrid
Introduction to Apache Kafka and why it matters - MadridIntroduction to Apache Kafka and why it matters - Madrid
Introduction to Apache Kafka and why it matters - Madrid
 
Building Serverless EDA w_ AWS Lambda (1).pptx
Building Serverless EDA w_ AWS Lambda (1).pptxBuilding Serverless EDA w_ AWS Lambda (1).pptx
Building Serverless EDA w_ AWS Lambda (1).pptx
 
Partner Connect APAC - 2022 - April
Partner Connect APAC - 2022 - AprilPartner Connect APAC - 2022 - April
Partner Connect APAC - 2022 - April
 
Transforming Financial Services with Event Streaming Data
Transforming Financial Services with Event Streaming DataTransforming Financial Services with Event Streaming Data
Transforming Financial Services with Event Streaming Data
 
Spring Boot+Kafka: the New Enterprise Platform
Spring Boot+Kafka: the New Enterprise PlatformSpring Boot+Kafka: the New Enterprise Platform
Spring Boot+Kafka: the New Enterprise Platform
 
apidays LIVE Singapore 2021 - A cloud-native approach to open banking in acti...
apidays LIVE Singapore 2021 - A cloud-native approach to open banking in acti...apidays LIVE Singapore 2021 - A cloud-native approach to open banking in acti...
apidays LIVE Singapore 2021 - A cloud-native approach to open banking in acti...
 
Driving Success In The Cloud With NGINX
Driving Success In The Cloud With NGINXDriving Success In The Cloud With NGINX
Driving Success In The Cloud With NGINX
 
Using PaaS to run APIs and Microservices in Production
Using PaaS to run APIs and Microservices in ProductionUsing PaaS to run APIs and Microservices in Production
Using PaaS to run APIs and Microservices in Production
 
Greetings david cutler inform and connect
Greetings   david cutler inform and connectGreetings   david cutler inform and connect
Greetings david cutler inform and connect
 

Mehr von confluent

Mehr von confluent (20)

Unlocking the Power of IoT: A comprehensive approach to real-time insights
Unlocking the Power of IoT: A comprehensive approach to real-time insightsUnlocking the Power of IoT: A comprehensive approach to real-time insights
Unlocking the Power of IoT: A comprehensive approach to real-time insights
 
Workshop híbrido: Stream Processing con Flink
Workshop híbrido: Stream Processing con FlinkWorkshop híbrido: Stream Processing con Flink
Workshop híbrido: Stream Processing con Flink
 
Industry 4.0: Building the Unified Namespace with Confluent, HiveMQ and Spark...
Industry 4.0: Building the Unified Namespace with Confluent, HiveMQ and Spark...Industry 4.0: Building the Unified Namespace with Confluent, HiveMQ and Spark...
Industry 4.0: Building the Unified Namespace with Confluent, HiveMQ and Spark...
 
AWS Immersion Day Mapfre - Confluent
AWS Immersion Day Mapfre   -   ConfluentAWS Immersion Day Mapfre   -   Confluent
AWS Immersion Day Mapfre - Confluent
 
Eventos y Microservicios - Santander TechTalk
Eventos y Microservicios - Santander TechTalkEventos y Microservicios - Santander TechTalk
Eventos y Microservicios - Santander TechTalk
 
Q&A with Confluent Experts: Navigating Networking in Confluent Cloud
Q&A with Confluent Experts: Navigating Networking in Confluent CloudQ&A with Confluent Experts: Navigating Networking in Confluent Cloud
Q&A with Confluent Experts: Navigating Networking in Confluent Cloud
 
Citi TechTalk Session 2: Kafka Deep Dive
Citi TechTalk Session 2: Kafka Deep DiveCiti TechTalk Session 2: Kafka Deep Dive
Citi TechTalk Session 2: Kafka Deep Dive
 
Build real-time streaming data pipelines to AWS with Confluent
Build real-time streaming data pipelines to AWS with ConfluentBuild real-time streaming data pipelines to AWS with Confluent
Build real-time streaming data pipelines to AWS with Confluent
 
Q&A with Confluent Professional Services: Confluent Service Mesh
Q&A with Confluent Professional Services: Confluent Service MeshQ&A with Confluent Professional Services: Confluent Service Mesh
Q&A with Confluent Professional Services: Confluent Service Mesh
 
Citi Tech Talk: Event Driven Kafka Microservices
Citi Tech Talk: Event Driven Kafka MicroservicesCiti Tech Talk: Event Driven Kafka Microservices
Citi Tech Talk: Event Driven Kafka Microservices
 
Citi Tech Talk: Messaging Modernization
Citi Tech Talk: Messaging ModernizationCiti Tech Talk: Messaging Modernization
Citi Tech Talk: Messaging Modernization
 
Citi Tech Talk: Data Governance for streaming and real time data
Citi Tech Talk: Data Governance for streaming and real time dataCiti Tech Talk: Data Governance for streaming and real time data
Citi Tech Talk: Data Governance for streaming and real time data
 
Confluent & GSI Webinars series: Session 2
Confluent & GSI Webinars series: Session 2Confluent & GSI Webinars series: Session 2
Confluent & GSI Webinars series: Session 2
 
Data In Motion Paris 2023
Data In Motion Paris 2023Data In Motion Paris 2023
Data In Motion Paris 2023
 
The Future of Application Development - API Days - Melbourne 2023
The Future of Application Development - API Days - Melbourne 2023The Future of Application Development - API Days - Melbourne 2023
The Future of Application Development - API Days - Melbourne 2023
 
The Playful Bond Between REST And Data Streams
The Playful Bond Between REST And Data StreamsThe Playful Bond Between REST And Data Streams
The Playful Bond Between REST And Data Streams
 
The Journey to Data Mesh with Confluent
The Journey to Data Mesh with ConfluentThe Journey to Data Mesh with Confluent
The Journey to Data Mesh with Confluent
 
Citi Tech Talk: Monitoring and Performance
Citi Tech Talk: Monitoring and PerformanceCiti Tech Talk: Monitoring and Performance
Citi Tech Talk: Monitoring and Performance
 
Confluent Partner Tech Talk with Reply
Confluent Partner Tech Talk with ReplyConfluent Partner Tech Talk with Reply
Confluent Partner Tech Talk with Reply
 
Citi Tech Talk Disaster Recovery Solutions Deep Dive
Citi Tech Talk  Disaster Recovery Solutions Deep DiveCiti Tech Talk  Disaster Recovery Solutions Deep Dive
Citi Tech Talk Disaster Recovery Solutions Deep Dive
 

Kürzlich hochgeladen

Architecting Cloud Native Applications
Architecting Cloud Native ApplicationsArchitecting Cloud Native Applications
Architecting Cloud Native Applications
WSO2
 
+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...
+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...
+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...
?#DUbAI#??##{{(☎️+971_581248768%)**%*]'#abortion pills for sale in dubai@
 
Modular Monolith - a Practical Alternative to Microservices @ Devoxx UK 2024
Modular Monolith - a Practical Alternative to Microservices @ Devoxx UK 2024Modular Monolith - a Practical Alternative to Microservices @ Devoxx UK 2024
Modular Monolith - a Practical Alternative to Microservices @ Devoxx UK 2024
Victor Rentea
 

Kürzlich hochgeladen (20)

Strategies for Landing an Oracle DBA Job as a Fresher
Strategies for Landing an Oracle DBA Job as a FresherStrategies for Landing an Oracle DBA Job as a Fresher
Strategies for Landing an Oracle DBA Job as a Fresher
 
ICT role in 21st century education and its challenges
ICT role in 21st century education and its challengesICT role in 21st century education and its challenges
ICT role in 21st century education and its challenges
 
Apidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, Adobe
Apidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, AdobeApidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, Adobe
Apidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, Adobe
 
Manulife - Insurer Transformation Award 2024
Manulife - Insurer Transformation Award 2024Manulife - Insurer Transformation Award 2024
Manulife - Insurer Transformation Award 2024
 
[BuildWithAI] Introduction to Gemini.pdf
[BuildWithAI] Introduction to Gemini.pdf[BuildWithAI] Introduction to Gemini.pdf
[BuildWithAI] Introduction to Gemini.pdf
 
EMPOWERMENT TECHNOLOGY GRADE 11 QUARTER 2 REVIEWER
EMPOWERMENT TECHNOLOGY GRADE 11 QUARTER 2 REVIEWEREMPOWERMENT TECHNOLOGY GRADE 11 QUARTER 2 REVIEWER
EMPOWERMENT TECHNOLOGY GRADE 11 QUARTER 2 REVIEWER
 
2024: Domino Containers - The Next Step. News from the Domino Container commu...
2024: Domino Containers - The Next Step. News from the Domino Container commu...2024: Domino Containers - The Next Step. News from the Domino Container commu...
2024: Domino Containers - The Next Step. News from the Domino Container commu...
 
Exploring Multimodal Embeddings with Milvus
Exploring Multimodal Embeddings with MilvusExploring Multimodal Embeddings with Milvus
Exploring Multimodal Embeddings with Milvus
 
Strategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff
Strategize a Smooth Tenant-to-tenant Migration and Copilot TakeoffStrategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff
Strategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff
 
Apidays New York 2024 - Passkeys: Developing APIs to enable passwordless auth...
Apidays New York 2024 - Passkeys: Developing APIs to enable passwordless auth...Apidays New York 2024 - Passkeys: Developing APIs to enable passwordless auth...
Apidays New York 2024 - Passkeys: Developing APIs to enable passwordless auth...
 
MS Copilot expands with MS Graph connectors
MS Copilot expands with MS Graph connectorsMS Copilot expands with MS Graph connectors
MS Copilot expands with MS Graph connectors
 
Architecting Cloud Native Applications
Architecting Cloud Native ApplicationsArchitecting Cloud Native Applications
Architecting Cloud Native Applications
 
Apidays New York 2024 - Accelerating FinTech Innovation by Vasa Krishnan, Fin...
Apidays New York 2024 - Accelerating FinTech Innovation by Vasa Krishnan, Fin...Apidays New York 2024 - Accelerating FinTech Innovation by Vasa Krishnan, Fin...
Apidays New York 2024 - Accelerating FinTech Innovation by Vasa Krishnan, Fin...
 
+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...
+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...
+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...
 
Navigating the Deluge_ Dubai Floods and the Resilience of Dubai International...
Navigating the Deluge_ Dubai Floods and the Resilience of Dubai International...Navigating the Deluge_ Dubai Floods and the Resilience of Dubai International...
Navigating the Deluge_ Dubai Floods and the Resilience of Dubai International...
 
ProductAnonymous-April2024-WinProductDiscovery-MelissaKlemke
ProductAnonymous-April2024-WinProductDiscovery-MelissaKlemkeProductAnonymous-April2024-WinProductDiscovery-MelissaKlemke
ProductAnonymous-April2024-WinProductDiscovery-MelissaKlemke
 
Rising Above_ Dubai Floods and the Fortitude of Dubai International Airport.pdf
Rising Above_ Dubai Floods and the Fortitude of Dubai International Airport.pdfRising Above_ Dubai Floods and the Fortitude of Dubai International Airport.pdf
Rising Above_ Dubai Floods and the Fortitude of Dubai International Airport.pdf
 
Modular Monolith - a Practical Alternative to Microservices @ Devoxx UK 2024
Modular Monolith - a Practical Alternative to Microservices @ Devoxx UK 2024Modular Monolith - a Practical Alternative to Microservices @ Devoxx UK 2024
Modular Monolith - a Practical Alternative to Microservices @ Devoxx UK 2024
 
Axa Assurance Maroc - Insurer Innovation Award 2024
Axa Assurance Maroc - Insurer Innovation Award 2024Axa Assurance Maroc - Insurer Innovation Award 2024
Axa Assurance Maroc - Insurer Innovation Award 2024
 
"I see eyes in my soup": How Delivery Hero implemented the safety system for ...
"I see eyes in my soup": How Delivery Hero implemented the safety system for ..."I see eyes in my soup": How Delivery Hero implemented the safety system for ...
"I see eyes in my soup": How Delivery Hero implemented the safety system for ...
 

Why Build an Apache Kafka® Connector

  • 1. 1 Why Build an Apache Kafka® Connector Series - Building Kafka Connectors - the Why and How Part 1 -Why Build a Kafka Connector Part 2 -How to Build a Kafka Connector
  • 2. 2 Speakers Sree Karuthody, Sr. Manager, Technology Partnerships Jeff Bean, Partner Solution Architect Sid Rabindran, Dir. Partner Programs and Tech Ecosystem Partners Lisa Sensmeier, Partner Marketing
  • 3. 3 What is Event Streaming Kafka and Connecting to Kafka Value of building a connector Verified Integrations Program Q and A Agenda Key business drivers behind connecting to Kafka The Verified Integrations Program, benefits, how to participate, process
  • 4. 44 What is Event Streaming
  • 5. 55 The New Enterprise Reality Innovate or be disrupted ● Deliver new, real-time customer experiences ● Create new business models ● Deliver massive internal efficiencies & risk reduction Every company is a software company ● Capital One 10,000 of 40,000 employees are software engineers ● Goldman Sachs 1.5B lines of code across 7,000+ applications Innovation is about events, not “data” ● Event = something happened ● Your business = a continually updating stream of events ● Your success = your ability to respond to these events
  • 6. 66 The Rise of Event Streaming Data as a continuous stream of events 60% Fortune 100 Companies Using Apache Kafka
  • 7. 77 Auto / Transport Without Event Streaming With Event Streaming Event Streaming Enables New Outcomes Call for driver availability No knowledge of driver arrival No data on feature usage Real-time driver-rider match Real-time ETA Real-time sensor diagnostics Banking Nightly updated account balance Batch fraud checks Batch regulatory reporting Real-time account updates Real-time credit card fraud alerts Real-time regulatory reporting Retail Post-order “out of stock” emails No upsell through personalization Batch point-of sale reports Real-time inventory Real-time recommendations Real-time sales reporting
  • 8. 88 Confluent is Enabling Event-driven Transformation across Industries Healthcare & Pharma ● Patient monitoring ● Prescription control ● Lab alerts ● Medication tracking Banking & Capital Markets ● Customer 360 ● Fraud detection ● Mainframe offload ● Trade data capture Retail ● Inventory management ● Product catalog ● A/B testing ● Customized experiences Telecommunications ● Personalized ads ● Customer 360 ● Network integrity Automotive & Transportation ● Connected car ● Fleet management ● Manufacturing data processing Travel & Leisure ● Visitor segmentation ● Booking systems ● Pricing services ● Fraud detection
  • 10. 1010 Confluent Enables Your Event Streaming Success Confluent founders are original creators of Kafka Confluent team wrote 80% of Kafka commits Confluent Platform extends Apache Kafka to be a secure, enterprise-ready platform Confluent helps enterprises successfully deploy event streaming at scale and accelerate time to market Hall of Innovation CTO Innovation Award Winner 2019
  • 11. 11 Confluent Platform Operations and Security Development & Stream Processing Support,services,training&partners Apache Kafka Security plugins | Role-Based Access Control Control Center | Replicator | Auto Data Balancer | Operator Connectors Clients | REST Proxy MQTT Proxy | Schema Registry KSQL Connect Continuous Commit Log Streams Complete Event Streaming Platform Mission-critical Reliability Freedom of Choice Datacenter Public Cloud Confluent Cloud Self-Managed Software Fully-Managed Service
  • 12. 12 Real-Time Inventory Real-Time Fraud Detection Real-Time Customer 360 Machine Learning Models Real-Time Data Transformation ... Contextual Event-Driven Applications Universal Event Pipeline Data Stores Logs 3rd Party Apps Custom Apps/Microservices TREAMSSTREAMS CONNECT CLIENTS With Confluent
  • 14. 15 Without Kafka and Confluent
  • 15. 16 ● End to end monitoring and analytics ● Enterprise-grade security ● Multi-datacenter replication ● SQL-based stream processing ● Cloud-native deployment with K8s operator ● Run as self-managed software or as fully managed service with Confluent Cloud Enterprise streaming platform built by the original creators of Apache Kafka Edge Cloud Data LakeDatabases Datacenter IoT SaaS AppsMobile Microservices Machine Learning Confluent Platform
  • 16. 17 Deploy and Stream with Confluent on Any Cloud Confluent Platform The Enterprise Distribution of Apache Kafka Deploy on any platform on-premises or in public clouds Fully-Managed Service Confluent Cloud Apache Kafka Re-engineered for the Cloud Available on the leading public clouds VM Self-Managed Software
  • 17. 1818 Apache Kafka™ Connect API – Streaming Data Capture JDBC Mongo MySQL Elastic Cassandra HDFS Kafka Connect API Kafka Pipeline Connector Connector Connector Connector Connector Connector Sources Sinks Fault tolerant Manage hundreds of data sources and sinks Preserves data schema Part of Apache Kafka project Integrated within Confluent Platform’s Control Center
  • 18. 19 Confluent Hub Connectors are posted - Verified Partner Integrations - Confluent provided and supported confluent.io/hub
  • 19. 20
  • 20. 2121 Value of building a connector
  • 21. 22 Leverage the Ecosystem Search/Log Alternative Processing Databases/Data Stores JDBC Data Lake/Data Warehouse File/Messaging/Custom MQTT FTP Container/Deployment Platforms Kafka and Confluent Ecosystem JMS SQS
  • 22. 23 New Opportunities Engage with Early Adopters and Early Majority Kafka users Help customers access new, event driven applications Expand your reach into new markets, new use cases and new opportunities Extend the use of your product - a more complete solution offering Kafka adoption continues to grow
  • 23. 24 Visibility and access on Confluent Hub Joint go-to-market activities including Logo exchange Blog opportunity Online talk, field events - and lead sharing Marketing development fund availability Joint collateral Confluent sales enablement activities Joint tutorials Marketing Benefits Go To Market with Confluent
  • 24. 25 "Customers want to ingest data streams in real time from Apache Kafka into Kinetica for immediate action and analysis. Because the Confluent Platform adds significant value for enterprises, we built out the Kinetica connector using Connect APIs, offering a deeper level of integration with the Confluent Platform." -- Irina Farooq, CPO, Kinetica “Neo4j and Confluent share a customer base that is determined to push the boundaries of detecting data connections as interactions and events occur. Driven by customer need to realize more value from their streaming data, we have integrated Neo4j and Kafka both as a sink or source in a Confluent setup. As a result, Confluent and Neo4j customers will be able to power their financial fraud investigations, social media analyses, network & IT management use cases, and more with real-time graph analysis.” -- Philip Rathle, VP of Products, Neo4j "A Verified Gold Connector with Confluent Platform is important to our customers who want a validated and optimized way to enable operational data flows to and from Couchbase, an enterprise-class NoSQL database, and Kafka. With the Kafka Connect API, we have a deeper level of integration with the Confluent Platform so together, our joint solution is truly enterprise ready for any application modernization or cloud native application initiatives." -- Anthony Farinha, Senior Director, Business Development, Couchbase “In collaboration with Confluent we developed a Verified Gold connector that enables our customers to achieve the highest throughput rates possible. It also enables highly secure, resilient, and flexible connections between DataStax database products built on Apache Cassandra™ and Confluent’s event streaming platform. We promised our joint enterprise customers a fully supported microservices-based application stack and this partnership delivers on that promise.“ —Kathryn Erickson, Senior Director of Strategic Partnerships, DataStax "Imply is a real-time analytics solution, built on Apache Druid, to store, query, and visualize event-driven data. Connecting Imply to Kafka and Confluent Platform enables high-throughput streaming and sub-second interactive queries at scale. Together they enable enterprise data applications to analyze clickstreams, use behavior, network telemetry and more." -- Gian Merlino, Chief Technology Officer, Imply "Attunity, a division of Qlik, partners with Confluent to provide technologies that solve the very real problem of streaming data continuously so you can run your business in real time. -- Itamar Ankorion, Managing Director, Data Integration and SVP, Technology Alliances, Qlik What our Partners are saying Driven by customer need to realize more value from their streaming data ..we developed a Verified Gold Connector to enable the highest throughput rates possible. With the Kakfa Connect API, we have a deeper level of integration with Confluent Platform so our solution is truly enterprise ready...
  • 26. 27 Helps you Build and verify your Connector or integration At no cost to Partner What is the Verified Integrations Program
  • 27. 28 GOLD: Kafka Connect API Standard: Other integrations, such as: ● Consumer / producer ● Stream processor ● Platforms ● Complimentary Verification Levels
  • 28. 29 Best practices documentation Dev/test assistance Verification service and support How We Help Support
  • 29. 30 TSANet Multi vendor case management process and tool Solves multi vendor problems faster and easier Alleviates customer concerns about finger pointing Program Benefits
  • 30. 31 Kafka Connect API Confluent Platform: Commercial standard for Apache Kafka Standardized source and sink Schema registry integration Control center Better user experience Better scalability Free, easy to use, fault tolerant framework Why Gold? Best Integration for Confluent Platform
  • 31. 32 Process 1. Initiated Discussions to start the process 2. Guidance Building the integration or connector 3. Submitted Start the testing with Confluent 4. Verified Published on Confluent.io/Hub, start demand gen activities
  • 32. 33 Requirements Gold Verification Guide ● Criteria detail ● Best practices ● Testing ● Quickstart Guide / Tutorial Standard Verification Guide ● Criteria detail ● Integration points ● Quickstart / Tutorial Documentation
  • 33. 34 How to Submit Gold ● Contact Info ● Connector ● Documentation ○ Product Brief ○ End-to-end usage ○ Version info ○ Config info ○ Testing and results Standard ● Contact Info ● Software Package ● Documentation ○ Product Brief ○ Evaluation Guide ○ Demonstration Checklist
  • 34. 35 Building the connector Have a connector? Review the Checklist Contact us to discuss Submit your connector New Connector Review the Connector Verification Guide Contact us to start the process Sign Up or Contact Us confluent.io/verified-integrations-program
  • 35. 37 Sign up - questions or to start the process: Verified Integrations Program confluent.io/verified-integrations-program/ Attend the online talk part 2: How to build a Kafka connector confluent.io/online-talks/ Kafka Summit San Francisco: kafka-summit.org/ code KS19Online25 for 25% off Q&A
  • 36. 38