Kafka Summit NYC 2017 - Venice: A Distributed Database on top of Kafka

•

0 gefällt mir•681 views

Presentation by Matthew Wise, LinkedIn

Matthew Wise
Venice
A distributed database on top of Kafka

History

4
Derived Data

5
Voldemort RO

6
BuildAndPush
Versioned
Stores

7
Virginia
Many
Datacenters

8
600+ Stores
100+ TB Pushed Daily
40+ Billion Queries/Day

9
Random Access
Low Latency
Hadoop

10
Random Access
Low Latency
Kafka

11
Streaming
Ingest

12
Hybrid

Venice

V
V
V
Kafka

Distributed K/V database
Kafka partitioning
Versioned pushes
Topic maps to version

V
V
V
Hadoop

MirrorMaker
Datacenter
Datacenter
Datacenter
Many
Datacenters

18
Venice Requirements
Message Buffer
Cross Datacenter Mirroring
Dynamic Topic Creation
Dynamic Topic Deletion
Mirrored Topic Creation
Mirrored Topic Deletion
Scalable Consumption
Data Guarantees

Kafka

20
V
Message Buffer

21
M V
Mirroring

22
V
Partitioned

23
V
Topics

24
1 2 3 3 4 5
Data Guarantees:
At least once delivery

Integration Work

26
M
Identity Mirroring

27
1 2 3 4 5
Data Guarantees:
Exactly once delivery

28
M
Dynamic Topic Deletion

Big Architectures

30
Kafka
Does
Data

31
Data
Control
Kafka
Does
Control?

Data
Control
Distributed Systems:
Need Control

Summary

Venice

Matthew Wise – linkedin.com/in/matthewrswise
Venice
A distributed Database on top of Kafka

Empfohlen

Real-Time Dynamic Data Export Using the Kafka Ecosystem

Real-Time Dynamic Data Export Using the Kafka Ecosystem

Real-Time Dynamic Data Export Using the Kafka Ecosystemconfluent

Kafka Summit SF 2017 - Database Streaming at WePay

Kafka Summit SF 2017 - Database Streaming at WePay

Kafka Summit SF 2017 - Database Streaming at WePayconfluent

Kafka Summit SF 2017 - Query the Application, Not a Database: “Interactive Qu...

Kafka Summit SF 2017 - Query the Application, Not a Database: “Interactive Qu...

Kafka Summit SF 2017 - Query the Application, Not a Database: “Interactive Qu...confluent

Real-world Streaming Architectures

Real-world Streaming Architectures

Real-world Streaming Architecturesconfluent

Kafka Summit NYC 2017 Hanging Out with Your Past Self in VR

Kafka Summit NYC 2017 Hanging Out with Your Past Self in VR

Kafka Summit NYC 2017 Hanging Out with Your Past Self in VRconfluent

Putting the Micro into Microservices with Stateful Stream Processing

Putting the Micro into Microservices with Stateful Stream Processing

Putting the Micro into Microservices with Stateful Stream Processingconfluent

You Must Construct Additional Pipelines: Pub-Sub on Kafka at Blizzard

You Must Construct Additional Pipelines: Pub-Sub on Kafka at Blizzard

You Must Construct Additional Pipelines: Pub-Sub on Kafka at Blizzard confluent

Kafka Summit NYC 2017 - Data Processing at LinkedIn with Apache Kafka

Kafka Summit NYC 2017 - Data Processing at LinkedIn with Apache Kafka

Kafka Summit NYC 2017 - Data Processing at LinkedIn with Apache Kafkaconfluent

Empfohlen

Real-Time Dynamic Data Export Using the Kafka Ecosystem

Real-Time Dynamic Data Export Using the Kafka Ecosystem

Real-Time Dynamic Data Export Using the Kafka Ecosystemconfluent

Kafka Summit SF 2017 - Database Streaming at WePay

Kafka Summit SF 2017 - Database Streaming at WePay

Kafka Summit SF 2017 - Database Streaming at WePayconfluent

Kafka Summit SF 2017 - Query the Application, Not a Database: “Interactive Qu...

Kafka Summit SF 2017 - Query the Application, Not a Database: “Interactive Qu...

Kafka Summit SF 2017 - Query the Application, Not a Database: “Interactive Qu...confluent

Real-world Streaming Architectures

Real-world Streaming Architectures

Real-world Streaming Architecturesconfluent

Kafka Summit NYC 2017 Hanging Out with Your Past Self in VR

Kafka Summit NYC 2017 Hanging Out with Your Past Self in VR

Kafka Summit NYC 2017 Hanging Out with Your Past Self in VRconfluent

Putting the Micro into Microservices with Stateful Stream Processing

Putting the Micro into Microservices with Stateful Stream Processing

Putting the Micro into Microservices with Stateful Stream Processingconfluent

You Must Construct Additional Pipelines: Pub-Sub on Kafka at Blizzard

You Must Construct Additional Pipelines: Pub-Sub on Kafka at Blizzard

You Must Construct Additional Pipelines: Pub-Sub on Kafka at Blizzard confluent

Kafka Summit NYC 2017 - Data Processing at LinkedIn with Apache Kafka

Kafka Summit NYC 2017 - Data Processing at LinkedIn with Apache Kafka

Kafka Summit NYC 2017 - Data Processing at LinkedIn with Apache Kafkaconfluent

Integrating Apache Kafka and Elastic Using the Connect Framework

Integrating Apache Kafka and Elastic Using the Connect Framework

Integrating Apache Kafka and Elastic Using the Connect Frameworkconfluent

Kafka Summit SF 2017 - Riot's Journey to Global Kafka Aggregation

Kafka Summit SF 2017 - Riot's Journey to Global Kafka Aggregation

Kafka Summit SF 2017 - Riot's Journey to Global Kafka Aggregationconfluent

Kafka Summit SF 2017 - Keynote - Go Against the Flow: Databases and Stream Pr...

Kafka Summit SF 2017 - Keynote - Go Against the Flow: Databases and Stream Pr...

Kafka Summit SF 2017 - Keynote - Go Against the Flow: Databases and Stream Pr...confluent

Capital One Delivers Risk Insights in Real Time with Stream Processing

Capital One Delivers Risk Insights in Real Time with Stream Processing

Capital One Delivers Risk Insights in Real Time with Stream Processingconfluent

Low-latency data applications with Kafka and Agg indexes | Tino Tereshko, Fir...

Low-latency data applications with Kafka and Agg indexes | Tino Tereshko, Fir...

Low-latency data applications with Kafka and Agg indexes | Tino Tereshko, Fir...HostedbyConfluent

Analyzing Petabyte Scale Financial Data with Apache Pinot and Apache Kafka | ...

Analyzing Petabyte Scale Financial Data with Apache Pinot and Apache Kafka | ...

Analyzing Petabyte Scale Financial Data with Apache Pinot and Apache Kafka | ...HostedbyConfluent

It's Time To Stop Using Lambda Architecture | Yaroslav Tkachenko, Shopify

It's Time To Stop Using Lambda Architecture | Yaroslav Tkachenko, Shopify

It's Time To Stop Using Lambda Architecture | Yaroslav Tkachenko, ShopifyHostedbyConfluent

Kafka Summit SF 2017 - Fast Data in Supply Chain Planning

Kafka Summit SF 2017 - Fast Data in Supply Chain Planning

Kafka Summit SF 2017 - Fast Data in Supply Chain Planningconfluent

Streaming ETL to Elastic with Apache Kafka and KSQL

Streaming ETL to Elastic with Apache Kafka and KSQL

Streaming ETL to Elastic with Apache Kafka and KSQLconfluent

Apache kafka-a distributed streaming platform

Apache kafka-a distributed streaming platform

Apache kafka-a distributed streaming platformconfluent

Sub-Second SQL Search, Aggregations and Joins with Kafka and Rockset | Dhruba...

Sub-Second SQL Search, Aggregations and Joins with Kafka and Rockset | Dhruba...

Sub-Second SQL Search, Aggregations and Joins with Kafka and Rockset | Dhruba...HostedbyConfluent

Introduction to apache kafka, confluent and why they matter

Introduction to apache kafka, confluent and why they matter

Introduction to apache kafka, confluent and why they matterPaolo Castagna

Introduction to Apache Kafka and Confluent... and why they matter

Introduction to Apache Kafka and Confluent... and why they matter

Introduction to Apache Kafka and Confluent... and why they matterconfluent

Kafka Lag Monitoring For Human Beings (Elad Leev, AppsFlyer) Kafka Summit 2020

Kafka Lag Monitoring For Human Beings (Elad Leev, AppsFlyer) Kafka Summit 2020

Kafka Lag Monitoring For Human Beings (Elad Leev, AppsFlyer) Kafka Summit 2020HostedbyConfluent

5 lessons learned for successful migration to Confluent cloud | Natan Silinit...

5 lessons learned for successful migration to Confluent cloud | Natan Silinit...

5 lessons learned for successful migration to Confluent cloud | Natan Silinit...HostedbyConfluent

How to use Standard SQL over Kafka: From the basics to advanced use cases | F...

How to use Standard SQL over Kafka: From the basics to advanced use cases | F...

How to use Standard SQL over Kafka: From the basics to advanced use cases | F...HostedbyConfluent

Bravo Six, Going Realtime. Transitioning Activision Data Pipeline to Streamin...

Bravo Six, Going Realtime. Transitioning Activision Data Pipeline to Streamin...

Bravo Six, Going Realtime. Transitioning Activision Data Pipeline to Streamin...HostedbyConfluent

Billions of Messages in Real Time: Why Paypal & LinkedIn Trust an Engagement ...

Billions of Messages in Real Time: Why Paypal & LinkedIn Trust an Engagement ...

Billions of Messages in Real Time: Why Paypal & LinkedIn Trust an Engagement ...confluent

Kafka: Journey from Just Another Software to Being a Critical Part of PayPal ...

Kafka: Journey from Just Another Software to Being a Critical Part of PayPal ...

Kafka: Journey from Just Another Software to Being a Critical Part of PayPal ...confluent

Real-time Data Streaming from Oracle to Apache Kafka

Real-time Data Streaming from Oracle to Apache Kafka

Real-time Data Streaming from Oracle to Apache Kafka confluent

Introducing Venice - Strata NYC 2017

Introducing Venice - Strata NYC 2017

Introducing Venice - Strata NYC 2017Felix GV

Introducing Venice

Introducing Venice

Introducing VeniceYan Yan

Weitere ähnliche Inhalte

Was ist angesagt?

Integrating Apache Kafka and Elastic Using the Connect Framework

Integrating Apache Kafka and Elastic Using the Connect Framework

Integrating Apache Kafka and Elastic Using the Connect Frameworkconfluent

Kafka Summit SF 2017 - Riot's Journey to Global Kafka Aggregation

Kafka Summit SF 2017 - Riot's Journey to Global Kafka Aggregation

Kafka Summit SF 2017 - Riot's Journey to Global Kafka Aggregationconfluent

Kafka Summit SF 2017 - Keynote - Go Against the Flow: Databases and Stream Pr...

Kafka Summit SF 2017 - Keynote - Go Against the Flow: Databases and Stream Pr...

Kafka Summit SF 2017 - Keynote - Go Against the Flow: Databases and Stream Pr...confluent

Capital One Delivers Risk Insights in Real Time with Stream Processing

Capital One Delivers Risk Insights in Real Time with Stream Processing

Capital One Delivers Risk Insights in Real Time with Stream Processingconfluent

Low-latency data applications with Kafka and Agg indexes | Tino Tereshko, Fir...

Low-latency data applications with Kafka and Agg indexes | Tino Tereshko, Fir...

Low-latency data applications with Kafka and Agg indexes | Tino Tereshko, Fir...HostedbyConfluent

Analyzing Petabyte Scale Financial Data with Apache Pinot and Apache Kafka | ...

Analyzing Petabyte Scale Financial Data with Apache Pinot and Apache Kafka | ...

Analyzing Petabyte Scale Financial Data with Apache Pinot and Apache Kafka | ...HostedbyConfluent

It's Time To Stop Using Lambda Architecture | Yaroslav Tkachenko, Shopify

It's Time To Stop Using Lambda Architecture | Yaroslav Tkachenko, Shopify

It's Time To Stop Using Lambda Architecture | Yaroslav Tkachenko, ShopifyHostedbyConfluent

Kafka Summit SF 2017 - Fast Data in Supply Chain Planning

Kafka Summit SF 2017 - Fast Data in Supply Chain Planning

Kafka Summit SF 2017 - Fast Data in Supply Chain Planningconfluent

Streaming ETL to Elastic with Apache Kafka and KSQL

Streaming ETL to Elastic with Apache Kafka and KSQL

Streaming ETL to Elastic with Apache Kafka and KSQLconfluent

Apache kafka-a distributed streaming platform

Apache kafka-a distributed streaming platform

Apache kafka-a distributed streaming platformconfluent

Sub-Second SQL Search, Aggregations and Joins with Kafka and Rockset | Dhruba...

Sub-Second SQL Search, Aggregations and Joins with Kafka and Rockset | Dhruba...

Sub-Second SQL Search, Aggregations and Joins with Kafka and Rockset | Dhruba...HostedbyConfluent

Introduction to apache kafka, confluent and why they matter

Introduction to apache kafka, confluent and why they matter

Introduction to apache kafka, confluent and why they matterPaolo Castagna

Introduction to Apache Kafka and Confluent... and why they matter

Introduction to Apache Kafka and Confluent... and why they matter

Introduction to Apache Kafka and Confluent... and why they matterconfluent

Kafka Lag Monitoring For Human Beings (Elad Leev, AppsFlyer) Kafka Summit 2020

Kafka Lag Monitoring For Human Beings (Elad Leev, AppsFlyer) Kafka Summit 2020

Kafka Lag Monitoring For Human Beings (Elad Leev, AppsFlyer) Kafka Summit 2020HostedbyConfluent

5 lessons learned for successful migration to Confluent cloud | Natan Silinit...

5 lessons learned for successful migration to Confluent cloud | Natan Silinit...

5 lessons learned for successful migration to Confluent cloud | Natan Silinit...HostedbyConfluent

How to use Standard SQL over Kafka: From the basics to advanced use cases | F...

How to use Standard SQL over Kafka: From the basics to advanced use cases | F...

How to use Standard SQL over Kafka: From the basics to advanced use cases | F...HostedbyConfluent

Bravo Six, Going Realtime. Transitioning Activision Data Pipeline to Streamin...

Bravo Six, Going Realtime. Transitioning Activision Data Pipeline to Streamin...

Bravo Six, Going Realtime. Transitioning Activision Data Pipeline to Streamin...HostedbyConfluent

Billions of Messages in Real Time: Why Paypal & LinkedIn Trust an Engagement ...

Billions of Messages in Real Time: Why Paypal & LinkedIn Trust an Engagement ...

Billions of Messages in Real Time: Why Paypal & LinkedIn Trust an Engagement ...confluent

Kafka: Journey from Just Another Software to Being a Critical Part of PayPal ...

Kafka: Journey from Just Another Software to Being a Critical Part of PayPal ...

Kafka: Journey from Just Another Software to Being a Critical Part of PayPal ...confluent

Real-time Data Streaming from Oracle to Apache Kafka

Real-time Data Streaming from Oracle to Apache Kafka

Real-time Data Streaming from Oracle to Apache Kafka confluent

Was ist angesagt? (20)

Integrating Apache Kafka and Elastic Using the Connect Framework

Integrating Apache Kafka and Elastic Using the Connect Framework

Integrating Apache Kafka and Elastic Using the Connect Framework

Kafka Summit SF 2017 - Riot's Journey to Global Kafka Aggregation

Kafka Summit SF 2017 - Riot's Journey to Global Kafka Aggregation

Kafka Summit SF 2017 - Riot's Journey to Global Kafka Aggregation

Kafka Summit SF 2017 - Keynote - Go Against the Flow: Databases and Stream Pr...

Kafka Summit SF 2017 - Keynote - Go Against the Flow: Databases and Stream Pr...

Kafka Summit SF 2017 - Keynote - Go Against the Flow: Databases and Stream Pr...

Capital One Delivers Risk Insights in Real Time with Stream Processing

Capital One Delivers Risk Insights in Real Time with Stream Processing

Capital One Delivers Risk Insights in Real Time with Stream Processing

Low-latency data applications with Kafka and Agg indexes | Tino Tereshko, Fir...

Low-latency data applications with Kafka and Agg indexes | Tino Tereshko, Fir...

Low-latency data applications with Kafka and Agg indexes | Tino Tereshko, Fir...

Analyzing Petabyte Scale Financial Data with Apache Pinot and Apache Kafka | ...

Analyzing Petabyte Scale Financial Data with Apache Pinot and Apache Kafka | ...

Analyzing Petabyte Scale Financial Data with Apache Pinot and Apache Kafka | ...

It's Time To Stop Using Lambda Architecture | Yaroslav Tkachenko, Shopify

It's Time To Stop Using Lambda Architecture | Yaroslav Tkachenko, Shopify

It's Time To Stop Using Lambda Architecture | Yaroslav Tkachenko, Shopify

Kafka Summit SF 2017 - Fast Data in Supply Chain Planning

Kafka Summit SF 2017 - Fast Data in Supply Chain Planning

Kafka Summit SF 2017 - Fast Data in Supply Chain Planning

Streaming ETL to Elastic with Apache Kafka and KSQL

Streaming ETL to Elastic with Apache Kafka and KSQL

Streaming ETL to Elastic with Apache Kafka and KSQL

Apache kafka-a distributed streaming platform

Apache kafka-a distributed streaming platform

Apache kafka-a distributed streaming platform

Sub-Second SQL Search, Aggregations and Joins with Kafka and Rockset | Dhruba...

Sub-Second SQL Search, Aggregations and Joins with Kafka and Rockset | Dhruba...

Sub-Second SQL Search, Aggregations and Joins with Kafka and Rockset | Dhruba...

Introduction to apache kafka, confluent and why they matter

Introduction to apache kafka, confluent and why they matter

Introduction to apache kafka, confluent and why they matter

Introduction to Apache Kafka and Confluent... and why they matter

Introduction to Apache Kafka and Confluent... and why they matter

Introduction to Apache Kafka and Confluent... and why they matter

Kafka Lag Monitoring For Human Beings (Elad Leev, AppsFlyer) Kafka Summit 2020

Kafka Lag Monitoring For Human Beings (Elad Leev, AppsFlyer) Kafka Summit 2020

Kafka Lag Monitoring For Human Beings (Elad Leev, AppsFlyer) Kafka Summit 2020

5 lessons learned for successful migration to Confluent cloud | Natan Silinit...

5 lessons learned for successful migration to Confluent cloud | Natan Silinit...

5 lessons learned for successful migration to Confluent cloud | Natan Silinit...

How to use Standard SQL over Kafka: From the basics to advanced use cases | F...

How to use Standard SQL over Kafka: From the basics to advanced use cases | F...

How to use Standard SQL over Kafka: From the basics to advanced use cases | F...

Bravo Six, Going Realtime. Transitioning Activision Data Pipeline to Streamin...

Bravo Six, Going Realtime. Transitioning Activision Data Pipeline to Streamin...

Bravo Six, Going Realtime. Transitioning Activision Data Pipeline to Streamin...

Billions of Messages in Real Time: Why Paypal & LinkedIn Trust an Engagement ...

Billions of Messages in Real Time: Why Paypal & LinkedIn Trust an Engagement ...

Billions of Messages in Real Time: Why Paypal & LinkedIn Trust an Engagement ...

Kafka: Journey from Just Another Software to Being a Critical Part of PayPal ...

Kafka: Journey from Just Another Software to Being a Critical Part of PayPal ...

Kafka: Journey from Just Another Software to Being a Critical Part of PayPal ...

Real-time Data Streaming from Oracle to Apache Kafka

Real-time Data Streaming from Oracle to Apache Kafka

Real-time Data Streaming from Oracle to Apache Kafka

Ähnlich wie Kafka Summit NYC 2017 - Venice: A Distributed Database on top of Kafka

Introducing Venice - Strata NYC 2017

Introducing Venice - Strata NYC 2017

Introducing Venice - Strata NYC 2017Felix GV

Introducing Venice

Introducing Venice

Introducing VeniceYan Yan

Dev Days Europe - Kafka based Global Data Mesh at Wix

Dev Days Europe - Kafka based Global Data Mesh at Wix

Dev Days Europe - Kafka based Global Data Mesh at WixNatan Silnitsky

Kafka based Global Data Mesh at Wix with Natan Silnitsky | Kafka Summit Londo...

Kafka based Global Data Mesh at Wix with Natan Silnitsky | Kafka Summit Londo...

Kafka based Global Data Mesh at Wix with Natan Silnitsky | Kafka Summit Londo...HostedbyConfluent

Kafka Summit London - Kafka based Global Data Mesh at Wix

Kafka Summit London - Kafka based Global Data Mesh at Wix

Kafka Summit London - Kafka based Global Data Mesh at WixNatan Silnitsky

Devoxx Ukraine - Kafka based Global Data Mesh

Devoxx Ukraine - Kafka based Global Data Mesh

Devoxx Ukraine - Kafka based Global Data MeshNatan Silnitsky

Kafka based Global Data Mesh at Wix

Kafka based Global Data Mesh at Wix

Kafka based Global Data Mesh at WixNatan Silnitsky

Can Apache Kafka Replace a Database? – The 2021 Update | Kai Waehner, Confluent

Can Apache Kafka Replace a Database? – The 2021 Update | Kai Waehner, Confluent

Can Apache Kafka Replace a Database? – The 2021 Update | Kai Waehner, ConfluentHostedbyConfluent

Migrating to Multi Cluster Managed Kafka - DevopStars 2022

Migrating to Multi Cluster Managed Kafka - DevopStars 2022

Migrating to Multi Cluster Managed Kafka - DevopStars 2022Natan Silnitsky

Data Streaming with Apache Kafka & MongoDB - EMEA

Data Streaming with Apache Kafka & MongoDB - EMEA

Data Streaming with Apache Kafka & MongoDB - EMEAAndrew Morgan

Webinar: Data Streaming with Apache Kafka & MongoDB

Webinar: Data Streaming with Apache Kafka & MongoDB

Webinar: Data Streaming with Apache Kafka & MongoDBMongoDB

vCloud Technical deck - cb.ppt

vCloud Technical deck - cb.ppt

vCloud Technical deck - cb.pptjuergenJaeckel

Data analytics at scale implementing stateful stream processing - publish

Data analytics at scale implementing stateful stream processing - publish

Data analytics at scale implementing stateful stream processing - publishCodeValue

Migrating to Multi Cluster Managed Kafka - ApacheKafkaIL

Migrating to Multi Cluster Managed Kafka - ApacheKafkaIL

Migrating to Multi Cluster Managed Kafka - ApacheKafkaILNatan Silnitsky

Migrating to Multi Cluster Managed Kafka - Conf42 - CloudNative

Migrating to Multi Cluster Managed Kafka - Conf42 - CloudNative

Migrating to Multi Cluster Managed Kafka - Conf42 - CloudNative Natan Silnitsky

Devoxx UK - Migrating to Multi Cluster Managed Kafka

Devoxx UK - Migrating to Multi Cluster Managed Kafka

Devoxx UK - Migrating to Multi Cluster Managed KafkaNatan Silnitsky

Leveraging Mainframe Data for Modern Analytics

Leveraging Mainframe Data for Modern Analytics

Leveraging Mainframe Data for Modern Analyticsconfluent

Introduction to Apache Kafka and Confluent... and why they matter!

Introduction to Apache Kafka and Confluent... and why they matter!

Introduction to Apache Kafka and Confluent... and why they matter!Paolo Castagna

Confluent and Elastic

Confluent and Elastic

Confluent and ElasticPaolo Castagna

Architecture patterns for distributed, hybrid, edge and global Apache Kafka d...

Architecture patterns for distributed, hybrid, edge and global Apache Kafka d...

Architecture patterns for distributed, hybrid, edge and global Apache Kafka d...Kai Wähner

Ähnlich wie Kafka Summit NYC 2017 - Venice: A Distributed Database on top of Kafka (20)

Introducing Venice - Strata NYC 2017

Introducing Venice - Strata NYC 2017

Introducing Venice - Strata NYC 2017

Introducing Venice

Introducing Venice

Introducing Venice

Dev Days Europe - Kafka based Global Data Mesh at Wix

Dev Days Europe - Kafka based Global Data Mesh at Wix

Dev Days Europe - Kafka based Global Data Mesh at Wix

Kafka based Global Data Mesh at Wix with Natan Silnitsky | Kafka Summit Londo...

Kafka based Global Data Mesh at Wix with Natan Silnitsky | Kafka Summit Londo...

Kafka based Global Data Mesh at Wix with Natan Silnitsky | Kafka Summit Londo...

Kafka Summit London - Kafka based Global Data Mesh at Wix

Kafka Summit London - Kafka based Global Data Mesh at Wix

Kafka Summit London - Kafka based Global Data Mesh at Wix

Devoxx Ukraine - Kafka based Global Data Mesh

Devoxx Ukraine - Kafka based Global Data Mesh

Devoxx Ukraine - Kafka based Global Data Mesh

Kafka based Global Data Mesh at Wix

Kafka based Global Data Mesh at Wix

Kafka based Global Data Mesh at Wix

Can Apache Kafka Replace a Database? – The 2021 Update | Kai Waehner, Confluent

Can Apache Kafka Replace a Database? – The 2021 Update | Kai Waehner, Confluent

Can Apache Kafka Replace a Database? – The 2021 Update | Kai Waehner, Confluent

Migrating to Multi Cluster Managed Kafka - DevopStars 2022

Migrating to Multi Cluster Managed Kafka - DevopStars 2022

Migrating to Multi Cluster Managed Kafka - DevopStars 2022

Data Streaming with Apache Kafka & MongoDB - EMEA

Data Streaming with Apache Kafka & MongoDB - EMEA

Data Streaming with Apache Kafka & MongoDB - EMEA

Webinar: Data Streaming with Apache Kafka & MongoDB

Webinar: Data Streaming with Apache Kafka & MongoDB

Webinar: Data Streaming with Apache Kafka & MongoDB

vCloud Technical deck - cb.ppt

vCloud Technical deck - cb.ppt

vCloud Technical deck - cb.ppt

Data analytics at scale implementing stateful stream processing - publish

Data analytics at scale implementing stateful stream processing - publish

Data analytics at scale implementing stateful stream processing - publish

Migrating to Multi Cluster Managed Kafka - ApacheKafkaIL

Migrating to Multi Cluster Managed Kafka - ApacheKafkaIL

Migrating to Multi Cluster Managed Kafka - ApacheKafkaIL

Migrating to Multi Cluster Managed Kafka - Conf42 - CloudNative

Migrating to Multi Cluster Managed Kafka - Conf42 - CloudNative

Migrating to Multi Cluster Managed Kafka - Conf42 - CloudNative

Devoxx UK - Migrating to Multi Cluster Managed Kafka

Devoxx UK - Migrating to Multi Cluster Managed Kafka

Devoxx UK - Migrating to Multi Cluster Managed Kafka

Leveraging Mainframe Data for Modern Analytics

Leveraging Mainframe Data for Modern Analytics

Leveraging Mainframe Data for Modern Analytics

Introduction to Apache Kafka and Confluent... and why they matter!

Introduction to Apache Kafka and Confluent... and why they matter!

Introduction to Apache Kafka and Confluent... and why they matter!

Confluent and Elastic

Confluent and Elastic

Confluent and Elastic

Architecture patterns for distributed, hybrid, edge and global Apache Kafka d...

Architecture patterns for distributed, hybrid, edge and global Apache Kafka d...

Architecture patterns for distributed, hybrid, edge and global Apache Kafka d...

Mehr von confluent

Catch the Wave: SAP Event-Driven and Data Streaming for the Intelligence Ente...

Catch the Wave: SAP Event-Driven and Data Streaming for the Intelligence Ente...

Catch the Wave: SAP Event-Driven and Data Streaming for the Intelligence Ente...confluent

Santander Stream Processing with Apache Flink

Santander Stream Processing with Apache Flink

Santander Stream Processing with Apache Flinkconfluent

Unlocking the Power of IoT: A comprehensive approach to real-time insights

Unlocking the Power of IoT: A comprehensive approach to real-time insights

Unlocking the Power of IoT: A comprehensive approach to real-time insightsconfluent

Workshop híbrido: Stream Processing con Flink

Workshop híbrido: Stream Processing con Flink

Workshop híbrido: Stream Processing con Flinkconfluent

Industry 4.0: Building the Unified Namespace with Confluent, HiveMQ and Spark...

Industry 4.0: Building the Unified Namespace with Confluent, HiveMQ and Spark...

Industry 4.0: Building the Unified Namespace with Confluent, HiveMQ and Spark...confluent

AWS Immersion Day Mapfre - Confluent

AWS Immersion Day Mapfre - Confluent

AWS Immersion Day Mapfre - Confluentconfluent

Eventos y Microservicios - Santander TechTalk

Eventos y Microservicios - Santander TechTalk

Eventos y Microservicios - Santander TechTalkconfluent

Q&A with Confluent Experts: Navigating Networking in Confluent Cloud

Q&A with Confluent Experts: Navigating Networking in Confluent Cloud

Q&A with Confluent Experts: Navigating Networking in Confluent Cloudconfluent

Citi TechTalk Session 2: Kafka Deep Dive

Citi TechTalk Session 2: Kafka Deep Dive

Citi TechTalk Session 2: Kafka Deep Diveconfluent

Build real-time streaming data pipelines to AWS with Confluent

Build real-time streaming data pipelines to AWS with Confluent

Build real-time streaming data pipelines to AWS with Confluentconfluent

Q&A with Confluent Professional Services: Confluent Service Mesh

Q&A with Confluent Professional Services: Confluent Service Mesh

Q&A with Confluent Professional Services: Confluent Service Meshconfluent

Citi Tech Talk: Event Driven Kafka Microservices

Citi Tech Talk: Event Driven Kafka Microservices

Citi Tech Talk: Event Driven Kafka Microservicesconfluent

Confluent & GSI Webinars series - Session 3

Confluent & GSI Webinars series - Session 3

Confluent & GSI Webinars series - Session 3confluent

Citi Tech Talk: Messaging Modernization

Citi Tech Talk: Messaging Modernization

Citi Tech Talk: Messaging Modernizationconfluent

Citi Tech Talk: Data Governance for streaming and real time data

Citi Tech Talk: Data Governance for streaming and real time data

Citi Tech Talk: Data Governance for streaming and real time dataconfluent

Confluent & GSI Webinars series: Session 2

Confluent & GSI Webinars series: Session 2

Confluent & GSI Webinars series: Session 2confluent

Data In Motion Paris 2023

Data In Motion Paris 2023

Data In Motion Paris 2023confluent

Confluent Partner Tech Talk with Synthesis

Confluent Partner Tech Talk with Synthesis

Confluent Partner Tech Talk with Synthesisconfluent

The Future of Application Development - API Days - Melbourne 2023

The Future of Application Development - API Days - Melbourne 2023

The Future of Application Development - API Days - Melbourne 2023confluent

The Playful Bond Between REST And Data Streams

The Playful Bond Between REST And Data Streams

The Playful Bond Between REST And Data Streamsconfluent

Mehr von confluent (20)

Catch the Wave: SAP Event-Driven and Data Streaming for the Intelligence Ente...

Catch the Wave: SAP Event-Driven and Data Streaming for the Intelligence Ente...

Catch the Wave: SAP Event-Driven and Data Streaming for the Intelligence Ente...

Santander Stream Processing with Apache Flink

Santander Stream Processing with Apache Flink

Santander Stream Processing with Apache Flink

Unlocking the Power of IoT: A comprehensive approach to real-time insights

Unlocking the Power of IoT: A comprehensive approach to real-time insights

Unlocking the Power of IoT: A comprehensive approach to real-time insights

Workshop híbrido: Stream Processing con Flink

Workshop híbrido: Stream Processing con Flink

Workshop híbrido: Stream Processing con Flink

Industry 4.0: Building the Unified Namespace with Confluent, HiveMQ and Spark...

Industry 4.0: Building the Unified Namespace with Confluent, HiveMQ and Spark...

Industry 4.0: Building the Unified Namespace with Confluent, HiveMQ and Spark...

AWS Immersion Day Mapfre - Confluent

AWS Immersion Day Mapfre - Confluent

AWS Immersion Day Mapfre - Confluent

Eventos y Microservicios - Santander TechTalk

Eventos y Microservicios - Santander TechTalk

Eventos y Microservicios - Santander TechTalk

Q&A with Confluent Experts: Navigating Networking in Confluent Cloud

Q&A with Confluent Experts: Navigating Networking in Confluent Cloud

Q&A with Confluent Experts: Navigating Networking in Confluent Cloud

Citi TechTalk Session 2: Kafka Deep Dive

Citi TechTalk Session 2: Kafka Deep Dive

Citi TechTalk Session 2: Kafka Deep Dive

Build real-time streaming data pipelines to AWS with Confluent

Build real-time streaming data pipelines to AWS with Confluent

Build real-time streaming data pipelines to AWS with Confluent

Q&A with Confluent Professional Services: Confluent Service Mesh

Q&A with Confluent Professional Services: Confluent Service Mesh

Q&A with Confluent Professional Services: Confluent Service Mesh

Citi Tech Talk: Event Driven Kafka Microservices

Citi Tech Talk: Event Driven Kafka Microservices

Citi Tech Talk: Event Driven Kafka Microservices

Confluent & GSI Webinars series - Session 3

Confluent & GSI Webinars series - Session 3

Confluent & GSI Webinars series - Session 3

Citi Tech Talk: Messaging Modernization

Citi Tech Talk: Messaging Modernization

Citi Tech Talk: Messaging Modernization

Citi Tech Talk: Data Governance for streaming and real time data

Citi Tech Talk: Data Governance for streaming and real time data

Citi Tech Talk: Data Governance for streaming and real time data

Confluent & GSI Webinars series: Session 2

Confluent & GSI Webinars series: Session 2

Confluent & GSI Webinars series: Session 2

Data In Motion Paris 2023

Data In Motion Paris 2023

Data In Motion Paris 2023

Confluent Partner Tech Talk with Synthesis

Confluent Partner Tech Talk with Synthesis

Confluent Partner Tech Talk with Synthesis

The Future of Application Development - API Days - Melbourne 2023

The Future of Application Development - API Days - Melbourne 2023

The Future of Application Development - API Days - Melbourne 2023

The Playful Bond Between REST And Data Streams

The Playful Bond Between REST And Data Streams

The Playful Bond Between REST And Data Streams

Kürzlich hochgeladen

A healthy diet for your Java application Devoxx France.pdf

A healthy diet for your Java application Devoxx France.pdf

A healthy diet for your Java application Devoxx France.pdfMarharyta Nedzelska

PREDICTING RIVER WATER QUALITY ppt presentation

PREDICTING RIVER WATER QUALITY ppt presentation

PREDICTING RIVER WATER QUALITY ppt presentationvaddepallysandeep122

What is Advanced Excel and what are some best practices for designing and cre...

What is Advanced Excel and what are some best practices for designing and cre...

What is Advanced Excel and what are some best practices for designing and cre...Technogeeks

Odoo 14 - eLearning Module In Odoo 14 Enterprise

Odoo 14 - eLearning Module In Odoo 14 Enterprise

Odoo 14 - eLearning Module In Odoo 14 Enterprisepreethippts

UI5ers live - Custom Controls wrapping 3rd-party libs.pptx

UI5ers live - Custom Controls wrapping 3rd-party libs.pptx

UI5ers live - Custom Controls wrapping 3rd-party libs.pptxAndreas Kunz

Recruitment Management Software Benefits (Infographic)

Recruitment Management Software Benefits (Infographic)

Recruitment Management Software Benefits (Infographic)Hr365.us smith

How To Manage Restaurant Staff -BTRESTRO

How To Manage Restaurant Staff -BTRESTRO

How To Manage Restaurant Staff -BTRESTROmotivationalword821

Dealing with Cultural Dispersion — Stefano Lambiase — ICSE-SEIS 2024

Dealing with Cultural Dispersion — Stefano Lambiase — ICSE-SEIS 2024

Dealing with Cultural Dispersion — Stefano Lambiase — ICSE-SEIS 2024StefanoLambiase

SpotFlow: Tracking Method Calls and States at Runtime

SpotFlow: Tracking Method Calls and States at Runtime

SpotFlow: Tracking Method Calls and States at Runtimeandrehoraa

Precise and Complete Requirements? An Elusive Goal

Precise and Complete Requirements? An Elusive Goal

Precise and Complete Requirements? An Elusive GoalLionel Briand

Maximizing Efficiency and Profitability with OnePlan’s Professional Service A...

Maximizing Efficiency and Profitability with OnePlan’s Professional Service A...

Maximizing Efficiency and Profitability with OnePlan’s Professional Service A...OnePlan Solutions

Call Us🔝>༒+91-9711147426⇛Call In girls karol bagh (Delhi)

Call Us🔝>༒+91-9711147426⇛Call In girls karol bagh (Delhi)

Call Us🔝>༒+91-9711147426⇛Call In girls karol bagh (Delhi)jennyeacort

Ahmed Motair CV April 2024 (Senior SW Developer)

Ahmed Motair CV April 2024 (Senior SW Developer)

Ahmed Motair CV April 2024 (Senior SW Developer)Ahmed Mater

Xen Safety Embedded OSS Summit April 2024 v4.pdf

Xen Safety Embedded OSS Summit April 2024 v4.pdf

Xen Safety Embedded OSS Summit April 2024 v4.pdfStefano Stabellini

Post Quantum Cryptography – The Impact on Identity

Post Quantum Cryptography – The Impact on Identity

Post Quantum Cryptography – The Impact on Identityteam-WIBU

Innovate and Collaborate- Harnessing the Power of Open Source Software.pdf

Innovate and Collaborate- Harnessing the Power of Open Source Software.pdf

Innovate and Collaborate- Harnessing the Power of Open Source Software.pdfYashikaSharma391629

SuccessFactors 1H 2024 Release - Sneak-Peek by Deloitte Germany

SuccessFactors 1H 2024 Release - Sneak-Peek by Deloitte Germany

SuccessFactors 1H 2024 Release - Sneak-Peek by Deloitte GermanyChristoph Pohl

Intelligent Home Wi-Fi Solutions | ThinkPalm

Intelligent Home Wi-Fi Solutions | ThinkPalm

Intelligent Home Wi-Fi Solutions | ThinkPalmSujith Sukumaran

Salesforce Implementation Services PPT By ABSYZ

Salesforce Implementation Services PPT By ABSYZ

Salesforce Implementation Services PPT By ABSYZABSYZ Inc

Sending Calendar Invites on SES and Calendarsnack.pdf

Sending Calendar Invites on SES and Calendarsnack.pdf

Sending Calendar Invites on SES and Calendarsnack.pdf31events.com

Kürzlich hochgeladen (20)

A healthy diet for your Java application Devoxx France.pdf

A healthy diet for your Java application Devoxx France.pdf

A healthy diet for your Java application Devoxx France.pdf

PREDICTING RIVER WATER QUALITY ppt presentation

PREDICTING RIVER WATER QUALITY ppt presentation

PREDICTING RIVER WATER QUALITY ppt presentation

What is Advanced Excel and what are some best practices for designing and cre...

What is Advanced Excel and what are some best practices for designing and cre...

What is Advanced Excel and what are some best practices for designing and cre...

Odoo 14 - eLearning Module In Odoo 14 Enterprise

Odoo 14 - eLearning Module In Odoo 14 Enterprise

Odoo 14 - eLearning Module In Odoo 14 Enterprise

UI5ers live - Custom Controls wrapping 3rd-party libs.pptx

UI5ers live - Custom Controls wrapping 3rd-party libs.pptx

UI5ers live - Custom Controls wrapping 3rd-party libs.pptx

Recruitment Management Software Benefits (Infographic)

Recruitment Management Software Benefits (Infographic)

Recruitment Management Software Benefits (Infographic)

How To Manage Restaurant Staff -BTRESTRO

How To Manage Restaurant Staff -BTRESTRO

How To Manage Restaurant Staff -BTRESTRO

Dealing with Cultural Dispersion — Stefano Lambiase — ICSE-SEIS 2024

Dealing with Cultural Dispersion — Stefano Lambiase — ICSE-SEIS 2024

Dealing with Cultural Dispersion — Stefano Lambiase — ICSE-SEIS 2024

SpotFlow: Tracking Method Calls and States at Runtime

SpotFlow: Tracking Method Calls and States at Runtime

SpotFlow: Tracking Method Calls and States at Runtime

Precise and Complete Requirements? An Elusive Goal

Precise and Complete Requirements? An Elusive Goal

Precise and Complete Requirements? An Elusive Goal

Maximizing Efficiency and Profitability with OnePlan’s Professional Service A...

Maximizing Efficiency and Profitability with OnePlan’s Professional Service A...

Maximizing Efficiency and Profitability with OnePlan’s Professional Service A...

Call Us🔝>༒+91-9711147426⇛Call In girls karol bagh (Delhi)

Call Us🔝>༒+91-9711147426⇛Call In girls karol bagh (Delhi)

Call Us🔝>༒+91-9711147426⇛Call In girls karol bagh (Delhi)

Ahmed Motair CV April 2024 (Senior SW Developer)

Ahmed Motair CV April 2024 (Senior SW Developer)

Ahmed Motair CV April 2024 (Senior SW Developer)

Xen Safety Embedded OSS Summit April 2024 v4.pdf

Xen Safety Embedded OSS Summit April 2024 v4.pdf

Xen Safety Embedded OSS Summit April 2024 v4.pdf

Post Quantum Cryptography – The Impact on Identity

Post Quantum Cryptography – The Impact on Identity

Post Quantum Cryptography – The Impact on Identity

Innovate and Collaborate- Harnessing the Power of Open Source Software.pdf

Innovate and Collaborate- Harnessing the Power of Open Source Software.pdf

Innovate and Collaborate- Harnessing the Power of Open Source Software.pdf

SuccessFactors 1H 2024 Release - Sneak-Peek by Deloitte Germany

SuccessFactors 1H 2024 Release - Sneak-Peek by Deloitte Germany

SuccessFactors 1H 2024 Release - Sneak-Peek by Deloitte Germany

Intelligent Home Wi-Fi Solutions | ThinkPalm

Intelligent Home Wi-Fi Solutions | ThinkPalm

Intelligent Home Wi-Fi Solutions | ThinkPalm

Salesforce Implementation Services PPT By ABSYZ

Salesforce Implementation Services PPT By ABSYZ

Salesforce Implementation Services PPT By ABSYZ

Sending Calendar Invites on SES and Calendarsnack.pdf

Sending Calendar Invites on SES and Calendarsnack.pdf

Sending Calendar Invites on SES and Calendarsnack.pdf

Kafka Summit NYC 2017 - Venice: A Distributed Database on top of Kafka

1. Matthew Wise Venice A distributed database on top of Kafka

4. 4 Derived Data

5. 5 Voldemort RO

6. 6 BuildAndPush Versioned Stores

7. 7 Virginia Many Datacenters

8. 8 600+ Stores 100+ TB Pushed Daily 40+ Billion Queries/Day

9. 9 Random Access Low Latency Hadoop

10. 10 Random Access Low Latency Kafka

11. 11 Streaming Ingest

14. V V V Kafka

15. Distributed K/V database Kafka partitioning Versioned pushes Topic maps to version

16. V V V Hadoop

17. MirrorMaker Datacenter Datacenter Datacenter Many Datacenters

18. 18 Venice Requirements Message Buffer Cross Datacenter Mirroring Dynamic Topic Creation Dynamic Topic Deletion Mirrored Topic Creation Mirrored Topic Deletion Scalable Consumption Data Guarantees

20. 20 V Message Buffer

21. 21 M V Mirroring

22. 22 V Partitioned

23. 23 V Topics

24. 24 1 2 3 3 4 5 Data Guarantees: At least once delivery

25. Integration Work

26. 26 M Identity Mirroring

27. 27 1 2 3 4 5 Data Guarantees: Exactly once delivery

28. 28 M Dynamic Topic Deletion

29. Big Architectures

30. 30 Kafka Does Data

31. 31 Data Control Kafka Does Control?

32. Data Control Distributed Systems: Need Control

35. Matthew Wise – linkedin.com/in/matthewrswise Venice A distributed Database on top of Kafka

Hinweis der Redaktion

KIP 98