Maximizing the Capabilities of Kafka – Real-Time Streaming of Event-Driven Data | Sanjai Marimadaiah, Push Technology

•

0 likes•369 views

Push Technology, a verified gold technology partner, developed the Diffusion Kafka Adapter to meet the evolving requirements of the Kafka community. Diffusion is an Intelligent Event-Data Platform. Using Push’s Kafka Adapter, developers extend Kafka solutions efficiently and securely over the Internet, streaming real-time, event-driven data to millions of end-user apps and IoT devices. The adapter automatically maps Kafka message types to JSON, allowing web, mobile, and IoT clients to securely consume the data stored as rich data structures within Kafka. The adapter enables developers to configure how imported topics are structured, and to import topics that match a regular expression. The adapter can translate data from Kafka topics to Diffusion topics, and from Diffusion to Kafka. The adapter is designed it to make it quick and easy to integrate Kafka with Diffusion. The presentation will provide real-world examples of how the adapter is used to power Kafka in an event-driven world.

Technology

© 2021 Push Technology Ltd
Use Case - Streaming Market Data
Risk Modeling
Market Data
FLAT TOPIC STRUCTURE
Perfect for large volumes of Data
Kafka Streams API
Library to build streaming apps
Responsibility on Consumer
LOG-BASED EVENT-BROKER
Efficient for replaying/distributing
data to services & micro-services
Applications
over the Internet
Audit

© 2021 Push Technology Ltd
Requirements to Distribute Data at Internet Scale
v MASSIVE SCALABILITY
Millions of Topics & Connections
v EFFICIENT USE OF BANDWIDTH
Reduce cost of data consumption
Flow-Control, Delay, Conflation
v REAL-TIME RESPONSE
Latency in milli-seconds
v SIMPLIFIED CLIENT SOFTWARE
Time to Market and Maintenance
v OPEN PROTOCOLS & INTERFACES
MQTT, REST API, HTTP
Pub/Sub, Req-Response
REAL-TIME EVENT BROKER
An event-broker with critical
real-time streaming functionality
DEEP HIERARCHICAL TOPIC TREE
Organize and enrich data for efficient
distribution of specific data at scale
SECURITY : FINE GRAINED & ROLE-BASED
Access to specific parts of data in a JSON
Per-topic permissions in a hierarchical model
CAPABILITIES OF SOLUTION

© 2021 Push Technology Ltd
Extend Kafka with Diffusion® to Distribute Data at Scale
Derivative Pricing
Risk Modeling
Audit
Dashboards
Market Data
Diffusion®
Intelligent Event-Data Platform
Client Applications
over the internet

© 2021 Push Technology Ltd
Diffusion® Extends Kafka to Distribute Data at Scale
BUILT-IN ADAPTER TO KAFKA
Easy to ingest data from Kafka Topics
REAL-TIME EVENT BROKER
Queue-based event broker with Event History,
Pub/Sub, Request-Response, Flow-Control
DELTA STREAMING of EVENT DATA
Sends only bytes changed from previous message
OPEN PROTOCOL SUPPORT
MQTT, REST API, HTTP to server variety of clients
REAL-TIME DATA ENRICHMENT
Deep Hierarchical Topic Trees
DSLs to enrich data and set security policies

© 2021 Push Technology Ltd
Learn more about Kafka + Diffusion®
Go.PushTechnology.com/KafkaSummit
v Kafka Tutorials
v Sample Code on GitHub
v Implement on Diffusion Cloud for FREE

© 2021 Push Technology Ltd
© 2020 Push Technology Ltd Private and Confidential
Thank You

What's hot

Modern data processing applications built on Kafka and InfluxDB deliver the performance, reliability, and flexibility that customers need for robust real-time data pipeline solutions. As the saying goes, the pipeline is greater than the sum of its Kafka and InfluxDB parts. In this session, Russ Savage, Director of Product Management at InfluxData will discuss basic concepts of integrating Kafka and InfluxDB while highlighting how companies are creating fault-tolerant, scalable and fast data pipelines with the power of InfluxDB and Kafka.

Kafka & InfluxDB: BFFs for Enterprise Data Applications | Russ Savage, Influx...

HostedbyConfluent

Government Track Welcome Address

HostedbyConfluent

APIs have become ubiquitous as a way of exposing the capabilities of the enterprise both internally and externally. However, are APIs alone enough? There is a strong resurgence in interest in asynchronous communication and event driven architecture. Applications want to receive events immediately so they can respond in real time, and furthermore they also want the benefit of being decoupled from the availability and performance characteristics of the systems providing that data. However, whilst the way that APIs are socialized, exposed, versioned etc. is well matured in the form of API management technology. We are now on the cusp of seeing first class support for event endpoint management to provide the same sophistication for discovering, exposing and consuming events.

Should we manage events like APIs? | Kim Clark, IBM

HostedbyConfluent

Apache Druid is a high-performance distributed analytics store for modern analytics applications. It supports ingesting millions of events per second and sub-second query processing. Druid supports various types of data sources for ingestion, including Apache Kafka. You can immediately query on stream events once they get ingested into Druid. Since Kafka provides scalable and robust data delivery while Druid supports advanced complex analysis on streams, Kafka and Druid are widely used together for BI and operational analytics use cases, which require interactivity, scalability, real-time, and performance. This talk is based on our real-world experiences building out streaming analytics stacks powering production use cases across many industries.

Druid + Kafka: transform your data-in-motion to analytics-in-motion | Gian Me...

HostedbyConfluent

Every 2 seconds, another person becomes a victim of identity theft. The number of online account takeovers is constantly increasing. In this talk we'll show how stream processing was used to combat this for Tesco, one of Europe's largest retailers. The massive scale of e-commerce makes it an interesting target for malicious users. We implemented a risk-management platform built around Kafka and the Confluent Platform to detect and prevent attacks, including those that come through the website's authentication page. We'll present how this project evolved over 2 years to its current state in production, together with some of the challenges we encountered on the way. As the project has had a couple of phases, we will see and compare alternative designs, summarize their pros and cons, and refer them to well known techniques - like Event Sourcing. We'll discuss the architecture and integration with external systems, before moving onto a detailed examination of the stream processors implementation and key internals such as co-partitioning of data. We'll also cover the role of stack components that we used, including Kafka Connect and Schema Registry, as well as the deployment platform, Kubernetes. Over the course of the talk we will put special emphasis on highlighting key factors to take into consideration when designing data pipelines and stream processing platform.

Risk Management in Retail with Stream Processing (Daniel Jagielski, Virtuslab...

HostedbyConfluent

Data Mess to Data Mesh | Jay Kreps, CEO, Confluent | Kafka Summit Americas 20...

HostedbyConfluent

Kafka Summit NYC 2017 - The Rise of the Streaming Platform

confluent

Kubernetes became the de-facto standard for running cloud-native applications. And many users turn to it also to run stateful applications such as Apache Kafka. You can use different tools to deploy Kafka on Kubernetes - write your own YAML files, use Helm Charts, or go for one of the available operators. But there is one thing all of these have in common. You still need very good knowledge of Kubernetes to make sure your Kafka cluster works properly in all situations. This talk will cover different Kubernetes features such as resources, affinity, tolerations, pod disruption budgets, topology spread constraints and more. And it will explain why they are important for Apache Kafka and how to use them. If you are interested in running Kafka on Kubernetes and do not know all of these, this is a talk for you.

Building a Codeless Log Pipeline w/ Confluent Sink Connector | Pollyanna Vale...

HostedbyConfluent

Event Driven Programming & Architecture

marcoemrich

Confluent Cloud makes Devops engineers lives a lot more easier. Yet moving 1500 microservices, 10K topics and 100K partitions to a multi-cluster Confluent cloud can be a challenge. In this talk you will hear about 5 lessons that Wix has learned in order to successfully meet this challenge. These lessons include: 1. Automation, Automation, Automation - all the process has to be completely automated at such scale 2. Prefer a gradual approach - E.g. migrate topics in small chunks and not all at once. Reduces risks if things go bad 3. Cleanup first - avoid migrating unused topics or topics with too many unnecessary partitions

5 lessons learned for successful migration to Confluent cloud | Natan Silinit...

HostedbyConfluent

Data mesh is a relatively recent term that describes a set of principles that good modern data systems uphold. A kind of “microservices” for the data-centric world. While the data mesh is not technology-specific as a pattern, the building of systems that adopt and implement data mesh principles have a relatively long history under different guises. In this talk, we share our recommendations and picks of what every developer should know about building a streaming data mesh with Kafka. We introduce the four principles of the data mesh: domain-driven decentralization, data as a product, self-service data platform, and federated governance. We then cover topics such as the differences between working with event streams versus centralized approaches and highlight the key characteristics that make streams a great fit for implementing a mesh, such as their ability to capture both real-time and historical data. We’ll examine how to onboard data from existing systems into a mesh, modelling the communication within the mesh, how to deal with changes to your domain’s “public” data, give examples of global standards for governance, and discuss the importance of taking a product-centric view on data sources and the data sets they share.

Apache Kafka and the Data Mesh | Michael Noll, Confluent

HostedbyConfluent

Using Kafka to stream data into TigerGraph, a distributed graph database, is a common pattern in our customers’ data architecture. We have seen the integration in three different layers around TigerGraph’s data flow architecture, and many key use case areas such as customer 360, entity resolution, fraud detection, machine learning, and recommendation engine. Firstly, TigerGraph’s internal data ingestion architecture relies on Kafka as an internal component. Secondly, TigerGraph has a builtin Kafka Loader, which can connect directly with an external Kafka cluster for data streaming. Thirdly, users can use an external Kafka cluster to connect other cloud data sources to TigerGraph cloud database solutions through the built-in Kafka Loader feature. In this session, we will present the high-level architecture in three different approaches and demo the data streaming process.

Comparing three data ingestion approaches where Apache Kafka integrates with ...

HostedbyConfluent

Chris D'Agostino | Kafka Summit 2018 Keynote (Building an Enterprise Streamin...

confluent

Studying the ""how"" of Kafka makes you better at using Kafka, but studying its ""whys"" makes you better at so much more. In looking at the tradeoffs behind a system like Kafka, we learn to reason more clearly about distributed systems and to make high-stakes technology adoption decisions more effectively. These are skills we all want to improve! In this talk, we'll examine trade-offs on which our favorite distributed messaging system takes opinionated positions: - Whether to store data contiguously or using an index - How many storage tiers are best? - Where should metadata live? - And more. It's always useful to dissect a modern distributed system with the goal of understanding it better, and it's even better to learn to deeper architectural principles in the process. Come to this talk for a generous helping of both.

Why Kafka Works the Way It Does (And Not Some Other Way) | Tim Berglund, Conf...

HostedbyConfluent

Introduction to networking options available in Confluent Cloud Self Serve provisioning of confluent Kafka clusters. VPC Peering, VNet Peering, Transit Gateway and Private Link Options for AWS, GCP, Azure networking offering. Caveats of confluent's cloud networking solutions customers should be aware of. Details of two major pieces of the architecture of Confluent Cloud - Data Plane Network and Control Plane.

Confluent Cloud Networking | Rajan Sundaram, Confluent

HostedbyConfluent

Transforming Financial Services with Event Streaming Data

confluent

Legacy systems are the kings of our IT architecture. They rule the evolution of the technology ecosystem that hosts them thanks to the control they have acquired over time on core data and key business processes. Legacy systems however pose severe limits in addressing critical business needs, as well as seizing opportunities for future growth. Digital Integration Hub (DIH) is a quite popular architecture to modernize legacy systems allowing the IT to evolve faster and better integrate modern technologies. In this talk we’ll: - Review DIH general architecture - Illustrate different patterns to offload data from legacy systems to Kafka with pros and cons in term of consistency and latency - Present different ways to consume data offloaded from legacy system to Kafka and their fit with different use cases - Show how to close the loop and use Kafka and CQRS pattern to handle all the data moving backward from applications to legacy systems

From legacy systems to microservices and back | Andera Gioia, Quantyca

HostedbyConfluent

Apache Kafka is getting used as an event backbone in new organizations every day. We would love to send every byte of data through the event bus. Like traditional REST APIs, a contract-first approach is very useful when designing event-driven architectures. In the case of asynchronous APIs, we have the AsyncAPI specification to document the endpoints where the schema of the records become the main part of the contract payload. Microcks allows us to deploy a testing and mocking platform to have a unified view of the endpoints to speed-up application delivery. In this session we will: - Go over the evolution of API specifications - Review the approach for contract-first design with Apache Kafka - Introduce the AsyncAPI specification - Take an overview of an implementation example for automated mocking and testing

Automated Apache Kafka Mocking and Testing with AsyncAPI | Hugo Guerrero, Red...

HostedbyConfluent

One challenge of widespread adoption of any technology within an organisation is balancing organic growth and maintaining standards and best practice. At AO.com - one of the UK's largest online electrical retailers - we’ve invested in tooling to simplify application onboarding into the event processing platform. This includes creating topics, defining access to the platform and supporting our governance functions. A key part of any data function is data governance and discoverability. Through standardised definitions of Kafka topics and use of Avro schemas, we can map which topics exist, what data they contain and who has access to them. This allows us to support multiple cross-functional teams using automatic data gathering. In this session, AO.com and Confluent Professional Services will share how we tackled the challenge of platform adoption and provide hands-on examples of the open-source tools, ""Kafka Clusterstate Tools"" and ""Kafka Streams Inspector"", we developed.

Data governance and discoverability at AO.com | Jon Vines, AO.com and Christo...

HostedbyConfluent

Elastically Scaling Kafka Using Confluent

confluent

What's hot (20)

Kafka & InfluxDB: BFFs for Enterprise Data Applications | Russ Savage, Influx...

Government Track Welcome Address

Should we manage events like APIs? | Kim Clark, IBM

Druid + Kafka: transform your data-in-motion to analytics-in-motion | Gian Me...

Risk Management in Retail with Stream Processing (Daniel Jagielski, Virtuslab...

Data Mess to Data Mesh | Jay Kreps, CEO, Confluent | Kafka Summit Americas 20...

Kafka Summit NYC 2017 - The Rise of the Streaming Platform

Building a Codeless Log Pipeline w/ Confluent Sink Connector | Pollyanna Vale...

Event Driven Programming & Architecture

5 lessons learned for successful migration to Confluent cloud | Natan Silinit...

Apache Kafka and the Data Mesh | Michael Noll, Confluent

Comparing three data ingestion approaches where Apache Kafka integrates with ...

Chris D'Agostino | Kafka Summit 2018 Keynote (Building an Enterprise Streamin...

Why Kafka Works the Way It Does (And Not Some Other Way) | Tim Berglund, Conf...

Confluent Cloud Networking | Rajan Sundaram, Confluent

Transforming Financial Services with Event Streaming Data

From legacy systems to microservices and back | Andera Gioia, Quantyca

Automated Apache Kafka Mocking and Testing with AsyncAPI | Hugo Guerrero, Red...

Data governance and discoverability at AO.com | Jon Vines, AO.com and Christo...

Elastically Scaling Kafka Using Confluent

Similar to Maximizing the Capabilities of Kafka – Real-Time Streaming of Event-Driven Data | Sanjai Marimadaiah, Push Technology

Improving the Reliability of Market Data Subscription Feeds With Ruchir Vani | Current 2022 Nasdaq Cloud Data Service has democratized access to financial data not only for companies but also for researchers and educators. When providing data feeds as a subscription service, there is no out of the box standard for subscribers’ Kafka client configurations or system designs. As you might imagine, ensuing optimal performance with an endless variety of clients comes with many challenges such as: Preventing Client Side Data Loss Performance Inefficiencies Client Recovery Data Feed Discovery Schema Management In this talk we will discuss those challenges and introduce the Nasdaq Cloud Data Service SDK, an Open Source library for Kafka Consumers that tackles these issues and allows for uniform resilience, performance and operations among varied client configurations. https://github.com/Nasdaq/NasdaqCloudDataService-SDK-Java

Improving the Reliability of Market Data Subscription Feeds With Ruchir Vani ...

HostedbyConfluent

Confluent_AWS_ImmersionDay_Q42023.pdf

Ahmed791434

Traditional data pipelines often face scalability issues and challenges related to cost, their monolithic design, and reliance on batch data processing. They also typically operate under the premise that all data needs to be stored in a single centralized data source before it's put to practical use. Confluent Cloud on Amazon Web Services (AWS) provides a fully managed cloud-native platform that helps you simplify the way you build real-time data flows using streaming data pipelines and Apache Kafka.

Build real-time streaming data pipelines to AWS with Confluent

confluent

AWS Data Lake / Lake House + Confluent Cloud for Serverless Apache Kafka. Learn about use cases, architectures, and features. Data must be continuously collected, processed, and reactively used in applications across the entire enterprise - some in real time, some in batch mode. In other words: As an enterprise becomes increasingly software-defined, it needs a data platform designed primarily for "data in motion" rather than "data at rest." Apache Kafka is now mainstream when it comes to data in motion! The Kafka API has become the de facto standard for event-driven architectures and event streaming. Unfortunately, the cost of running it yourself is very often too expensive when you add factors like scaling, administration, support, security, creating connectors...and everything else that goes with it. Resources in enterprises are scarce: this applies to both the best team members and the budget. The cloud - as we all know - offers the perfect solution to such challenges. Most likely, fully-managed cloud services such as AWS S3, DynamoDB or Redshift are already in use. Now it is time to implement "fully-managed" for Kafka as well - with Confluent Cloud on AWS. Building a central integration layer that doesn't care where or how much data is coming from. Implementing scalable data stream processing to gain real-time insights Leveraging fully managed connectors (like S3, Redshift, Kinesis, MongoDB Atlas & more) to quickly access data Confluent Cloud in action? Let's show how ao.com made it happen! Translated with www.DeepL.com/Translator (free version)

Serverless Kafka on AWS as Part of a Cloud-native Data Lake Architecture

Kai Wähner

Solace Singapore User Group: Sumeet Puri

Solace

Cloud computing - dien toan dam may

Nguyen Duong

Parting the mists of cloud computing 030211 print

CloudExpoEurope

Microsoft Windows Azure - Platfrom Appfabric Service Bus And Access Control P...

Microsoft Private Cloud

Banv meetup 04162014

ozkan01

SentinelOne-Connector-For-Fortinet-Launch-Deck-Final.pptx Optimizing slideshow... SentinelOne-Connector-For-Fortinet-Launch-Deck-Final.pptx Optimizing slideshow... SentinelOne-Connector-For-Fortinet-Launch-Deck-Final.pptx Optimizing slideshow... SentinelOne-Connector-For-Fortinet-Launch-Deck-Final.pptx Optimizing slideshow... SentinelOne-Connector-For-Fortinet-Launch-Deck-Final.pptx Optimizing slideshow... SentinelOne-Connector-For-Fortinet-Launch-Deck-Final.pptx Optimizing slideshow... the advanced robot went over the hill to find the bunker filled with explosives

Slides-Discover-Power-of-Live-Data(2).pdf

butthead7

Open Standards Enabling Digital Transformation

Solace

Set Your Data In Motion - CTO Roundtable

confluent

Technical thought leadership presentation to discuss how leading organizations move to real-time architecture to support business growth and enhance customer experience. This is a forum to discuss use cases with your peers to understand how other digital-native companies are utilizing data in motion to drive competitive advantage. Agenda: - Data in Motion with Event Streaming and Apache Kafka - Streaming ETL Pipelines - IT Modernisation and Hybrid Multi-Cloud - Customer Experience and Customer 360 - IoT and Big Data Processing - Machine Learning and Analytics

Event Streaming CTO Roundtable for Cloud-native Kafka Architectures

Kai Wähner

Streaming Data and Stream Processing with Apache Kafka

confluent

Kafka communicates within a larger complex and evolving environment. The current modular approach to the integration means that the structure of the software stack is much more dynamic than in the past and operators no longer have the time to become intimate with how dependent components interact. The number of dependencies combined with lack of familiarity can create significant risks to the business including increased outages and longer time to resolve incidents. Both can result in loss of revenue and customers. These risks are significantly reduced by applying best-practice monitoring. Monitoring can provide a complete end-to-end view of the touch points within the application flow, so they are presented in comprehensive service-based views. This provides the user with a true single-pane of glass for monitoring and alerting for Kafka and its dependent technologies.

Reduce Risk with End to End Monitoring of Middleware-based Applications

SL Corporation

Solution Demonstration Overview - Steve Wallo

scoopnewsgroup

Come learn how OrangeX in Spain is transforming enterprise networks with AWS Cloud and Nokia's Nuage SD-WAN. Nuage's SD-WAN is delivered with management, control, and user plane in the AWS cloud. It is powered by Amazon EC2 i3.metal and AWS hybrid IT services via true multi-tenant, built-in multi-segment orchestrators, and multi-tier management spanning from operators to partners to direct enterprise portals. This enables the solution to be extended directly to service provider private backbones and cloud data centers and services, as well as between enterprise sites. The solution further enables SD-WAN over last mile broadband and leased wholesale circuits to access existing private backbones to leverage their reliable mid-mile connectivity and existing connectivity to enterprise sites. With SD-WAN becoming an essential function in vCPE deployments, Nuage’s SD-WAN has been established as the platform for simplifying the delivery and deployment of virtual network functions, such as security and agility, on-premises, and in the cloud. Learn factors to consider in using SD-WAN solution with AWS, agile implementation of the solution, and leveraging a cloud-ready operating model. We share sufficient references for you to take away to continue learning.

Enterprise Network Transformation Powered by OrangeX, with Nokia Nuage and AW...

Amazon Web Services

Confluent Partner Tech Talk with Reply

confluent

Cloud 12 08 V2

Pini Cohen

This hands on workshop for OpenContrail will be led by Sreelakshmi Sarva & Aniket Daptari. This is a labs session so we will have hard RSVP limits. Please RSVP only if you are confident that you will be able to attend. About Sreelakshmi Sarva Sree is currently working as part of solution engineering team at Juniper’s Contrail team. She is responsible for delivering & managing SDN solutions & partnerships relating to Contrail. She has been with Juniper for the last 13 years working on various Routing, Switching, Network programmability & virtualization platforms. Prior to Juniper, She worked at Nortel networks in the Systems Engineering group. Sree received her Masters in Computer Science from University of Texas at Dallas and Bachelor’s in Computer Science from India. About Aniket Daptari Aniket is currently working as part of Juniper Networks' Contrail Cloud Solutions team. He is responsible for delivering SDN solutions and technology partnerships related to Contrail. He has been with Juniper for the last 3 years working on various Network programmability & virtualization platforms. Prior to Juniper, he worked at Cisco Systems in the Internet Systems Business Unit (Catalyst 6500). Aniket received his Masters in Computer Science from University of Southern California and a graduate certificate in Management Science and Engineering from Stanford University. Course Abstract This session will be the first of a series of OpenContrail hands-on tutorials for developers who want to get deep into OpenContrail code. This “Basic OpenContrail Programming” Hands-on Session will focus on making developers proficient in writing and contributing code for our OpenContrail Project. Session will cover the following areas 1) Contrail Overview · Use Cases · Architecture recap 2) Contrail Hands on · Demo + Hands on - Configuration , VN, VM, Network Policies etc · DevStack introduction

Banv meetup-contrail

nvirters

Similar to Maximizing the Capabilities of Kafka – Real-Time Streaming of Event-Driven Data | Sanjai Marimadaiah, Push Technology (20)

Improving the Reliability of Market Data Subscription Feeds With Ruchir Vani ...

Confluent_AWS_ImmersionDay_Q42023.pdf

Build real-time streaming data pipelines to AWS with Confluent

Serverless Kafka on AWS as Part of a Cloud-native Data Lake Architecture

Solace Singapore User Group: Sumeet Puri

Cloud computing - dien toan dam may

Parting the mists of cloud computing 030211 print

Microsoft Windows Azure - Platfrom Appfabric Service Bus And Access Control P...

Banv meetup 04162014

Slides-Discover-Power-of-Live-Data(2).pdf

Open Standards Enabling Digital Transformation

Set Your Data In Motion - CTO Roundtable

Event Streaming CTO Roundtable for Cloud-native Kafka Architectures

Streaming Data and Stream Processing with Apache Kafka

Reduce Risk with End to End Monitoring of Middleware-based Applications

Solution Demonstration Overview - Steve Wallo

Enterprise Network Transformation Powered by OrangeX, with Nokia Nuage and AW...

Confluent Partner Tech Talk with Reply

Cloud 12 08 V2

Banv meetup-contrail

More from HostedbyConfluent

"In this talk, attendees will be provided with an introduction to Kafka Connect and the basics of Single Message Transforms (SMTs) and how they can be used to transform data streams in a simple and efficient way. SMTs are a powerful feature of Kafka Connect that allow custom logic to be applied to individual messages as they pass through the data pipeline. The session will explain how SMTs work, the types of transformations they can be used for, and how they can be applied in a modular and composable way. Further, the session will discuss where SMTs fit in with Kafka Connect and when they should be used. Examples will be provided of how SMTs can be used to solve common data integration challenges, such as data enrichment, filtering, and restructuring. Attendees will also learn about the limitations of SMTs and when it might be more appropriate to use other tools or frameworks. Additionally, an overview of the alternatives to SMTs, such as Kafka Streams and KSQL, will be provided. This will help attendees make an informed decision about which approach is best for their specific use case. Whether attendees are developers, data engineers, or data scientists, this talk will provide valuable insights into how Kafka Connect and SMTs can help streamline data processing workflows. Attendees will come away with a better understanding of how these tools work and how they can be used to solve common data integration challenges."

Transforming Data Streams with Kafka Connect: An Introduction to Single Messa...

HostedbyConfluent

"While Apache Kafka lacks native support for topic renaming, there are scenarios where renaming topics becomes necessary. This presentation will delve into the utilization of MirrorMaker 2.0 as a solution for renaming Kafka topics. It will illustrate how MirrorMaker 2.0 can efficiently facilitate the migration of messages from the old topic to the new one and how Kafka Connect Metrics can be employed to monitor the mirroring progress. The discussion will encompass the complexity of renaming Kafka topics, addressing certain limitations, and exploring potential workarounds when using MirrorMaker 2.0 for this purpose. Despite not being originally designed for topic renaming, MirrorMaker 2.0 has a suitable solution for renaming Kafka topics. Blog Post : https://engineering.hellofresh.com/renaming-a-kafka-topic-d6ff3aaf3f03"

Renaming a Kafka Topic | Kafka Summit London

HostedbyConfluent

"Trendyol, Turkey's leading e-commerce company, is committed to positively impacting the lives of millions of customers. Our decision-making processes are entirely driven by data. As a data warehouse team, our primary goal is to provide accurate and up-to-date data, enabling the extraction of valuable business insights. We utilize the benefits provided by Kafka and Kafka Connect to facilitate the transfer of data from the source to our analytical environment. We recently transitioned our Kafka Connect clusters from on-premise VMs to Kubernetes. This shift was driven by our desire to effectively manage rapid growth(marked by a growing number of producers, consumers, and daily messages), ensuring proper monitoring and consistency. Consistency is crucial, especially in instances where we employ Single Message Transforms to manipulate records like filtering based on their keys or converting a JSON Object into a JSON string. Monitoring our cluster's health is key and we achieve this through Grafana dashboards and alerts generated through kube-state-metrics. Additionally, Kafka Connect's JMX metrics, coupled with NewRelic, are employed for comprehensive monitoring. The session will aim to explain our approach to NRT data ingestion, outlining the role of Kafka and Kafka Connect, our transition journey to K8s, and methods employed to monitor the health of our clusters."

Evolution of NRT Data Ingestion Pipeline at Trendyol

HostedbyConfluent

"Join our lightning talk to delve into the strategies vital for maintaining a resilient Kafka service. While proactive monitoring is key for issue prevention, failures will still occur. Rapid detection tools will enable you to identify and resolve problems before they impact end-users. This session explores the techniques employed by Kafka cloud providers for this detection, many of which are also applicable if you are managing independent Kafka clusters or applications. The talk focuses on health-checking, a powerful tool that encompasses an application and its monitoring to validate Kafka environment availability. The session navigates through Kafka health-check methods, sharing best practices, identifying common pitfalls, and highlighting the monitoring of critical performance metrics like throughput and latency for early issue detection. Attendees will gain valuable insights into the art of health-checking their Kafka environment, equipping them with the tools to identify and address issues before they escalate into critical problems. We invite all Kafka enthusiasts to join us in this talk to foster a deeper understanding of Kafka health-checking and ensure the continued smooth operation of your Kafka environment."

Ensuring Kafka Service Resilience: A Dive into Health-Checking Techniques

HostedbyConfluent

"Stream processing systems traditionally gave their users the choice between at least once processing and at most once processing: accepting duplicate data or missing data. But ideally we would provide exactly-once processing, where every event in the input data is represented exactly once in the output. Kafka provides a transaction API that enables exactly-once when using Kafka as your source and sink. But this API has turned out to not be well suited for use by high level streaming systems, requiring various work arounds to still provide transactional processing. In this talk, I’ll cover how the transaction API works, and how systems like Arroyo and Flink have used it to build exactly-once support, and how improvements to the transactional API will enable better end-to-end support for consistent stream processing."

Exactly-once Stream Processing with Arroyo and Kafka

HostedbyConfluent

"In this talk, we will explore the exciting world of IoT and computer vision by presenting a unique project: Fish Plays Pokemon. Using an ESP Eye camera connected to an ESP32 and other IoT devices, to monitor fish's movements in an aquarium. This project showcases the power of IoT and computer vision, demonstrating how even a fish can play a popular video game. We will discuss the challenges we faced during development, including real-time processing, IoT device integration, and Kafka message consumption. By the end of the talk, attendees will have a better understanding of how to combine IoT, computer vision, and the usage of a serverless cloud to create innovative projects. They will also learn how to integrate IoT devices with Kafka to simulate keyboard behavior, opening up endless possibilities for real-time interactions between the physical and digital worlds."

Fish Plays Pokemon | Kafka Summit London

HostedbyConfluent

Tiered Storage 101 | Kafla Summit London

HostedbyConfluent

"Real-time 24/7 monitoring and verification of massive data is challenging – even more so for the world’s second largest manufacturer of memory chips and semiconductors. Tolerance levels are incredibly small, any small defect needs to be identified and dealt with immediately. The goal of semiconductor manufacturing is to improve yield and minimize unnecessary work. However, even with real-time data collection, the data was not easy to manipulate by users and it took many days to enable stream processing requests – limiting its usefulness and value to the business. You’ll hear why SK hynix switched to Confluent and how we developed a self-service stream process portal on top of it. Now users have an easy-to-use service to manipulate the data they want. Results have been impressive, stream processing requests are available the same day – previously taking 5 days! We were also able to drive down costs by 10% as stream processing requests no longer require additional hardware. What you’ll take away from our talk: - What were the pain points in the previous environment - How we transitioned to Confluent without service downtime - Creating a self-service stream processing portal built on top of Connect and ksqlDB - Use case of stream process portal"

Building a Self-Service Stream Processing Portal: How And Why

HostedbyConfluent

"Discover how default configurations might impact ingestion times, especially when dealing with large files. We'll explore a real-world scenario with a 20,000,000+ line file, assessing metrics and exploring the bottleneck in the default setup. Understand the intricacies of batch size calculations and how to optimize them based on your unique data characteristics. Walk away with actionable insights as we showcase a practical example, turning a 7-hour ingestion process into a mere 30 minutes for over 30,000,000 records in a Kafka topic. Uncover metrics, configurations, and best practices to elevate the performance of your Kafka Connect CSV source connectors. Don't miss this opportunity to optimize your data pipeline and ensure smooth, efficient data flow."

From the Trenches: Improving Kafka Connect Source Connector Ingestion from 7 ...

HostedbyConfluent

"In order to meet the current and ever-increasing demand for near-zero RPO/RTO systems, a focus on resiliency is critical. While Kafka offers built-in resiliency features, a perfect blend of client and cluster resiliency is necessary in order to achieve a highly resilient Kafka client application. At Fidelity Investments, Kafka is used for a variety of event streaming needs such as core brokerage trading platforms, log aggregation, communication platforms, and data migrations. In this lightening talk, we will discuss the governance framework that has enabled producers and consumers to achieve their SLAs during unprecedented failure scenarios. We will highlight how we automated resiliency tests through chaos engineering and tightly integrated observability dashboards for Kafka clients to analyze and optimize client configurations. And finally, we will summarize the chaos test suite and the ""test, test and test"" mantra that are helping Fidelity Investments reach its goal of a future with zero down-time."

Future with Zero Down-Time: End-to-end Resiliency with Chaos Engineering and ...

HostedbyConfluent

"There are various strategies for securely connecting to Kafka clusters between different networks or over the public internet. Many cloud providers even offer endpoints that privately route traffic between networks and are not exposed to the internet. But, depending on your network setup and how you are running Kafka, these options ... might not be an option! In this session, we’ll discuss how you can use SSH bastions or a self managed PrivateLink endpoint to establish connectivity to your Kafka clusters without exposing brokers directly to the internet. We explain the required network configuration, and show how we at Materialize have contributed to librdkafka to simplify these scenarios and avoid fragile workarounds."

Navigating Private Network Connectivity Options for Kafka Clusters

HostedbyConfluent

"In my talk, we will examine all the stages of building our self-service Streaming Data Platform based on Apache Flink and Kafka Connect, from the selection of a solution for stateful streaming data processing, right up to the successful design of a robust self-service platform, covering the challenges that we’ve met. I will share our experience in providing non-Java developers with a company-wide self-service solution, which allows them to quickly and easily develop their streaming data pipelines. Additionally, I will highlight specific business use cases that would not have been implemented without our platform.0 characters0 characters"

Apache Flink: Building a Company-wide Self-service Streaming Data Platform

HostedbyConfluent

"Almost everyone has heard about large language models, and tens of millions of people have tried out OpenAI ChatGPT and Google Bard. However, the intricate architecture and underlying mathematics driving these remarkable systems remain elusive to many. LLM's are fascinating - so let's grab a drink and find out how these systems are built and dive deep into their inner workings. In the length of time it to enjoy a round of drinks, you'll understand the inner workings of these models. We'll take our first sip of word vectors, enjoy the refreshing taste of the transformer, and drain a glass understanding how these models are trained on phenomenally large quantities of data. Large language models for your streaming application - explained with a little maths and a lot of pub stories"

Explaining How Real-Time GenAI Works in a Noisy Pub

HostedbyConfluent

"Monitoring is a fundamental operation when running Kafka and Kafka applications in production. There are numerous metrics available when using Kafka, however the sheer number is overwhelming, making it challenging to know where to start and how to properly utilise them. This session will introduce you to some of the key metrics that should be monitored and best practices in fine tuning your monitoring. We will delve into which metrics are the indicators for cluster’s availability and performance and are the most helpful when debugging client applications."

TL;DR Kafka Metrics | Kafka Summit London

HostedbyConfluent

Kafka Streams relies on state restoration for maintaining standby tasks as failure recovery mechanism as well as for restoring the state after rebalance scenarios. When you are scaling up or down your application instances, it is necessary to know the current state of the restoration process for each active and standby task in order to prevent a long restoration process as much as possible. During this presentation, you will get an understanding of how KIP-869 provides valuable information about the current active task restoration after a rebalance and KIP-988 opens a window to the continuous process of standby restoration. When you encounter a situation in which you need to choose whether or not to scale up or down your application instances, both KIPs will be an invaluable ally for you.

A Window Into Your Kafka Streams Tasks | KSL

HostedbyConfluent

"In this talk, we will dive into the world of Kafka producer configs and explore how to understand and optimize them for better performance. We will cover the different types of configs, their impact on performance, and how to tune them to achieve the best results. Whether you're new to Kafka or a seasoned pro, this session will provide valuable insights and practical tips for improving your Kafka producer performance. - Introduction to Kafka producer internal and workflow - Understanding the producer configs like linger.ms, batch.size, buffer.memory and their impact on performance - Learning about producer configs like max.block.ms, delivery.timeout.ms, request.timeout.ms and retries to make producer more resilient. - Discuss configs like enable.idempotence, max.in.flight.requests.per.connection and transaction related configs to achieve delivery guarantees. - Q&A session with attendees to address specific questions and concerns."

Mastering Kafka Producer Configs: A Guide to Optimizing Performance

HostedbyConfluent

"Data contracts are one of the hottest topics in the data management community. A data contract is a formal agreement between a data producer and its consumers, aimed at reducing data downtime and improving data quality. Schemas are an important part of data contracts, but they are not the only relevant element. In this talk, we’ll: 1. see why data contracts are so important but also difficult to implement; 2. identify the characteristics of a well-designed data contract: discuss the anatomy of a data contract, its main elements and, how to formally describe them; 3. show how to manage the lifecycle of a data contract leveraging Confluent Platform's services."

Data Contracts Management: Schema Registry and Beyond

HostedbyConfluent

"In the realm of stateful stream processing, Apache Flink has emerged as a powerful and versatile platform. However, the conventional SQL-based approach often limits the full potential of Flink applications. We will delve into the benefits of adopting a code-first approach, which provides developers with greater control over application logic, facilitates complex transformations, and enables more efficient handling of state and time. We will also discuss how the code-first approach can lead to more maintainable and testable code, ultimately improving the overall quality of your Flink applications. Whether you're a seasoned Flink developer or just starting your journey, this talk will provide valuable insights into how a code-first approach can revolutionize your stream processing applications."

Code-First Approach: Crafting Efficient Flink Apps

HostedbyConfluent

"Change Data Capture (CDC) has become a commodity in data engineering, much in part due to the ever-rising success of Debezium [1]. But is that all there is? In this lightning talk, we’ll outline the current state of the CDC ecosystem, and understand why adopting a Debezium alternative is still a hard sell. If you’ve ever wondered what else is out there, but can’t keep up with the sprawling of new tools in the ecosystem; we’ll wrap it up for you! [1] https://debezium.io/"

Debezium vs. the World: An Overview of the CDC Ecosystem

HostedbyConfluent

"Separation of compute and storage has become the de-facto standard in the data industry for batch processing. The addition of tiered storage to open source Apache Kafka is the first step in bringing true separation of compute and storage to the streaming world. In this talk, we'll discuss in technical detail how to take the concept of tiered storage to its logical extreme by building an Apache Kafka protocol compatible system that has zero local disks. Eliminating all local disks in the system requires not only separating storage from compute, but also separating data from metadata. This is a monumental task that requires reimagining Kafka's architecture from the ground up, but the benefits are worth it. This approach enables a stateless, elastic, and serverless deployment model that minimizes operational overhead and also drives inter-zone networking costs to almost zero."

Beyond Tiered Storage: Serverless Kafka with No Local Disks

HostedbyConfluent

More from HostedbyConfluent (20)

Transforming Data Streams with Kafka Connect: An Introduction to Single Messa...

Renaming a Kafka Topic | Kafka Summit London

Evolution of NRT Data Ingestion Pipeline at Trendyol

Ensuring Kafka Service Resilience: A Dive into Health-Checking Techniques

Exactly-once Stream Processing with Arroyo and Kafka

Fish Plays Pokemon | Kafka Summit London

Tiered Storage 101 | Kafla Summit London

Building a Self-Service Stream Processing Portal: How And Why

From the Trenches: Improving Kafka Connect Source Connector Ingestion from 7 ...

Future with Zero Down-Time: End-to-end Resiliency with Chaos Engineering and ...

Navigating Private Network Connectivity Options for Kafka Clusters

Apache Flink: Building a Company-wide Self-service Streaming Data Platform

Explaining How Real-Time GenAI Works in a Noisy Pub

TL;DR Kafka Metrics | Kafka Summit London

A Window Into Your Kafka Streams Tasks | KSL

Mastering Kafka Producer Configs: A Guide to Optimizing Performance

Data Contracts Management: Schema Registry and Beyond

Code-First Approach: Crafting Efficient Flink Apps

Debezium vs. the World: An Overview of the CDC Ecosystem

Beyond Tiered Storage: Serverless Kafka with No Local Disks

Recently uploaded

Evaluating the top large language models.pdf

ChristopherTHyatt

How to Troubleshoot Apps for the Modern Connected Worker

ThousandEyes

BooK Now Call us at +918448380779 to hire a gorgeous and seductive call girl for sex. Take a Delhi Escort Service. The help of our escort agency is mostly meant for men who want sexual Indian Escorts In Delhi NCR. It should be noted that any impersonator will get 100 attention from our Young Girls Escorts in Delhi. They will assume the position of reliable allies. VIP Call Girl With Original Photos Book Tonight +918448380779 Our Cheap Price 1 Hour not available 2 Hours 5000 Full Night 8000 TAG: Call Girls in Delhi, Noida, Gurgaon, Ghaziabad, Connaught Place, Greater Kailash Delhi, Lajpat Nagar Delhi, Mayur Vihar Delhi, Chanakyapuri Delhi, New Friends Colony Delhi, Majnu Ka Tilla, Karol Bagh, Malviya Nagar, Saket, Khan Market, Noida Sector 18, Noida Sector 76, Noida Sector 51, Gurgaon Mg Road, Iffco Chowk Gurgaon, Rajiv Chowk Gurgaon All Delhi Ncr Free Home Deliver

08448380779 Call Girls In Greater Kailash - I Women Seeking Men

Delhi Call girls

Automating Google Workspace (GWS) & more with Apps Script

wesley chun

Bajaj Allianz Life Insurance Company - Insurer Innovation Award 2024

The Digital Insurer

In an era where artificial intelligence (AI) stands at the forefront of business innovation, Information Architecture (IA) is at the core of functionality. See “There’s No AI Without IA” – (from 2016 but even more relevant today) Understanding and leveraging how Information Architecture (IA) supports AI synergies between knowledge engineering and prompt engineering is critical for senior leaders looking to successfully deploy AI for internal and externally facing knowledge processes. This webinar be a high-level overview of the methodologies that can elevate AI-driven knowledge processes supporting both employees and customers. Core Insights Include: Strategic Knowledge Engineering: Delve into how structuring AI's knowledge base is required to prevent hallucinations, enable contextual retrieval of accurate information. This will include discussion of gold standard libraries of use cases support testing various LLMs and structures and configurations of knowledge base. Precision in Prompt Engineering: Learn the art of crafting prompts that direct AI to deliver targeted, relevant responses, thereby optimizing customer experiences and business outcomes. Unified Approach for Enhanced AI Performance: Explore the intersection of knowledge and prompt engineering to develop AI systems that are not only more responsive but also aligned with overarching business strategies. Guiding Principles for Implementation: Equip yourself with best practices, ethical guidelines, and strategic considerations for embedding these technologies into your business ecosystem effectively. This webinar is designed to empower business and technology leaders with the knowledge to harness the full potential of AI, ensuring their organizations not only keep pace with digital transformation but lead the charge. Join us to map a roadmap to fully leverage Information Architecture (IA) and AI chart a course towards a future where AI is a key pillar of strategic innovation and business success.

EIS-Webinar-Prompt-Knowledge-Eng-2024-04-08.pptx

Earley Information Science

This presentations targets students or working professionals. You may know Google for search, YouTube, Android, Chrome, and Gmail, but did you know Google has many developer tools, platforms & APIs? This comprehensive yet still high-level overview outlines the most impactful tools for where to run your code, store & analyze your data. It will also inspire you as to what's possible. This talk is 50 minutes in length.

Powerful Google developer tools for immediate impact! (2023-24 C)

wesley chun

2024: Domino Containers - The Next Step. News from the Domino Container commu...

Martijn de Jong

Axa Assurance Maroc - Insurer Innovation Award 2024

The Digital Insurer

Discord is a free app offering voice, video, and text chat functionalities, primarily catering to the gaming community. It serves as a hub for users to create and join servers tailored to their interests. Discord’s ecosystem comprises servers, each functioning as a distinct online community with its own channels dedicated to specific topics or activities. Users can engage in text-based discussions, voice calls, or video chats within these channels. Understanding Discord Servers Discord servers are virtual spaces where users congregate to interact, share content, and build communities. Servers may revolve around gaming, hobbies, interests, or fandoms, providing a platform for like-minded individuals to connect. Communication Features Discord offers a range of communication tools, including text channels for messaging, voice channels for real-time audio conversations, and video channels for face-to-face interactions. These features facilitate seamless communication and collaboration. What Does NSFW Mean? The acronym NSFW stands for “Not Safe For Work,” indicating content that may be inappropriate for professional or public settings. NSFW Content NSFW content encompasses material that is sexually explicit, violent, or otherwise graphic in nature. It often includes nudity, profanity, or depictions of sensitive topics.

Understanding Discord NSFW Servers A Guide for Responsible Users.pdf

UK Journal

Data Cloud, More than a CDP by Matt Robison

Anna Loughnan Colquhoun

Tech Trends Report 2024 Future Today Institute.pdf

hans926745

With more memory available, system performance of three Dell devices increased, which can translate to a better user experience Conclusion When your system has plenty of RAM to meet your needs, you can efficiently access the applications and data you need to finish projects and to-do lists without sacrificing time and focus. Our test results show that with more memory available, three Dell PCs delivered better performance and took less time to complete the Procyon Office Productivity benchmark. These advantages translate to users being able to complete workflows more quickly and multitask more easily. Whether you need the mobility of the Latitude 5440, the creative capabilities of the Precision 3470, or the high performance of the OptiPlex Tower Plus 7010, configuring your system with more RAM can help keep processes running smoothly, enabling you to do more without compromising performance.

Boost PC performance: How more available memory can improve productivity

Principled Technologies

As privacy and data protection regulations evolve rapidly, organizations operating in multiple jurisdictions face mounting challenges to ensure compliance and safeguard customer data. With state-specific privacy laws coming up in multiple states this year, it is essential to understand what their unique data protection regulations will require clearly. How will data privacy evolve in the US in 2024? How to stay compliant? Our panellists will guide you through the intricacies of these states' specific data privacy laws, clarifying complex legal frameworks and compliance requirements. This webinar will review: - The essential aspects of each state's privacy landscape and the latest updates - Common compliance challenges faced by organizations operating in multiple states and best practices to achieve regulatory adherence - Valuable insights into potential changes to existing regulations and prepare your organization for the evolving landscape

TrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments

TrustArc

presentation ICT roal in 21st century education

jfdjdjcjdnsjd

08448380779 Call Girls In Diplomatic Enclave Women Seeking Men

Delhi Call girls

ProductAnonymous-April2024-WinProductDiscovery-MelissaKlemke

Product Anonymous

08448380779 Call Girls In Civil Lines Women Seeking Men

Delhi Call girls

Presentation on how to chat with PDF using ChatGPT code interpreter

naman860154

The 7 Things I Know About Cyber Security After 25 Years | April 2024

Rafal Los

Recently uploaded (20)

Evaluating the top large language models.pdf

How to Troubleshoot Apps for the Modern Connected Worker

08448380779 Call Girls In Greater Kailash - I Women Seeking Men

Automating Google Workspace (GWS) & more with Apps Script

Bajaj Allianz Life Insurance Company - Insurer Innovation Award 2024

EIS-Webinar-Prompt-Knowledge-Eng-2024-04-08.pptx

Powerful Google developer tools for immediate impact! (2023-24 C)

2024: Domino Containers - The Next Step. News from the Domino Container commu...

Axa Assurance Maroc - Insurer Innovation Award 2024

Understanding Discord NSFW Servers A Guide for Responsible Users.pdf

Data Cloud, More than a CDP by Matt Robison

Tech Trends Report 2024 Future Today Institute.pdf

Boost PC performance: How more available memory can improve productivity

TrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments

presentation ICT roal in 21st century education

08448380779 Call Girls In Diplomatic Enclave Women Seeking Men

ProductAnonymous-April2024-WinProductDiscovery-MelissaKlemke

08448380779 Call Girls In Civil Lines Women Seeking Men

Presentation on how to chat with PDF using ChatGPT code interpreter

The 7 Things I Know About Cyber Security After 25 Years | April 2024

Maximizing the Capabilities of Kafka – Real-Time Streaming of Event-Driven Data | Sanjai Marimadaiah, Push Technology

3. © 2021 Push Technology Ltd Use Case - Streaming Market Data Risk Modeling Market Data FLAT TOPIC STRUCTURE Perfect for large volumes of Data Kafka Streams API Library to build streaming apps Responsibility on Consumer LOG-BASED EVENT-BROKER Efficient for replaying/distributing data to services & micro-services Applications over the Internet Audit

4. © 2021 Push Technology Ltd Requirements to Distribute Data at Internet Scale v MASSIVE SCALABILITY Millions of Topics & Connections v EFFICIENT USE OF BANDWIDTH Reduce cost of data consumption Flow-Control, Delay, Conflation v REAL-TIME RESPONSE Latency in milli-seconds v SIMPLIFIED CLIENT SOFTWARE Time to Market and Maintenance v OPEN PROTOCOLS & INTERFACES MQTT, REST API, HTTP Pub/Sub, Req-Response REAL-TIME EVENT BROKER An event-broker with critical real-time streaming functionality DEEP HIERARCHICAL TOPIC TREE Organize and enrich data for efficient distribution of specific data at scale SECURITY : FINE GRAINED & ROLE-BASED Access to specific parts of data in a JSON Per-topic permissions in a hierarchical model CAPABILITIES OF SOLUTION

5. © 2021 Push Technology Ltd Extend Kafka with Diffusion® to Distribute Data at Scale Derivative Pricing Risk Modeling Audit Dashboards Market Data Diffusion® Intelligent Event-Data Platform Client Applications over the internet

6. © 2021 Push Technology Ltd Diffusion® Extends Kafka to Distribute Data at Scale BUILT-IN ADAPTER TO KAFKA Easy to ingest data from Kafka Topics REAL-TIME EVENT BROKER Queue-based event broker with Event History, Pub/Sub, Request-Response, Flow-Control DELTA STREAMING of EVENT DATA Sends only bytes changed from previous message OPEN PROTOCOL SUPPORT MQTT, REST API, HTTP to server variety of clients REAL-TIME DATA ENRICHMENT Deep Hierarchical Topic Trees DSLs to enrich data and set security policies

Maximizing the Capabilities of Kafka – Real-Time Streaming of Event-Driven Data | Sanjai Marimadaiah, Push Technology

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to Maximizing the Capabilities of Kafka – Real-Time Streaming of Event-Driven Data | Sanjai Marimadaiah, Push Technology

Similar to Maximizing the Capabilities of Kafka – Real-Time Streaming of Event-Driven Data | Sanjai Marimadaiah, Push Technology (20)

More from HostedbyConfluent

More from HostedbyConfluent (20)

Recently uploaded

Recently uploaded (20)

Maximizing the Capabilities of Kafka – Real-Time Streaming of Event-Driven Data | Sanjai Marimadaiah, Push Technology