Apache Kafka is an open-source distributed event streaming platform used for building real-time data pipelines and streaming apps. It was developed by LinkedIn in 2011 to solve problems with data integration and processing. Kafka uses a publish-subscribe messaging model and is designed to be fast, scalable, and durable. It allows both streaming and storage of data and acts as a central data backbone for large organizations.

1. Introduction to
Apache Kafka
By: Dhaval Thumar
WWW.AIMDEK.COM

2. Agenda
• What is Apache Kafka?
• How Kafka works?
• History
• Kafka Products
• Kafka Core Components

3. What is Apache Kafka?
● Apache Kafka is a distributed streaming Open source platform developed by
Apache Software Foundation, written in Scala and Java
○ A distributed system is a system with multiple components located on
different machines that communicate and coordinate actions via messages
in order to appear as a single system to the end-user.
○ Create real time data streams
○ Processing real time data stream

4. What is Apache Kafka?

5. How Kafka Works?
• Kafka adopted Pub-Sub messaging architecture.
Brocker
Publisher Subscriber
Message Broker
Message Producer Message Consumer

6. History
• Developed by Linkedin in 2011 to solve data integration problem

7. History

8. Kafka Products
• Kafka Broker - Central server system
• Kafka Client - Producer & Consumer Java & Scala Library
• Kafka Connect - Data Integration tool
• Kafka Stream - Java & Scala library for Real time stream processing application
• KSQL

9. Kafka Core Concept
• Producer
• Consumer
• Broker/Server
• Cluster
• Topic
• Partition
• Offset
• Consumer Groups

10. Kafka Producer, Consumer, Broker

11. Kafka Topic
Guys, I am collecting
data from multiple
producers. Which one
do you want?

12. Kafka Topic
Ok, give me the data
sent by the smart-
meter aproducer

13. Kafka Topic Well, the smart meter producer is sending three
different types of data.
1. Current-load-sent every minute
2. Consumed-units-sent every hour
3. Input-current-fluctuations-sent as and
when it happens Which one
do you want?

14. Kafka Topic
Well, give me the
current-load Message

15. Kafka Topic Ok, so you are looking for a current-load
message. That’s great! But we have a bunch of
smart meter producers, and all of them are
sending the current-load message. so which one
of those are you interested in?

16. Kafka Topic

17. Kafka Topic
Give me all the
data that you
received in the
current load

18. Live Demo

19. Questions and Answers

20. www.aimdek.com
AIMDek Technologies Pvt. Ltd.
+91 78747 88766 | +1 844-744-4423
Sales: sales@aimdek.com
General: hello@aimdek.com
AIMDek Technologies Inc.
+1 647-243-6116
Sales: sales@aimdek.com
General: hello@aimdek.com
AIMDek Technologies Inc.
+1 647-243-6116
Sales: sales@aimdek.com
General: hello@aimdek.com
www.aimdek.com www.aimdek.com
Canada India USA