1.
APAC H E KAF KA
THE NEXT 10 YEARS

2.
The World Changed
Cloud
Kafka
Use cases
JavaRaftHardware

3.
Evolution of Kafka

4.
Great Architectures
Have Conceptual Integrity

5.
Our North Star

6.
Picking the right abstraction
opens up the inherent efficiencies
of the hardware itself.

7.
High Performance from First Principles
Optimize
like crazy
Work with the
system, not
against it
Understand
the hardware,
OS, Java

8.
Rethinking Kafka’s
Architecture Principles

9.
Adding brokers will
increase capacity
Multi-tenancy
Linearly, in all
dimensions
Company-wide Scale

10.
Great
documentation
Observable
Easy to
automate
Plays well
with others
Efficient
(= cheap)
Operationally Friendly

11.
Easy to add /
remove brokers
Adapt to changing
workloads
Elasticity

12.
KIP-500
Replace Zookeeper with
Metadata Quorum
Control Plane
KIP-405
Kafka Tiered Storage
Data Plane
Design Considerations in Action

13.
KIP-405
Tiered Storage

14.
KAFKA COLD STORAGE
Add Kafka Broker
Principles in Action: Elasticity

15.
Infinite
history
Pluggable
storage
Easy capacity
planning
Additional Benefits of Storage Layer

16.
What to do with
all the time you’ll save?

17.
ZooKeeper Removal + New Controller

18.
Principles in Action:
Conceptual Integrity
• We love logs
• We want to use logs for
everything

19.
Async communication
via shared log vs.
point-to-point
Materialize metadata
state in memory
10,000,000 partitions
Principles in Action: Scalability

20.
Tuning
GC
Tweaking
servers
Security
Upgrades
Patches
Capacity
Backup

21.
No need to manage
2 distributed systems
History of state changes
in an immutable log
Principles in Action: Operationally Friendly

22.
Next-gen
Kafka
Architecture

23.
Elastic InfiniteIntegrated
The New Kafka

24.
Our Agile Future