Watch this talk here: https://www.confluent.io/online-talks/building-event-driven-applications-apache-kafka-and-confluent-platform
Apache Kafka® has become the de facto technology for real-time event streaming. Confluent Platform, developed by the creators of Apache Kafka, is an event-streaming platform that enables the ingest and processing of massive amounts of data in real time.
In this session, we will cover the easiest ways to start developing event-driven applications with Apache Kafka using Confluent Platform. We will also demo a contextual event-driven application built using our ecosystem of connectors, REST proxy, and a variety of native clients.
View now to learn:
-How to create Apache Kafka topics in minutes and process event streams in real time
-Check the health of an Apache Kafka broker using Confluent Control Center
-The latest enhancements to Confluent Platform that make it easier to run Apache Kafka at scale
-How to use KSQL, streaming SQL for Apache Kafka, to process event streams in real time using simple SQL queries
22. @gamussa | #eventdriven | @ConfluentINc
22
Integrating Microservices
through the database
● “I have a database and I know how to use it.”
● Eventually causes services to co-mingle.
● Violation the «bounded context»
● Great to use inside a service boundary!
● Terrible for sharing data or negotiating
change.
25. @gamussa | #eventdriven | @ConfluentINc
25
Integrating
microservices via RPC
● Avoids problems of database integration
● Feels natural
● Aligns with the request/response paradigm
● Problem: cascading failures
● Question: how do you debug this system?
● Answer: you build a log.
🤔
36. @gamussa | #eventdriven | @ConfluentINc
36
Ratings Service operating on the event stream
Order
Requested
ratings
raw-rating
Ratings
Service
KStreams API
used to count and
sum ratings as
they arrive
Browser
Webserver
KAFKA
ratings
counts
ratings
sums
ratings
37. @gamussa | #eventdriven | @ConfluentINc
37
Schema Management
Order
Requested
ratings
raw-rating
Ratings
Service
Browser
Webserver
KAFKA
ratings
counts
ratings
sums
ratings
Schema Registry
KStreams API
used to count and
sum ratings as
they arrive
38. @gamussa | #eventdriven | @ConfluentINc
38
Access legacy relation data
Order
Requested
ratings
raw-rating
Ratings
Service
Browser
Webserver
KAFKA
ratings
counts
ratings
sums
ratings
Schema Registry
Connect
Movies
KAFKA
KStreams API
used to count and
sum ratings as
they arrive
39. @gamussa | #eventdriven | @ConfluentINc
39
Materialize tables inside the app
Order
Requested
ratings
raw-rating
Ratings
Service
Browser
Webserver
KAFKA
ratings
counts
ratings
sums
ratings
Schema Registry
Connect
Movies
KAFKA
Lookup table
created inside the
Rating Service
Movies
40. @gamussa | #eventdriven | @ConfluentINc
40
Create a new table, persist it to the log
Order
Requested
ratings
raw-rating
Ratings
Service
Browser
Webserver
KAFKA
ratings
counts
ratings
sums
ratings
Schema Registry
Connect
Movies
KAFKA
Movies
rated-movies
rated-movies
Writable
table
created for
rated
movies
41. @gamussa | #eventdriven | @ConfluentINc
41
Create a new table, persist it to the log
Order
Requested
ratings
raw-rating
Ratings
Service
Browser
Webserver
KAFKA
ratings
counts
ratings
sums
ratings
Schema Registry
Connect
Movies
KAFKA
Movies
rated-movies
rated-movies
TRANSACTION
42. @gamussa | #eventdriven | @ConfluentINc
42
Hydrate a materialized view
Order
Requested
ratings
raw-rating
Ratings
Service
Browser
Webserver
KAFKA
ratings
counts
ratings
sums
ratings
Schema Registry
Connect
Movies
KAFKA
rated-movies
an embedded
database
CDC
47. @gamussa | #eventdriven | @ConfluentINc
47
Dynamic broker
configuration
Why
Helps operators manage their brokers
Features
● Change broker configs that don’t require
restarts
48. @gamussa | #eventdriven | @ConfluentINc
48
License management
Why
Makes managing the subscription license easier and
prevents unexpected expirations
Features
● License checking
● License submission
● Pending expiration alerts (3mo out, 1mo out, weekly
for the last month, daily for the last week)
● Expired license notice
49. @gamussa | #eventdriven | @ConfluentINc
49
Multi-cluster KSQL
Why
Runs queries on 1+ KSQL clusters
Features
● Many-to-1 mapping for KSQL-to-Kafka
Configuration
● confluent.controlcenter.ksql.<name of cluster>.url =
<list of urls>
● confluent.controlcenter.ksql.<name of
cluster>.advertised.url = <list of urls>
50. @gamussa | #eventdriven | @ConfluentINc
50
Multi-cluster Connect
Why
Run, monitor, and manage connectors on 1+ Connect
clusters
Features
● Many-to-1 mapping for Connect-to-Kafka
Configuration
● confluent.controlcenter.connect.<name>.
cluster = <list of urls>