Event-Driven Architectures (EDA ) are perceived as mythical objects that instantly transform your systems into ""real-time"" ones! BUT, come to think of it, aren't they already ""real-time""? I mean, adding an item to the cart is pretty much instant in ( most ) webshops. In fact, EDA solves an entirely different set of problems and with the help of Apache Kafka, we will walk through the (re)evolution path. Microservices are easy to get started with, but once we do, we keep stumbling across the same issues: data access, consistency, and failures ( sounds familiar? ). The solution? Patterns, patterns, patterns … You’ve probably heard about terms such as “Event Notification”, “Event-carried State Transfer”, or even “Event Sourcing”, but how can they be used to solve our problems? And more importantly, how can we use Apache Kafka to take advantage of these patterns? I guess we will find out soon!

1. Bogdan Sucaciu
Deloitte Digital | Pluralsight
Demystifying Event-Driven
Architectures with Apache
Kafka
bsucaciu.com
@bsucaciu
linkedin.com/bogdan-sucaciu

2. Bogdan Sucaciu
Deloitte Digital | Pluralsight
Demystifying Event-Driven
Architectures with Apache
Kafka
bsucaciu.com
@bsucaciu
linkedin.com/bogdan-sucaciu

3. Bogdan Sucaciu
Deloitte Digital | Pluralsight
Demystifying Event-Driven
Architectures with Apache
Kafka
bsucaciu.com
@bsucaciu
linkedin.com/bogdan-sucaciu

4. The Monolith

5. The Monolith
A lot of code
goes in here

6. The Problem
GOOD
BAD
Scalability

7. The Monolith

8. SOA

9. Microservices

10. Challenge
GOOD
BAD
Scalability

11. Challenge -> Solution
GOOD
BAD
Scalability

12. Challenge -> Solution -> New Challenge
GOOD
BAD
Data Access
Scalability

13. Microservices
Coupling
Distributed
Transactions
Timeouts
Dependency Management
Data Ownership

14. Event-Driven Architecture
Software architecture paradigm promoting
production, consumption of, and reaction of
events
- Wikipedia

15. EDA

16. EDA
REST
Queue
REST

17. EDA

18. EDA
Producer
Consumer
Event Stream
Event Stream
Event Stream
Consumer
Producer /
Consumer
Producer /
Consumer

19. EDA
Producer
Consumer
Event Stream
Event Stream
Event Stream
Consumer
Producer /
Consumer
Producer /
Consumer

20. EDA
Producer
Consumer
Event Stream
Event Stream
Event Stream
Consumer
Producer /
Consumer
Producer /
Consumer

21. EDA
Event
Stream
Logs
Topics
Queues
WebSockets
Reactive
APIs
Persisted Events
Reactive Events

22. EDA
Producer
Consumer
Event Stream
Event Stream
Event Stream
Consumer
Producer /
Consumer
Producer /
Consumer

23. EDA with Apache Kafka
Kafka Broker
Topics

24. EDA with Apache Kafka
Kafka Broker Inversion of Control
Fan-out
1 P-type / Event Stream
Producers don’t know
Consumers
Topics

25. Challenge
GOOD
BAD
Data Access
Scalability

26. Challenge -> Solution
GOOD
BAD
Data Access
Scalability

27. Challenge -> Solution -> New Challenge
GOOD
BAD
Data Access Consistency
Scalability

28. The key to building successful EDA is
understanding and accepting
Eventual Consistency

29. Past Present Future
Query Command
Event
A B
Message
A B
A ? A B
A ?B
A B

30. Event Notification

31. Event Producer Event Consumer
private static String TOPIC = "event-notification";
…
Properties props = new Properties();
props.put(BOOTSTRAP_SERVERS_CONFIG, "broker:9092");
props.put(KEY_SERIALIZER_CLASS_CONFIG, StringSerializer.class.getName());
props.put(VALUE_SERIALIZER_CLASS_CONFIG,
StringSerializer.class.getName());
KafkaProducer<String, String> producer = new KafkaProducer<>(props);
ProducerRecord<String, String> producerRecord = new
ProducerRecord<>(TOPIC, key, value);
producer.send(producerRecord);
private static String TOPIC = "event-notification";
…
Properties props = new Properties();
props.put(GROUP_ID_CONFIG, "event.consumer");
props.put(BOOTSTRAP_SERVERS_CONFIG, "broker:9092");
props.put(KEY_DESERIALIZER_CLASS_CONFIG, StringDeserializer.class);
props.put(VALUE_DESERIALIZER_CLASS_CONFIG, StringDeserializer.class);
KafkaConsumer<String, String> consumer = new KafkaConsumer<>(props);
consumer.subscribe(Collections.singletonList(TOPIC));
while(true) {
ConsumerRecords<String, String> records = consumer.poll(ofMillis(100));
records.forEach(record -> processRecord(record));
}

32. Producer Consumer 3
Consumer 2
Consumer 1

33. Producer Consumer 3
Consumer 2
Consumer 1

34. Event-Carried
State Transfer

35. Address Changed Event
P
Kafka Broker
C1
C2
C3
Only new address?
Old & new address?
User info?
?

36. Address Changed Event
P
Only new address
C1
C2
C3
Kafka Broker

37. Address Changed Event
P
C1
C2
C3
Kafka Broker

38. Address Changed Event
P
C1
C2
C3
Kafka Broker

39. Address Changed Event
P
Tax
C2
C3
Kafka Broker

40. Avoid querying back the
source system when using events!

41. Avoid querying back the
source system when using events!
Events still make sense
after 10 years in a datalake!

42. Distributed Source of Truth

43. Event Sourcing

44. Event Sourcing
Initial Amount 0
Add 500
Add 200
Subtract 300
0
Current Amount

45. Event Sourcing
Initial Amount 0
Add 500
Add 200
Subtract 300
Current Amount 500

46. Event Sourcing
Initial Amount 0
Add 500
Add 200
Subtract 300
Current Amount 700

47. Event Sourcing
Initial Amount 0
Add 500
Add 200
Subtract 300
Current Amount 400

48. GIT

49. Event Sourcing
Event Stream
C2
C1 C3

50. Game on
bsucaciu.com
@bsucaciu
linkedin.com/bogdan-sucaciu