In this presentation Adam Dubiel shares some insight on how monitoring works in Allegro microservices based architecture. He introduces also "Hermes" - messaging solution based on Apache Kafka and overall thoughts when working with this kind of software. Enjoy!
40. 1st iteration - Kafka leaders
Kafka does not distribute leaders uniformly
simple take-first algorithm
no tools to make it happen
go with #!/usr/bin/python
41. 2nd iteration - buffering
simple solution to complex problem
use Kafka producer buffer
we can operate without Kafka for 1h!
42.
43. and when we thought we have it all covered..
bam!
corrupt message logs
44. corrupt message logs
Kafka High Level Consumer can’t skip corrupt parts
no easy solution
time pressure - 24h retency
#!/usr/bin/python to the rescue!