0-330km/h: Porsche's Data Streaming Journey | Sridhar Mamella, Porsche

1. 1 Evolution of Kafka @ Porsche
2. 2 Over The Air One solution as the backbone of digital after-sales. Deployment of updates via OTA. Powered by Apache Kafka.
3. 3 Handling Data in Motion. Product teams set up their own Kafka clusters to reorganize their system architecture, focussing solely on purpose, underestimating management overhead. Wild Growth of Kafka Instances
4. 4 Distributing Data Simplified Enabling the data-driven company. One central platform, providing a unifying data architecture for setting clean standards and enabling cross-boundary data usage with a single source of truth. A central platform as the backbone of a data-driven company
5. 5 Simplicity by Design Streamzilla MirrorMaker Deployment Smooth multi-cluster orchestration Topic Management Minimum toil, maximum yield A single source of truth. The pipeline provides transparency about the cluster state, while increasing productivity and enhancing fault-tolerance & repeatability through automated execution. Cluster Direction Integrated self-healing capabilities
6. Where tradition meets innovation. Thank you for your attention!

Different teams set up their own clusters, focussing on purpose and eventually being overwhelmed by administrative overhead
When cooperation and communication comes to play, the data transportation service has the requirement to be reliable & highly available and to allow data to be delivered in real-time.
Streamzilla as the backbone of data transportation between business departments along all data domains aims to provide exactly those requirements:
All data bundled into a flow of data is being managed by a reliable solution.

This enables individual departments and the organization as a whole to introduce new systems, optimize their quality, minimize time-to-market while using a continuously updated & managed solution.
We built our own GitOps pipeline for Kafka. GitOps is a set of modern best practices for deploying & managing infrastructure declaratively.

Declaration as YAML file
Deployment of all creations, changes and deletions, including repartitioning & changing the replication factor
Compatible with MirrorMaker, no manual input required
ACL declaration in one line instead of multiple lines

Deployment up to the production environment
No SSH needed at any point
In our case connecting to prod via SSH is impossible, so the pipeline is mightier than any developer

No more human error, possibly “just” bugs
No manual input needed, freeing up completion time  increased time-to-market

Hier ansehen