Diese Präsentation wurde erfolgreich gemeldet.
Die SlideShare-Präsentation wird heruntergeladen. ×

Apache Kafka and MQTT - Overview, Comparison, Use Cases, Architectures

Anzeige
Anzeige
Anzeige
Anzeige
Anzeige
Anzeige
Anzeige
Anzeige
Anzeige
Anzeige
Anzeige
Anzeige

Hier ansehen

1 von 28 Anzeige

Apache Kafka and MQTT - Overview, Comparison, Use Cases, Architectures

Apache Kafka and MQTT are a perfect combination for many IoT use cases. This presentation covers the pros and cons of both technologies. Various use cases across industries, including connected vehicles, manufacturing, mobility services, and smart city are explored. The examples use different architectures, including lightweight edge scenarios, hybrid integrations, and serverless cloud solutions.

Blog series with more details here:
https://www.kai-waehner.de/blog/2021/03/15/apache-kafka-mqtt-sparkplug-iot-blog-series-part-1-of-5-overview-comparison/

Apache Kafka and MQTT are a perfect combination for many IoT use cases. This presentation covers the pros and cons of both technologies. Various use cases across industries, including connected vehicles, manufacturing, mobility services, and smart city are explored. The examples use different architectures, including lightweight edge scenarios, hybrid integrations, and serverless cloud solutions.

Blog series with more details here:
https://www.kai-waehner.de/blog/2021/03/15/apache-kafka-mqtt-sparkplug-iot-blog-series-part-1-of-5-overview-comparison/

Anzeige
Anzeige

Weitere Verwandte Inhalte

Diashows für Sie (20)

Ähnlich wie Apache Kafka and MQTT - Overview, Comparison, Use Cases, Architectures (20)

Anzeige

Weitere von Kai Wähner (17)

Anzeige

Apache Kafka and MQTT - Overview, Comparison, Use Cases, Architectures

  1. 1. Manufacturing 4.0, Connected Vehicles, Mobility Services, and Smart City Real Time Data Processing and Analytics powered by Apache Kafka and MQTT Kai Waehner Field CTO Confluent kai.waehner@confluent.io linkedin.com/in/kaiwaehner @KaiWaehner www.confluent.io www.kai-waehner.de Dominik Obermaier CTO HiveMQ dominik.obermaier@hivemq.com www.linkedin.com/in/dobermai @dobermai www.hivemq.com 1
  2. 2. Agenda 1. Industry Trends - Connected World, Industry 4.0 2. Manufacturing 3. Connected Vehicles 4. Mobility Services 5. Smart City 6. HiveMQ and Confluent
  3. 3. Agenda 1. Industry Trends - Connected World, Industry 4.0 2. Manufacturing 3. Connected Vehicles 4. Mobility Services 5. Smart City 6. HiveMQ and Confluent
  4. 4. Industry Trends - Connected World 4 People on the Internet
  5. 5. A Streaming Platform is the Underpinning of an Event-driven Architecture Ubiquitous connectivity Globally scalable platform for all event producers and consumers Immediate data access Data accessible to all consumers in real time Single system of record Persistent storage to enable reprocessing of past events Continuous queries Stream processing capabilities for in-line data transformation Microservices DBs SaaS apps Mobile Customer 360 Real-time fraud detection Data warehouse Producers Consumers Database change Microservices events SaaS data Customer experience s Streams of real time events Stream processing apps
  6. 6. Data Movement from Device to Enterprise Services SCADA ERP Enterprise Data Center • MQTT Protocol decouples data integration bindings • Bi-directional data movement between devices and data center • Support for unreliable network MQTT Broker ...
  7. 7. Agenda 1. Industry Trends - Connected World, Industry 4.0 2. Manufacturing 3. Connected Vehicles 4. Mobility Services 5. Smart City 6. HiveMQ and Confluent
  8. 8. Food Value Chain IoT-Based and Data-Driven Single source of truth across the food value chain (in the factories, and across regions) Business critical operations (tracking, calculations, alerts, …)
  9. 9. Daimler - Automotive Manufacturing Vehicle Diagnostic System (VDS) • Critical part of manufacturing process that cannot go down HiveMQ Solution • Efficient way to share information between test devices on factory floor and enterprise IT systems • Test devices behave properly if network connection is dropped and reconnected Results ● Rolled out to 24 factories around the world ● 10,000 testing devices connected ● Generating 470 million messages/month bit.ly/hivemq-daimler
  10. 10. Reference Architecture - Manufacturing
  11. 11. Agenda 1. Industry Trends - Connected World, Industry 4.0 2. Manufacturing 3. Connected Vehicles 4. Mobility Services 5. Smart City 6. HiveMQ and Confluent
  12. 12. ‘My Porsche’ A digital service platform for customers, fans, and enthusiasts https://medium.com/porschedev
  13. 13. Autonomic - Connecting Millions of Cars Transportation Mobility Cloud (TMC) = Accessing connected car data and sending remote commands as the foundation for mobility applications: driver safety, preventive maintenance, fleet management… HiveMQ Solution ● High degree of customization ● Scale to millions of connections ● Easy to create a proof of concept Results ● Millions of cars connected via TMC ● Extensive use of HiveMQ Extension SDK ○ forward all MQTT messages to Kafka ○ messages distributed to different microservices & applications bit.ly/hivemq-autonomic
  14. 14. Reference Architecture - Connected Vehicles
  15. 15. Agenda 1. Industry Trends - Connected World, Industry 4.0 2. Manufacturing 3. Connected Vehicles 4. Mobility Services 5. Smart City 6. HiveMQ and Confluent
  16. 16. DB Musterfirma | Vorname Name | Abteilung | Datum ("Einfügen > Kopf- und Fußzeile") 22 Customer timetable Operational timetable Assignments Railway station knowledge Dispositions Train positions Matching Aggregation Consolidation Apache Kafka Analysis Railway station Trains Mobile Apps Employees Deutsche Bahn AG | Reisendeninformation RI-Plattform Single Point of Truth
  17. 17. ZF - Enabling Next Generation of Mobility Cloud Ecosystem for Connected Vehicles • Cloud-agnostic services without vendor lock-in HiveMQ Solution ● Open extension system for custom extensions ● Flexible deployment model to deploy Connectivity Suite to a public cloud or on-premise ● Implement authorization and authentication the way they wanted Results ● Remote diagnostics for technical insight & management of vehicle performance ● Fleet monitoring ● Secure & reliable middleware between connected vehicles & cloud services bit.ly/hivemq-zf
  18. 18. Reference Architecture - Mobility Services
  19. 19. Other Components Kafka Ecosystem (3) Read Data TensorFlow I/O (5) Deploy Model (2) Preprocess Data (8a) Alert Car Real Time Kafka App TensorFlow Serving gRPC Car Sensor HiveMQ MQTT Broker MQTT Connector Kafka Connect or Confluent Proxy or HiveMQ Plugin Kafka Cluster Kafka Connect Grafana Elastic Search KSQL TensorFlow (04) Train Model TensorFlow Real Time Kafka Streams Application (Java / Scala) Real Time Edge Computing (C / librdkafka) TensorFlow Lite (1) Ingest Data (8b) Alert Driver (e.g. mobile App) (6a) Consume Car Data (6b) All Data (7) Potential Defect 25
  20. 20. https://github.com/kaiwaehner/hivemq-mqtt-tensorflow-kafka-realtime-iot-machine-learning-training-inference or http://bit.ly/kafka-mqtt-ml-demo Try it out in 30 minutes! Demo 100.000 Connected Cars (Kafka + MQTT + TensorFlow) 26
  21. 21. Agenda 1. Industry Trends - Connected World, Industry 4.0 2. Manufacturing 3. Connected Vehicles 4. Mobility Services 5. Smart City 6. HiveMQ and Confluent
  22. 22. NAV (Norwegian Work and Welfare Department): Life is a Stream of Events https://www.confluent.io/kafka-summit-sf18/life-is-a-stream-of-events/
  23. 23. Berlex - Cloud-based Traffic Control Systems Portable, cloud-based traffic signals • Reliable control and monitoring of traffic signals from the cloud HiveMQ Solution ● Ability to run without TLS ● Support for MQTT 5 shared subscriptions ● Clusters can be configured with ease to achieve the high availability ● Consolidate log files of all cluster nodes onto a central Syslog server Results ● Successful deployment in Sweden and Norway planned European expansion ● Real-time information delivery & constant monitoring ● High level of customer satisfaction bit.ly/hivemq-berlex
  24. 24. Reference Architecture - Smart City
  25. 25. Agenda 1. Industry Trends - Connected World, Industry 4.0 2. Manufacturing 3. Connected Vehicles 4. Mobility Services 5. Smart City 6. HiveMQ and Confluent
  26. 26. Confluent Platform Fully Managed Cloud Service Self Managed Software FREEDOM OF CHOICE COMMITTER-DRIVEN EXPERTISE Partners Training Professional Services Enterpris e Support Apache Kafka EFFICIENT OPERATIONS AT SCALE PRODUCTION- STAGE PREREQUISITES UNRESTRICTED DEVELOPER PRODUCTIVITY SQL-based Stream Processing KSQL (ksqlDB) Rich Pre-built Ecosystem Connectors | Hub | Schema Registry Multi-language Development non-Java clients | REST Proxy GUI-driven Mgmt & Monitoring Control Center Flexible DevOps Automation Operator | Ansible Dynamic Performance & Elasticity Auto Data Balancer | Tiered Storage Enterprise-grade Security RBAC | Secrets | Audit logs Data Compatibility Schema Registry | Schema Validation Global Resilience Multi-Region Clusters | Replicator Developer Operator Architect Open Source | Community licensed PARTNERSHIP FOR BUSINESS SUCCESS Complete Engagement Model Revenue / Cost / Risk Impact TCO / ROI Executive Buyer
  27. 27. HiveMQ Platform
  28. 28. Kai Waehner Technology Evangelist kai.waehner@confluent.io linkedin.com/in/kaiwaehner @KaiWaehner www.confluent.io www.kai-waehner.de Dominik Obermaier CTO HiveMQ dominik.obermaier@hivemq.com www.linkedin.com/in/dobermai @dobermai www.hivemq.com 34 Questions? Feedback? Please contact us!

×