Diese Präsentation wurde erfolgreich gemeldet.
Wir verwenden Ihre LinkedIn Profilangaben und Informationen zu Ihren Aktivitäten, um Anzeigen zu personalisieren und Ihnen relevantere Inhalte anzuzeigen. Sie können Ihre Anzeigeneinstellungen jederzeit ändern.

Streaming Data Platforms & Building Data-Centric Products (w/ Brandon O'Brien)

223 Aufrufe

Veröffentlicht am

Presented at Eastside DevOps Meetup Feb 20th at Expedia (https://www.meetup.com/Eastside-DevOps-Meetup/events/237421589/)

This presentation focused on reusable patterns for create a self-service data platform that facilitates data sharing across your organization, to make it easier to build data-powered products faster. Top patterns include making it easier for data producers to get data onto the platform, and to quickly spin up new data consumers using templated Apache Spark and Apache Storm apps.

Veröffentlicht in: Daten & Analysen
  • Als Erste(r) kommentieren

  • Gehören Sie zu den Ersten, denen das gefällt!

Streaming Data Platforms & Building Data-Centric Products (w/ Brandon O'Brien)

  1. 1. Streaming Data Platforms Brandon O’Brien @hakczar Expedia, Inc
  2. 2. Outline  Use Cases  Streaming Data Opportunities & Challenges  Streaming Data Platform at Expedia  Questions?
  3. 3. Outline  Use Cases  Streaming Data Opportunities & Challenges  Streaming Data Platform at Expedia  Questions?
  4. 4. Use Case: Streaming Analytics
  5. 5. Outline  Use Cases  Streaming Data Opportunities & Challenges  Streaming Data Platform at Expedia  Questions?
  6. 6. Streaming Data: Opportunities  Data -> Insights -> Products -> $$$  Fresher data = better  However…
  7. 7. Streaming Data: Challenges  Today’s Challenge: System integration  System level: N^2 integrations  Team level: Roadmap alignment
  8. 8. Streaming Data: Solutions  Solution:  Kafka with Streaming Data Platform  Decoupled systems & roadmaps  Easy publish  Easy consume  Central data depot  Talk Focus:  Streaming Data Platform that makes life easier for Kafka producers and consumers
  9. 9. Outline  Use Cases  Streaming Data Opportunities & Challenges  Streaming Data Platform at Expedia  Questions?
  10. 10. Platform Architecture
  11. 11. Streaming Data Platform  Self-Service  Centralized Data Depot (logical)  DevOps: Primer, CICD, Cloud  Elastic Components  Monitoring: Velocity, Lag
  12. 12. Streaming Data Platform cont’d  Producers  Publish data to HTTP Ingestor endpoint  Ingestor -> Kafka  Ingestor -> S3  Consumers  Create skeleton apps with built-in Kafka integration  CICD pipeline for deployments  Changes the Conversation
  13. 13. Streaming Data Platforms Brandon O’Brien @hakczar Expedia, Inc Questions?

×