Diese Präsentation wurde erfolgreich gemeldet.
Wir verwenden Ihre LinkedIn Profilangaben und Informationen zu Ihren Aktivitäten, um Anzeigen zu personalisieren und Ihnen relevantere Inhalte anzuzeigen. Sie können Ihre Anzeigeneinstellungen jederzeit ändern.
Seamless Guest Experience with Kafka Streams
Ram IndukuriHimani Arora Suresh Mulukaladu
About Knoldus
Technology consulting firm with focus on digital transformation
Functional. Reactive. Cloud Native
We are global cruise vacation company that controls and operates four global
brands. Royal Caribbean International, Azamar...
Royal Caribbean International - Facts
● World’s second largest global cruise line
● Represents more than 20% of global cru...
Guest Journey - touchpoints
Discover Book
Cruise
Excursion
s
Check In Board
Ship
Cruise Take
Excursion
s
Debark Create
Mem...
Excalibur
Digital Vision
Discover Book Cruise Pre Cruise
Planning
Check In Board Ship Cruise Take
Excursions
Debark Create
Memories
...
Architectural Challenges
1
How to Standardize Footprints
on all ships and shore ?
3
How do guests have seamless
experience...
Ship-Shore Highway - Event enabling Fleet
M
icroservice
sData
stores
IoT Devices
System of
Record
M
icroservice
s Data
sto...
SoR Ship-Shore Pattern
API Gateway
Micro Service
Data Store
System of
Record
Data Store
System of
Record
Micro Service
Change data Capture - Using Kafka Connect
Web
Call Center
Travel Agent
SQL
Mirror
Kafka
Connect
ConfluentKafka
Kafka
Shore...
Broadcasting events - Using Streams
Web
Call Center
Travel Agent
IIDR
Confluent Kafka
Business Events &
Change Data
Reserv...
Generating Business Events by Combining Topics
Booking ID(pk)
Guest ID
Guest ID(pk)
Name
Address
State Store
State Store
J...
Generating Business Events by Combining Topics
StreamsBuilder builder = new StreamsBuilder();
KStream<String, BookingDetai...
Guest Bookings Stream
Guest Details Stream
Transparent Handling of Late Arrival Records
SLIDING
WINDOW
JoinWindows.of(Dura...
Transparent Handling of Late Arrival Records
Guest Bookings Stream
Guest Details Stream
SLIDING
WINDOW
Grace Period set to...
Transparent Handling of Late Arrival Records
StreamsBuilder builder = new StreamsBuilder();
KStream<String, BookingDetails...
Consecutive Joins on Multiple Topics
Booking ID
Guest ID
Guest ID
Name
Address
State Store
State Store
JOIN
Guest ID
Purch...
Consecutive Joins on Multiple Topics
Booking ID
Guest ID
Guest ID
Name
Address
State Store
State Store
JOIN
Guest ID
Purch...
Transparent Handling of Late Arrival Records
StreamsBuilder builder = new StreamsBuilder();
KStream<String, BookingDetails...
Ensuring Streaming reliability - Graphite/Grafana
Kafka
Stream
App
JMX
Graphite Grafana
● Consumer Lag to detect surges
● ...
Ensuring Data Quality - Apache Spark
Apache Spark Apache Spark
Minio
incremental
Mirroring
1
2
3
4 5
Next Steps - Interactive Queries
Instance 1
Instance 2
Instance 3
GUEST API
Local Kafka
State Store
Local Kafka
State Stor...
Summary
● Real time guest information synchronization across ship and
shore was possible with kafka, connect and replicato...
More Information
Ram Indukuri ram@knoldus.com
Himani Arora himani@knoldus.com
Suresh Mulukuladu smulukuladu@rccl.com
For m...
Nächste SlideShare
Wird geladen in …5
×

Seamless Guest Experience with Kafka Streams (Ramaraju Indukurir and Himani Arora, Knoldus Inc and Suresh Mulukaladu, Royal Caribbean Cruises International) Kafka Summit London 2019

110 Aufrufe

Veröffentlicht am

The cruise industry has unique set of challenges when it comes to deploying products that directly interact with guests across ships. No matter what the challenges are, Applications that run on ships, always need to provide same functionality as on shore. To name few, Each ship is almost a separate datacenter and there are tens of ships. Second, the ships compute capacity is much lower than what an AWS data center can provide and hence any thing that is deployed on ship need to be lean.Third, the bandwidth, though improving, is still a big constraint in transferring data between ship and shore. Fourth, The company’s legacy systems power the core booking functionality and that information is still vital to every feature or application that happens across enterprise including ships.

Hence traditionally cruise industry shied away from real time integration and mostly resorts to a ‘rollover’ process, where in the data gets transferred at the beginning or end of the cruise. But this can’t be the solution with digital. Mobiles need to allow stream of changes real time, as guests go from ship to shore to ship or ship to ship or shore to ship. Our customer, a leading cruise company instituted a team to solve these synchronization issues and also enable event driven architecture. This presentation will address the evolution, idea and implementation of data synchronization and enabling event driven organization using Kafka and streams technologies at its core.

Veröffentlicht in: Technologie
  • Als Erste(r) kommentieren

  • Gehören Sie zu den Ersten, denen das gefällt!

Seamless Guest Experience with Kafka Streams (Ramaraju Indukurir and Himani Arora, Knoldus Inc and Suresh Mulukaladu, Royal Caribbean Cruises International) Kafka Summit London 2019

  1. 1. Seamless Guest Experience with Kafka Streams Ram IndukuriHimani Arora Suresh Mulukaladu
  2. 2. About Knoldus Technology consulting firm with focus on digital transformation Functional. Reactive. Cloud Native
  3. 3. We are global cruise vacation company that controls and operates four global brands. Royal Caribbean International, Azamara club, Celebrity cruises and Silversea Cruises.We are also 50% joint venture owner of German brand TUI Cruises and 49% owner of the spanish brand Pullmantur Cruceros. About Royal Caribbean International
  4. 4. Royal Caribbean International - Facts ● World’s second largest global cruise line ● Represents more than 20% of global cruisers ● 9.5 billion in revenues (2018) ● Serving more than 5 Million guests every year around the globe ● 60 ships in service and another 14 ships on the way ● Oasis class(World’s largest) ships carry close to 6000 guests and 2000 crew
  5. 5. Guest Journey - touchpoints Discover Book Cruise Excursion s Check In Board Ship Cruise Take Excursion s Debark Create Memories ● Research vacation options ● Book the cruise & air reservations ● Plan and Book activities on the cruise ● Complete online check-in and get boarding pass ● Pre cruise activities ● Get to the embarkation port ● Check-in to the cruise and board ● Get to your state room ● Book dinner reservations and activities ● Enjoy the cruise and activities ● Excursions at ports of call ● Debarkation ● Post cruise activities ● Fly back ● Get home
  6. 6. Excalibur
  7. 7. Digital Vision Discover Book Cruise Pre Cruise Planning Check In Board Ship Cruise Take Excursions Debark Create Memories Sensor Wearable TV API Layer Device Layer One App
  8. 8. Architectural Challenges 1 How to Standardize Footprints on all ships and shore ? 3 How do guests have seamless experience on ship(s) and shore if we rely on batch process ? 2 How do we build on what we have today ? Same infrastructure on ship/shore API and Microservices Event driven Architecture with real time synchronization
  9. 9. Ship-Shore Highway - Event enabling Fleet M icroservice sData stores IoT Devices System of Record M icroservice s Data stores IoT Devices System of Record Ship Shore Highway Event enablement, transmission, processing & Analytics across fleet
  10. 10. SoR Ship-Shore Pattern API Gateway Micro Service Data Store System of Record Data Store System of Record Micro Service
  11. 11. Change data Capture - Using Kafka Connect Web Call Center Travel Agent SQL Mirror Kafka Connect ConfluentKafka Kafka Shore Reservation SOR AWS Data Transformer Replicator Kafka Ship
  12. 12. Broadcasting events - Using Streams Web Call Center Travel Agent IIDR Confluent Kafka Business Events & Change Data Reservation SOR AWS Kafka Streams Replicator Kafka
  13. 13. Generating Business Events by Combining Topics Booking ID(pk) Guest ID Guest ID(pk) Name Address State Store State Store JOIN Guest ID Booking ID Name Address
  14. 14. Generating Business Events by Combining Topics StreamsBuilder builder = new StreamsBuilder(); KStream<String, BookingDetails> guestBookings = builder.stream(guestBookingTopic) .selectKey((key, value) -> value.getGuestId()); KStream<String, GuestDetails> guestDetails = builder.stream(guestDetailsTopic); KStream<String, GuestBookingDetails> guestBookingAndDetails = guestBookings .join(guestDetails, (guestBooking, guestDetail) -> /* business event message */, JoinWindows.of(Duration.ofHours(10));
  15. 15. Guest Bookings Stream Guest Details Stream Transparent Handling of Late Arrival Records SLIDING WINDOW JoinWindows.of(Duration.ofHours(10)) Late Arriving Record Event 1Event 2 Event 1Event 2 Event 12 Event 13 Event 13 Event 14 Event 14 Event 15
  16. 16. Transparent Handling of Late Arrival Records Guest Bookings Stream Guest Details Stream SLIDING WINDOW Grace Period set to: JoinWindows.of(Duration.ofHours(10)).grace(Duration.ofHours(24))) Event 1Event 2 Event 1Event 2 Event 12 Event 13 Event 13 Event 14 Event 14 Event 15
  17. 17. Transparent Handling of Late Arrival Records StreamsBuilder builder = new StreamsBuilder(); KStream<String, BookingDetails> guestBookings = builder.stream(guestBookingTopic) .selectKey((key, value) -> value.getGuestId()); KStream<String, GuestDetails> guestDetails = builder.stream(guestDetailsTopic); KStream<String, GuestBookingDetails> guestBookingAndDetails = guestBookings .join(guestDetails, (guestBooking, guestDetail) -> /* business event message */, JoinWindows.of(Duration.ofHours(10) .grace(Duration.ofHours(24)) );
  18. 18. Consecutive Joins on Multiple Topics Booking ID Guest ID Guest ID Name Address State Store State Store JOIN Guest ID Purchase ID Intermediate State Store
  19. 19. Consecutive Joins on Multiple Topics Booking ID Guest ID Guest ID Name Address State Store State Store JOIN Guest ID Purchase ID Intermedi ate Store State Store JOIN Guest ID Booking ID Purchase ID Name Address
  20. 20. Transparent Handling of Late Arrival Records StreamsBuilder builder = new StreamsBuilder(); KStream<String, BookingDetails> guestBookings = builder.stream(guestBookingTopic) .selectKey((key, value) -> value.getGuestId()); KStream<String, PurchaseDetails> guestPurchases = builder.stream(guestPurchasesTopic); .selectKey((key, value) -> value.getGuestId()); KStream<String, GuestDetails> guestDetails = builder.stream(guestDetailsTopic); KStream<String, GuestBookingDetails> guestBookingAndDetails = guestBookings .join(guestDetails, (guestBooking, guestDetail) -> /* business event message with booking id and guest details */, JoinWindows.of(Duration.ofHours(10) .join(guestPurchases, (guestBookingDetail,guestPurchase) -> /* business event message with guest details, booking and purchase details*/, JoinWindows.of(Duration.ofHours(10)));
  21. 21. Ensuring Streaming reliability - Graphite/Grafana Kafka Stream App JMX Graphite Grafana ● Consumer Lag to detect surges ● Idle time threshold to predict Thread health ● Bytes Received in last window indicating Kafka broker issues Lack of activity indicate either broker failure or app failure
  22. 22. Ensuring Data Quality - Apache Spark Apache Spark Apache Spark Minio incremental Mirroring 1 2 3 4 5
  23. 23. Next Steps - Interactive Queries Instance 1 Instance 2 Instance 3 GUEST API Local Kafka State Store Local Kafka State Store Local Kafka State Store host1:4047 host2:5047 host3:4460 Exposing RPC endpoint and information about its local state store Kafka Cluster
  24. 24. Summary ● Real time guest information synchronization across ship and shore was possible with kafka, connect and replicators ● Kafka streams and IIDR helped to reduce the latency in processing the CDC ● Isolate critical connectors into their own connect cluster to reduce risk ● Order guarantee is easier to achieve with streams than connectors. ● Invest in domain driven design ● Include legacy dependencies in critical path for planning feature delivery.
  25. 25. More Information Ram Indukuri ram@knoldus.com Himani Arora himani@knoldus.com Suresh Mulukuladu smulukuladu@rccl.com For more information Blog: https://blog.knoldus.com/join-semantics-kafka-streams/ Git Samples: https://github.com/knoldus/

×