5 lessons learned for successful migration to Confluent cloud | Natan Silinitsky, wix.com

1. 5 lessons learned for Successful Migration to Conﬂuent Cloud Natan Silnitsky Backend Infra Developer, Wix.com natans@wix.com twitter@NSilnitsky linkedin/natansilnitsky natansil.com

2. >200M Registered users (website builders) from 190 countries @NSilnitsky At Wix >2,000M Kafka messages produced / Day 2,000 Microservices

3. * easily manage

4. Data Center 2 Data Center 1 @NSilnitsky Confluent Cloud Kafka Cluster On premise Kafka Cluster I Kafka Cluster II Kafka Cluster III Kafka Cluster IV * AWS/GCP Microservice Microservice 2000

5. Automation, Automation, Automation 01

6. A Scala/Java high-level SDK for Apache Kafka. Greyhound @NSilnitsky

7. Site Apps Service Kafka Consumer Kafka Producer Greyhound wraps Kafka Ecom Catalog Service Kafka Broker Greyhound Producer Greyhound Consumer @NSilnitsky

8. Data Center 2 Data Center 1 Microservice @NSilnitsky Confluent Cloud Kafka Cluster On premise Kafka Cluster I Kafka Cluster II Kafka Cluster III Kafka Cluster IV * all at once?

9. Gradual over All at once 02

10. Data Center 2 Data Center 1 @NSilnitsky Kafka Cluster On premise Confluent Cloud Microservice HTTP HTTP Kafka Cluster I Kafka Cluster II Kafka Cluster III Kafka Cluster IV Option B Replicator Topic 5 Topic 4 Topic 3 Topic 2 Topic 1 Topic 6 Migration Orchestrator * self service - no downtime

11. First migrate test topics with relayed real traﬃc 03

12. Data Center 2 @NSilnitsky Kafka Cluster On premise Confluent Cloud Test app Real Topic->Test Topic Test Topics Kafka Cluster I Kafka Cluster II Kafka Cluster III Kafka Cluster IV Migration Orchestrator Replicator

13. Only needed topics/partitions 04

14. Data Center 2 Data Center 1 Microservice @NSilnitsky 0 Bytes 100 MB 170 MB Confluent Cloud

15. Adapt to Conﬂuent Cloud APIs 05

16. Data Center 2 Data Center 1 Offset lag Monitoring @NSilnitsky Confluent Cloud Kafka Cluster On premise __consumer_offsets __consumer_offsets Kafka Admin SDK Kafka Admin SDK Unavailable Available

17. So, We used Greyhound and dedicated orchestration services to have automatic, safe, and gradual migration to Conﬂuent Cloud @NSilnitsky

18. A Scala/Java high-level SDK for Apache Kafka. 0.1.5 is out! github.com/wix/greyhound

19. Thank You natans@wix.com twitter @NSilnitsky linkedin/natansilnitsky github.com/natansil

20. Slides & More slideshare.net/NatanSilnitsky medium.com/@natansil twitter.com/NSilnitsky natansil.com

5 lessons learned for successful migration to Confluent cloud | Natan Silinitsky, wix.com

Du liest eine Vorschau.

Hier ansehen

5 lessons learned for successful migration to Confluent cloud | Natan Silinitsky, wix.com

Weitere Verwandte Inhalte

Diashows für Sie (19)

Ähnlich wie 5 lessons learned for successful migration to Confluent cloud | Natan Silinitsky, wix.com (20)

Weitere von HostedbyConfluent (20)

Aktuellste (20)