Apache Pulsar: A Foundation Backbone for Clever Cloud - Pulsar Virtual Summit Europe 2021 Keynote

•Als PPTX, PDF herunterladen•

0 gefällt mir•222 views

As a Cloud Service Provider, Clever Cloud makes heavy use of messaging in its core architecture. Initially using other messaging solutions, we faced a number of different challenges. Then we discovered Apache Pulsar, which held the promise to solve these challenges. This session will dive into the adoption of Apache Pulsar at Clever Cloud, including how we worked with StreamNative to develop KoP (Kafka on Pulsar), and how we're using Apache Pulsar as the foundation for our cloud backbone.

Technologie

Pulsar Virtual Summit Europe 2021
Apache Pulsar : a foundation backbone for
Clever Cloud
Quentin Adam
CEO Clever Cloud

Iterate Faster
You Write Code, We Run It.

Pulsar Virtual Summit Europe 2021
Usage
Logs & Metrics Async Services Future

Pulsar Virtual Summit Europe 2021
Challenges

Pulsar Virtual Summit Europe 2021
#1 Clients scalability

Pulsar Virtual Summit Europe 2021
#2 Infinite Storage

Pulsar Virtual Summit Europe 2021
... ...
BookKeeper Storage Object Storage
Transparently read a topic across different
tiers

Pulsar Virtual Summit Europe 2021
#3 Queuing or Streaming? : both!

Pulsar Virtual Summit Europe 2021
#4 Multi tenancy

Pulsar Virtual Summit Europe 2021
Biscuit AuthN / AuthZ
● ACL
● Attenuation
● Decentralized
● Revocation
https://github.com/CleverCloud/biscuit-pulsar

Pulsar Virtual Summit Europe 2021
#5 Geo Replication

Pulsar Virtual Summit Europe 2021
Geo Replication
Broke
r
Broke
r
Producers
EU-FR-1
EU-DE-1
US-EAST-
1
Broke
r
Broke
r
Consumers
Brok
er
Brok
er
Consumer
s
Broke
r
Broke
r
Producers
Topic Topic
Topic

Pulsar Virtual Summit Europe 2021
Contributing

Pulsar Virtual Summit Europe 2021
✓ Multi-Tenancy
✓ Security
✓ TLS Encryption
✓ Authentication, Authorization
✓ Geo-replication
✓ Tiered storage
KoP (Kafka on Pulsar) is Kafka with :

Pulsar Virtual Summit Europe 2021
Pulsar-rs

Pulsar Virtual Summit Europe 2021
Topic ACL
Per topic Authorization

Pulsar Virtual Summit Europe 2021
@PierreZ @kannarfr
@gcouprie
Pulsar Contributors

Pulsar Virtual Summit Europe 2021
Building on Pulsar

Pulsar Virtual Summit Europe 2021
Topic as a Service
Messaging made simple
Customer Topic
Service Service
Service
https://www.clever-cloud.com/blog/engineering/2021/10/05/apache-pulsar-public-beta-release/

Pulsar Virtual Summit Europe 2021
Virtual Machine
Pulsar function: another execution level
Clever Cloud’s serverless
Web Assembly (WASM)
(from rust, typescript, kotlin, golang, c++...)
Unikernel
Scheduler
Bare Metal

Pulsar Virtual Summit Europe 2021
Thank you

Empfohlen

Better Kafka Performance Without Changing Any Code | Simon Ritter, AzulHostedbyConfluent

Interactive Analytics on Pulsar with Pulsar SQL - Pulsar Virtual Summit Europ...StreamNative

Stream-Native Processing with Pulsar FunctionsStreamlio

Twitter’s Apache Kafka Adoption Journey | Ming Liu, TwitterHostedbyConfluent

Everything you ever needed to know about Kafka on Kubernetes but were afraid ...HostedbyConfluent

Can Kafka Handle a Lyft Ride? (Andrey Falko & Can Cecen, Lyft) Kafka Summit 2020HostedbyConfluent

Utilizing Kafka Connect to Integrate Classic Monoliths into Modern Microservi...HostedbyConfluent

Tradeoffs in Distributed Systems Design: Is Kafka The Best? (Ben Stopford and...HostedbyConfluent

Empfohlen

Better Kafka Performance Without Changing Any Code | Simon Ritter, AzulHostedbyConfluent

Interactive Analytics on Pulsar with Pulsar SQL - Pulsar Virtual Summit Europ...StreamNative

Stream-Native Processing with Pulsar FunctionsStreamlio

Twitter’s Apache Kafka Adoption Journey | Ming Liu, TwitterHostedbyConfluent

Everything you ever needed to know about Kafka on Kubernetes but were afraid ...HostedbyConfluent

Can Kafka Handle a Lyft Ride? (Andrey Falko & Can Cecen, Lyft) Kafka Summit 2020HostedbyConfluent

Utilizing Kafka Connect to Integrate Classic Monoliths into Modern Microservi...HostedbyConfluent

Tradeoffs in Distributed Systems Design: Is Kafka The Best? (Ben Stopford and...HostedbyConfluent

Apache Pulsar at Yahoo! JapanStreamNative

Introducing KSML: Kafka Streams for low code environments | Jeroen van Dissel...HostedbyConfluent

Tips & Tricks for Apache Kafka®confluent

Not Your Mother's Kafka - Deep Dive into Confluent Cloud Infrastructure | Gwe...HostedbyConfluent

Monitoring and Resiliency Testing our Apache Kafka Clusters at Goldman Sachs ...HostedbyConfluent

A Modern C++ Kafka API | Kenneth Jia, Morgan StanleyHostedbyConfluent

Connect at Twitter-scale | Jordan Bull and Ryanne Dolan, TwitterHostedbyConfluent

Function Mesh for Apache Pulsar, the Way for Simple Streaming SolutionsStreamNative

Event-driven Applications with Kafka, Micronaut, and AWS Lambda | Dave Klein,...HostedbyConfluent

Simplifying Migration from Kafka to Pulsar - Pulsar Summit NA 2021StreamNative

Using the FLiPN Stack for Edge AI (Flink, NiFi, Pulsar) - Pulsar Summit Asia ...StreamNative

Administrative techniques to reduce Kafka costs | Anna Kepler, ViasatHostedbyConfluent

Streaming with Spring Cloud Stream and Apache Kafka - Soby ChackoVMware Tanzu

SFBigAnalytics_20190724: Monitor kafka like a ProChester Chen

Apache Pulsar with MQTT for Edge Computing - Pulsar Summit Asia 2021StreamNative

Failing to Cross the Streams – Lessons Learned the Hard Way | Philip Schmitt,...HostedbyConfluent

Better Kafka Performance Without Changing Any Code | Simon Ritter, AzulHostedbyConfluent

Give Your Confluent Platform Superpowers! (Sandeep Togrika, Intel and Bert Ha...HostedbyConfluent

Oops! I started a broker | Yinon Kahta, TaboolaHostedbyConfluent

Changing landscapes in data integration - Kafka Connect for near real-time da...HostedbyConfluent

Containers and Nutanix - Acropolis Container ServicesNEXTtour

Docker meetup-20-apr-17-openshitYusuf Hadiwinata Sutandar

Weitere ähnliche Inhalte

Was ist angesagt?