How to Isolate Tenants in a Data Distribution Platform with Joanna Eriksson
Melden
HostedbyConfluentFolgen
2. Jun 2023•0 gefällt mir•183 views
How to Isolate Tenants in a Data Distribution Platform with Joanna Eriksson
2. Jun 2023•0 gefällt mir•183 views
Downloaden Sie, um offline zu lesen
Melden
Technologie
"I will walk you through how we have achieved tenant isolation in our architecture where we have isolated tenants on an architecture level through Kafka topics and on a software level through threads. We have successfully used this design for years but as with all designs, it has its limitations. I’ll share both the good and the bad of this design. Finally, it’s not enough to talk about tenant isolation without talking about what a tenant actually is, because it’s not as obvious as it might first seem. Schibsted is a company consisting of about 70 subsidiaries and we have been operating with a data mesh-flavored architecture for years. At the heart of this architecture is an internally developed central data integration platform. One common challenge when having this kind of platform is the need for some kind of tenant isolation. Not having tenant isolation could result in a data producer sending bad data halting the processing for all, or a consumer not being able to receive the data, stopping processing for all consumers."
HostedbyConfluentFolgen
Recomendados
Sukumar Nayak-Detailed-Cloud Risk Management and AuditSukumar Nayak
Virtualization and cloud computingDeep Gupta
Cloud Security Architecture.pptxMoshe Ferber
Protecting your data at rest with Apache Kafka by Confluent and Vormetricconfluent
Implementing Domain Events with KafkaAndrei Rugina
MongoDB World 2018: Enterprise Cloud SecurityMongoDB
Más contenido relacionado
Similar a How to Isolate Tenants in a Data Distribution Platform with Joanna Eriksson
Lighthouse 20100120Infrastructure 2.0
Kafka and event driven architecture -apacoug20Vinay Kumar
Kafka and event driven architecture -og yatra20Vinay Kumar
VTU Open Elective 6th Sem CSE - Module 2 - Cloud ComputingSachin Gowda
Application Switching Principles with Proxy Load Balancer: (datacenter in a box)Luc Wijns
In Flux Limiting for a multi-tenant logging serviceDataWorks Summit/Hadoop Summit
Más de HostedbyConfluent
Chill, Distill, No Overkill: Best Practices to Stress Test Kafka with Siva Ku...HostedbyConfluent
Manage Consistent Configurations Across Multiple Kafka Environments with Naga...HostedbyConfluent
The Cost of Kafka’s High Availability on Cloud with Geetha AnneHostedbyConfluent
Case-Study: Building Real-Time Applications at Scale-Cyclist Crash Detection ...HostedbyConfluent
Migrating Your System to the Different Kafka Platforms Available wtih Bhavesh...HostedbyConfluent
Eliminating the Double Write Problem in Apache Kafka Using the Outbox Pattern...HostedbyConfluent
Último
Improving Employee Experiences on Cisco RoomOS Devices, Webex, and Microsoft ...ThousandEyes
Industry 4.0.pdfTery Lockitski
Cloud Composer workshop at Airflow Summit 2023.pdfLeah Cole
info_session_gdsc_tmsl .pptxNikitaSingh741518
Data Formats: Reading and writing JSON – XML - YAMLCSUC - Consorci de Serveis Universitaris de Catalunya
How to Manage Your Offshore Software Development Team EfficientlyCapital Numbers
How to Isolate Tenants in a Data Distribution Platform with Joanna Eriksson
- 1. How to isolate tenants in a data distribution platform? Joanna Eriksson
- 2. Source: Satellite image from Maxar Technologies
- 4. Domain
- 5. Speed
- 6. Destination
- 7. Multiple channels
- 9. Isolating tenants The idea behind tenant isolation is that your SaaS architecture introduces constructs that tightly control access to resources, and block any attempt to access resources of another tenant. Note that tenant isolation is separate from general security mechanisms. Source: Tenant isolation - SaaS Architecture Fundamentals
- 10. Isolating tenants in an internal data platform • Handling failures • Cater for different needs • Cost control • …
- 11. What is a tenant in an internal data platform? • Destination • Domain • Business areas • Event type • Latency requirements • Completeness • …
- 12. Example of separating tenants in practise
- 14. Realtime processing (SLO 5 sec) Elasticity is essential The traffic ≈ 90k/sec 00:00 00:00 00:00 00:00 00:00 00:00 00:00
- 16. Kafka topics ≈ 400 topics 30 brokers
- 17. Topic • One-to-one mapping between topic and destination of the data, also called data sink. • Data sink definition contains routing rule and configuration of destination. BrandX-1: eventType: PulseBase filter: get-client(.) == "brand-x" transform: transforms/brand-x.jslt postFilter: .event_type and (.user_id or .device_id) consumerId: BrandXAmplitude amplitude: apikey: 1234567890 batchSize: 50
- 18. Kafka topics
- 25. Scaling up
- 26. Scaling down
- 27. Fairness • Thousands of threads • JVM Thread Scheduling • Completely Fair Scheduler • Idle time
- 30. Fastforwarding to 2023
- 31. When is this a good design? ✅ Expandible • new data and new destinations ✅ Elasticity ✅ Simple design ✅ Good starting point ✅ Main purpose is handling failing integrations
- 32. When is this design not suitable? ❌ Strict security requirements ❌ Accurate cost control ❌ Very big scale ❌ Big diversity in use cases and requirements
- 33. The future • Hybrid solution • More customization • Isolating failures
- 34. Key points • Thread-consumer model for elasticity • Handle user errors and failing integrations • A tenant can be many things - define what makes sense based on your requirements