SlideShare a Scribd company logo
1 of 8
Download to read offline
GitOps with Confluent and
Kubernetes
Rohit Bakhshi, Confluent
Kafka Summit 2021
What is GitOps?
Continuous Deployment for cloud native applications
● Declarative Specs (Infrastructure as Code) in Git
● Automation to deploy and maintain to match the declarative spec
An API-driven Control Plane to deploy and manage Confluent
● Declarative API (CRDs) for Confluent Platform Services
● Declarative API (CRDs) for topics and RBAC policies
Integrates with Cloud-Native ecosystem for Security, Reliability, DevOps Automation
● Complete automated secure setup
● Secrets Management with Kubernetes Secrets and Hashicorp Vault
● One command scale up
● Fault tolerance to zone and node failure
Introducing Confluent for Kubernetes
An API-driven Control Plane to deploy and manage Confluent
● Declarative API (CRDs) for Confluent Platform Services
● Declarative API (CRDs) for topics and RBAC policies
Integrates with Cloud-Native ecosystem for Security, Reliability, DevOps Automation
● Complete automated secure setup
● Secrets Management with Kubernetes Secrets and Hashicorp Vault
● One command scale up
● Fault tolerance to zone and node failure
Introducing Confluent Operator 2.0
Confluent + Kubernetes => GitOps
5
Confluent Operator
Component
(Custom Resource)
Credentials
kafka.yaml
---
kafka:
spec:
listeners:
internal:
authentication:
type: plain
jaasConfig:
secretRef: credential
Helm
K8s Secrets
K8s API
Continuous Deployment on Kubernetes
Flux CD
Operator
Git Repo
Kafka CR
Zookeeper CR
ControlCenter CR
Confluent
Operator
KafkaTopic CRs
/cluster/prod
/cluster/qa
LDAP
Demo
QA
- SASL/Plain authentication
- TLS encryption with automatically
generated certs
Prod
- SASL/Plain authentication
- TLS encryption with custom certs
- Granular Role Based Access Control
Control Center
ZooKeeper x 3
Confluent Operator
Kubernetes
Kafka Broker x 3
Schema Registry
Connect
KsqlDB
Try it and Learn More!
Try Confluent for
Kubernetes
cnfl.io/download
Confluent Blog
cnfl.io/blog
Community
cnfl.io/meetups

More Related Content

What's hot

From legacy systems to microservices and back | Andera Gioia, Quantyca
From legacy systems to microservices and back | Andera Gioia, QuantycaFrom legacy systems to microservices and back | Andera Gioia, Quantyca
From legacy systems to microservices and back | Andera Gioia, QuantycaHostedbyConfluent
 
Building a Data Subscription Service with Kafka Connect (Danica Fine & Ajay V...
Building a Data Subscription Service with Kafka Connect (Danica Fine & Ajay V...Building a Data Subscription Service with Kafka Connect (Danica Fine & Ajay V...
Building a Data Subscription Service with Kafka Connect (Danica Fine & Ajay V...confluent
 
Rethinking Geo-replication for the Cloud | Luke Knepper, Confluent
Rethinking Geo-replication for the Cloud | Luke Knepper, ConfluentRethinking Geo-replication for the Cloud | Luke Knepper, Confluent
Rethinking Geo-replication for the Cloud | Luke Knepper, ConfluentHostedbyConfluent
 
Streaming with Structure | Kate Stanley and Salma Saeed, IBM
Streaming with Structure | Kate Stanley and Salma Saeed, IBMStreaming with Structure | Kate Stanley and Salma Saeed, IBM
Streaming with Structure | Kate Stanley and Salma Saeed, IBMHostedbyConfluent
 
Why Cloud-Native Kafka Matters: 4 Reasons to Stop Managing it Yourself
Why Cloud-Native Kafka Matters: 4 Reasons to Stop Managing it YourselfWhy Cloud-Native Kafka Matters: 4 Reasons to Stop Managing it Yourself
Why Cloud-Native Kafka Matters: 4 Reasons to Stop Managing it YourselfDATAVERSITY
 
Kafka Vienna Meetup 020719
Kafka Vienna Meetup 020719Kafka Vienna Meetup 020719
Kafka Vienna Meetup 020719Patrik Kleindl
 
Achieving end-to-end visibility into complex event-sourcing transactions usin...
Achieving end-to-end visibility into complex event-sourcing transactions usin...Achieving end-to-end visibility into complex event-sourcing transactions usin...
Achieving end-to-end visibility into complex event-sourcing transactions usin...HostedbyConfluent
 
How a distributed graph analytics platform uses Apache Kafka for data ingesti...
How a distributed graph analytics platform uses Apache Kafka for data ingesti...How a distributed graph analytics platform uses Apache Kafka for data ingesti...
How a distributed graph analytics platform uses Apache Kafka for data ingesti...HostedbyConfluent
 
Kafka for connected vehicle research | Pavle Bujanovic, Federal Highway Admin...
Kafka for connected vehicle research | Pavle Bujanovic, Federal Highway Admin...Kafka for connected vehicle research | Pavle Bujanovic, Federal Highway Admin...
Kafka for connected vehicle research | Pavle Bujanovic, Federal Highway Admin...HostedbyConfluent
 
Should we manage events like APIs? | Kim Clark, IBM
Should we manage events like APIs? | Kim Clark, IBMShould we manage events like APIs? | Kim Clark, IBM
Should we manage events like APIs? | Kim Clark, IBMHostedbyConfluent
 
Government Track Welcome Address
Government Track Welcome AddressGovernment Track Welcome Address
Government Track Welcome AddressHostedbyConfluent
 
Speed-Up Kafka Delivery with AsyncAPI & Microcks | Hugo Guerrero, Red Hat
Speed-Up Kafka Delivery with AsyncAPI & Microcks | Hugo Guerrero, Red HatSpeed-Up Kafka Delivery with AsyncAPI & Microcks | Hugo Guerrero, Red Hat
Speed-Up Kafka Delivery with AsyncAPI & Microcks | Hugo Guerrero, Red HatHostedbyConfluent
 
Toward Hybrid Cloud Serverless Transparency with Lithops Framework
Toward Hybrid Cloud Serverless Transparency with Lithops FrameworkToward Hybrid Cloud Serverless Transparency with Lithops Framework
Toward Hybrid Cloud Serverless Transparency with Lithops FrameworkLibbySchulze
 
Application Modernization Using Event Streaming Architecture (David Wadden, V...
Application Modernization Using Event Streaming Architecture (David Wadden, V...Application Modernization Using Event Streaming Architecture (David Wadden, V...
Application Modernization Using Event Streaming Architecture (David Wadden, V...HostedbyConfluent
 
Kafka & InfluxDB: BFFs for Enterprise Data Applications | Russ Savage, Influx...
Kafka & InfluxDB: BFFs for Enterprise Data Applications | Russ Savage, Influx...Kafka & InfluxDB: BFFs for Enterprise Data Applications | Russ Savage, Influx...
Kafka & InfluxDB: BFFs for Enterprise Data Applications | Russ Savage, Influx...HostedbyConfluent
 
Mind the App: How to Monitor Your Kafka Streams Applications | Bruno Cadonna,...
Mind the App: How to Monitor Your Kafka Streams Applications | Bruno Cadonna,...Mind the App: How to Monitor Your Kafka Streams Applications | Bruno Cadonna,...
Mind the App: How to Monitor Your Kafka Streams Applications | Bruno Cadonna,...HostedbyConfluent
 
Why Kafka Works the Way It Does (And Not Some Other Way) | Tim Berglund, Conf...
Why Kafka Works the Way It Does (And Not Some Other Way) | Tim Berglund, Conf...Why Kafka Works the Way It Does (And Not Some Other Way) | Tim Berglund, Conf...
Why Kafka Works the Way It Does (And Not Some Other Way) | Tim Berglund, Conf...HostedbyConfluent
 
apidays LIVE New York 2021 - Service reliability through autoscaling workload...
apidays LIVE New York 2021 - Service reliability through autoscaling workload...apidays LIVE New York 2021 - Service reliability through autoscaling workload...
apidays LIVE New York 2021 - Service reliability through autoscaling workload...apidays
 
스타트업을 위한 Confluent 세미나
스타트업을 위한 Confluent 세미나스타트업을 위한 Confluent 세미나
스타트업을 위한 Confluent 세미나confluent
 

What's hot (20)

From legacy systems to microservices and back | Andera Gioia, Quantyca
From legacy systems to microservices and back | Andera Gioia, QuantycaFrom legacy systems to microservices and back | Andera Gioia, Quantyca
From legacy systems to microservices and back | Andera Gioia, Quantyca
 
Building a Data Subscription Service with Kafka Connect (Danica Fine & Ajay V...
Building a Data Subscription Service with Kafka Connect (Danica Fine & Ajay V...Building a Data Subscription Service with Kafka Connect (Danica Fine & Ajay V...
Building a Data Subscription Service with Kafka Connect (Danica Fine & Ajay V...
 
Rethinking Geo-replication for the Cloud | Luke Knepper, Confluent
Rethinking Geo-replication for the Cloud | Luke Knepper, ConfluentRethinking Geo-replication for the Cloud | Luke Knepper, Confluent
Rethinking Geo-replication for the Cloud | Luke Knepper, Confluent
 
Streaming with Structure | Kate Stanley and Salma Saeed, IBM
Streaming with Structure | Kate Stanley and Salma Saeed, IBMStreaming with Structure | Kate Stanley and Salma Saeed, IBM
Streaming with Structure | Kate Stanley and Salma Saeed, IBM
 
Why Cloud-Native Kafka Matters: 4 Reasons to Stop Managing it Yourself
Why Cloud-Native Kafka Matters: 4 Reasons to Stop Managing it YourselfWhy Cloud-Native Kafka Matters: 4 Reasons to Stop Managing it Yourself
Why Cloud-Native Kafka Matters: 4 Reasons to Stop Managing it Yourself
 
Kafka Vienna Meetup 020719
Kafka Vienna Meetup 020719Kafka Vienna Meetup 020719
Kafka Vienna Meetup 020719
 
Achieving end-to-end visibility into complex event-sourcing transactions usin...
Achieving end-to-end visibility into complex event-sourcing transactions usin...Achieving end-to-end visibility into complex event-sourcing transactions usin...
Achieving end-to-end visibility into complex event-sourcing transactions usin...
 
How a distributed graph analytics platform uses Apache Kafka for data ingesti...
How a distributed graph analytics platform uses Apache Kafka for data ingesti...How a distributed graph analytics platform uses Apache Kafka for data ingesti...
How a distributed graph analytics platform uses Apache Kafka for data ingesti...
 
Kafka for connected vehicle research | Pavle Bujanovic, Federal Highway Admin...
Kafka for connected vehicle research | Pavle Bujanovic, Federal Highway Admin...Kafka for connected vehicle research | Pavle Bujanovic, Federal Highway Admin...
Kafka for connected vehicle research | Pavle Bujanovic, Federal Highway Admin...
 
Kubernetes 1.22
Kubernetes 1.22Kubernetes 1.22
Kubernetes 1.22
 
Should we manage events like APIs? | Kim Clark, IBM
Should we manage events like APIs? | Kim Clark, IBMShould we manage events like APIs? | Kim Clark, IBM
Should we manage events like APIs? | Kim Clark, IBM
 
Government Track Welcome Address
Government Track Welcome AddressGovernment Track Welcome Address
Government Track Welcome Address
 
Speed-Up Kafka Delivery with AsyncAPI & Microcks | Hugo Guerrero, Red Hat
Speed-Up Kafka Delivery with AsyncAPI & Microcks | Hugo Guerrero, Red HatSpeed-Up Kafka Delivery with AsyncAPI & Microcks | Hugo Guerrero, Red Hat
Speed-Up Kafka Delivery with AsyncAPI & Microcks | Hugo Guerrero, Red Hat
 
Toward Hybrid Cloud Serverless Transparency with Lithops Framework
Toward Hybrid Cloud Serverless Transparency with Lithops FrameworkToward Hybrid Cloud Serverless Transparency with Lithops Framework
Toward Hybrid Cloud Serverless Transparency with Lithops Framework
 
Application Modernization Using Event Streaming Architecture (David Wadden, V...
Application Modernization Using Event Streaming Architecture (David Wadden, V...Application Modernization Using Event Streaming Architecture (David Wadden, V...
Application Modernization Using Event Streaming Architecture (David Wadden, V...
 
Kafka & InfluxDB: BFFs for Enterprise Data Applications | Russ Savage, Influx...
Kafka & InfluxDB: BFFs for Enterprise Data Applications | Russ Savage, Influx...Kafka & InfluxDB: BFFs for Enterprise Data Applications | Russ Savage, Influx...
Kafka & InfluxDB: BFFs for Enterprise Data Applications | Russ Savage, Influx...
 
Mind the App: How to Monitor Your Kafka Streams Applications | Bruno Cadonna,...
Mind the App: How to Monitor Your Kafka Streams Applications | Bruno Cadonna,...Mind the App: How to Monitor Your Kafka Streams Applications | Bruno Cadonna,...
Mind the App: How to Monitor Your Kafka Streams Applications | Bruno Cadonna,...
 
Why Kafka Works the Way It Does (And Not Some Other Way) | Tim Berglund, Conf...
Why Kafka Works the Way It Does (And Not Some Other Way) | Tim Berglund, Conf...Why Kafka Works the Way It Does (And Not Some Other Way) | Tim Berglund, Conf...
Why Kafka Works the Way It Does (And Not Some Other Way) | Tim Berglund, Conf...
 
apidays LIVE New York 2021 - Service reliability through autoscaling workload...
apidays LIVE New York 2021 - Service reliability through autoscaling workload...apidays LIVE New York 2021 - Service reliability through autoscaling workload...
apidays LIVE New York 2021 - Service reliability through autoscaling workload...
 
스타트업을 위한 Confluent 세미나
스타트업을 위한 Confluent 세미나스타트업을 위한 Confluent 세미나
스타트업을 위한 Confluent 세미나
 

Similar to Confluent Private Cloud | Rohit Bakhshi, Staff Product Manager

[Global logic] container runtimes and kubernetes
[Global logic] container runtimes and kubernetes[Global logic] container runtimes and kubernetes
[Global logic] container runtimes and kubernetesGlobalLogic Ukraine
 
GCP Meetup #3 - Approaches to Cloud Native Architectures
GCP Meetup #3 - Approaches to Cloud Native ArchitecturesGCP Meetup #3 - Approaches to Cloud Native Architectures
GCP Meetup #3 - Approaches to Cloud Native Architecturesnine
 
Scale into Multi-Cloud with Containers
Scale into Multi-Cloud with ContainersScale into Multi-Cloud with Containers
Scale into Multi-Cloud with ContainersImesh Gunaratne
 
[DSC Europe 23] Ved Prakash - Intro to GKE setup of airflow with helm and ter...
[DSC Europe 23] Ved Prakash - Intro to GKE setup of airflow with helm and ter...[DSC Europe 23] Ved Prakash - Intro to GKE setup of airflow with helm and ter...
[DSC Europe 23] Ved Prakash - Intro to GKE setup of airflow with helm and ter...DataScienceConferenc1
 
Continuous Lifecycle London 2018 Event Keynote
Continuous Lifecycle London 2018 Event KeynoteContinuous Lifecycle London 2018 Event Keynote
Continuous Lifecycle London 2018 Event KeynoteWeaveworks
 
Openshift serverless Solution
Openshift serverless SolutionOpenshift serverless Solution
Openshift serverless SolutionRyan ZhangCheng
 
A Guide to Adopting Kubernetes
A Guide to Adopting KubernetesA Guide to Adopting Kubernetes
A Guide to Adopting KubernetesNGINX, Inc.
 
CD using ArgoCD(KnolX).pdf
CD using ArgoCD(KnolX).pdfCD using ArgoCD(KnolX).pdf
CD using ArgoCD(KnolX).pdfKnoldus Inc.
 
Kafka as a service in your organsation
Kafka as a service in your organsationKafka as a service in your organsation
Kafka as a service in your organsationSion Smith
 
ArgoCD Meetup PPT final.pdf
ArgoCD Meetup PPT final.pdfArgoCD Meetup PPT final.pdf
ArgoCD Meetup PPT final.pdfamanmakwana3
 
Develop and deploy Kubernetes applications with Docker - IBM Index 2018
Develop and deploy Kubernetes  applications with Docker - IBM Index 2018Develop and deploy Kubernetes  applications with Docker - IBM Index 2018
Develop and deploy Kubernetes applications with Docker - IBM Index 2018Patrick Chanezon
 
Accelerate Application Innovation Journey with Azure Kubernetes Service
Accelerate Application Innovation Journey with Azure Kubernetes Service Accelerate Application Innovation Journey with Azure Kubernetes Service
Accelerate Application Innovation Journey with Azure Kubernetes Service WinWire Technologies Inc
 
GitOps 101 Presentation.pdf
GitOps 101 Presentation.pdfGitOps 101 Presentation.pdf
GitOps 101 Presentation.pdfssuser31375f
 
Hybrid CI/CD with Kubernetes and Codefresh
Hybrid CI/CD with Kubernetes and CodefreshHybrid CI/CD with Kubernetes and Codefresh
Hybrid CI/CD with Kubernetes and CodefreshDevOps.com
 
GitOps for Event-Driven Architecture -- Kube-Style! With Duncan Doyle | Curre...
GitOps for Event-Driven Architecture -- Kube-Style! With Duncan Doyle | Curre...GitOps for Event-Driven Architecture -- Kube-Style! With Duncan Doyle | Curre...
GitOps for Event-Driven Architecture -- Kube-Style! With Duncan Doyle | Curre...HostedbyConfluent
 
Automate NGINX with DevOps Tools
Automate NGINX with DevOps ToolsAutomate NGINX with DevOps Tools
Automate NGINX with DevOps ToolsSupachai Jaturaprom
 
stackconf 2020 | The path to a Serverless-native era with Kubernetes by Paolo...
stackconf 2020 | The path to a Serverless-native era with Kubernetes by Paolo...stackconf 2020 | The path to a Serverless-native era with Kubernetes by Paolo...
stackconf 2020 | The path to a Serverless-native era with Kubernetes by Paolo...NETWAYS
 
DevOps and BigData Analytics
DevOps and BigData Analytics DevOps and BigData Analytics
DevOps and BigData Analytics sbbabu
 
The Best of Both Worlds: Introducing WSO2 API Manager 4.0.0
The Best of Both Worlds: Introducing WSO2 API Manager 4.0.0The Best of Both Worlds: Introducing WSO2 API Manager 4.0.0
The Best of Both Worlds: Introducing WSO2 API Manager 4.0.0WSO2
 

Similar to Confluent Private Cloud | Rohit Bakhshi, Staff Product Manager (20)

[Global logic] container runtimes and kubernetes
[Global logic] container runtimes and kubernetes[Global logic] container runtimes and kubernetes
[Global logic] container runtimes and kubernetes
 
DevOps on GCP Course Compared to AWS
DevOps on GCP Course Compared to AWSDevOps on GCP Course Compared to AWS
DevOps on GCP Course Compared to AWS
 
GCP Meetup #3 - Approaches to Cloud Native Architectures
GCP Meetup #3 - Approaches to Cloud Native ArchitecturesGCP Meetup #3 - Approaches to Cloud Native Architectures
GCP Meetup #3 - Approaches to Cloud Native Architectures
 
Scale into Multi-Cloud with Containers
Scale into Multi-Cloud with ContainersScale into Multi-Cloud with Containers
Scale into Multi-Cloud with Containers
 
[DSC Europe 23] Ved Prakash - Intro to GKE setup of airflow with helm and ter...
[DSC Europe 23] Ved Prakash - Intro to GKE setup of airflow with helm and ter...[DSC Europe 23] Ved Prakash - Intro to GKE setup of airflow with helm and ter...
[DSC Europe 23] Ved Prakash - Intro to GKE setup of airflow with helm and ter...
 
Continuous Lifecycle London 2018 Event Keynote
Continuous Lifecycle London 2018 Event KeynoteContinuous Lifecycle London 2018 Event Keynote
Continuous Lifecycle London 2018 Event Keynote
 
Openshift serverless Solution
Openshift serverless SolutionOpenshift serverless Solution
Openshift serverless Solution
 
A Guide to Adopting Kubernetes
A Guide to Adopting KubernetesA Guide to Adopting Kubernetes
A Guide to Adopting Kubernetes
 
CD using ArgoCD(KnolX).pdf
CD using ArgoCD(KnolX).pdfCD using ArgoCD(KnolX).pdf
CD using ArgoCD(KnolX).pdf
 
Kafka as a service in your organsation
Kafka as a service in your organsationKafka as a service in your organsation
Kafka as a service in your organsation
 
ArgoCD Meetup PPT final.pdf
ArgoCD Meetup PPT final.pdfArgoCD Meetup PPT final.pdf
ArgoCD Meetup PPT final.pdf
 
Develop and deploy Kubernetes applications with Docker - IBM Index 2018
Develop and deploy Kubernetes  applications with Docker - IBM Index 2018Develop and deploy Kubernetes  applications with Docker - IBM Index 2018
Develop and deploy Kubernetes applications with Docker - IBM Index 2018
 
Accelerate Application Innovation Journey with Azure Kubernetes Service
Accelerate Application Innovation Journey with Azure Kubernetes Service Accelerate Application Innovation Journey with Azure Kubernetes Service
Accelerate Application Innovation Journey with Azure Kubernetes Service
 
GitOps 101 Presentation.pdf
GitOps 101 Presentation.pdfGitOps 101 Presentation.pdf
GitOps 101 Presentation.pdf
 
Hybrid CI/CD with Kubernetes and Codefresh
Hybrid CI/CD with Kubernetes and CodefreshHybrid CI/CD with Kubernetes and Codefresh
Hybrid CI/CD with Kubernetes and Codefresh
 
GitOps for Event-Driven Architecture -- Kube-Style! With Duncan Doyle | Curre...
GitOps for Event-Driven Architecture -- Kube-Style! With Duncan Doyle | Curre...GitOps for Event-Driven Architecture -- Kube-Style! With Duncan Doyle | Curre...
GitOps for Event-Driven Architecture -- Kube-Style! With Duncan Doyle | Curre...
 
Automate NGINX with DevOps Tools
Automate NGINX with DevOps ToolsAutomate NGINX with DevOps Tools
Automate NGINX with DevOps Tools
 
stackconf 2020 | The path to a Serverless-native era with Kubernetes by Paolo...
stackconf 2020 | The path to a Serverless-native era with Kubernetes by Paolo...stackconf 2020 | The path to a Serverless-native era with Kubernetes by Paolo...
stackconf 2020 | The path to a Serverless-native era with Kubernetes by Paolo...
 
DevOps and BigData Analytics
DevOps and BigData Analytics DevOps and BigData Analytics
DevOps and BigData Analytics
 
The Best of Both Worlds: Introducing WSO2 API Manager 4.0.0
The Best of Both Worlds: Introducing WSO2 API Manager 4.0.0The Best of Both Worlds: Introducing WSO2 API Manager 4.0.0
The Best of Both Worlds: Introducing WSO2 API Manager 4.0.0
 

More from HostedbyConfluent

Build Real-time Machine Learning Apps on Generative AI with Kafka Streams
Build Real-time Machine Learning Apps on Generative AI with Kafka StreamsBuild Real-time Machine Learning Apps on Generative AI with Kafka Streams
Build Real-time Machine Learning Apps on Generative AI with Kafka StreamsHostedbyConfluent
 
When Only the Last Writer Wins We All Lose: Active-Active Geo-Replication in ...
When Only the Last Writer Wins We All Lose: Active-Active Geo-Replication in ...When Only the Last Writer Wins We All Lose: Active-Active Geo-Replication in ...
When Only the Last Writer Wins We All Lose: Active-Active Geo-Replication in ...HostedbyConfluent
 
Apache Kafka's Next-Gen Rebalance Protocol: Towards More Stable and Scalable ...
Apache Kafka's Next-Gen Rebalance Protocol: Towards More Stable and Scalable ...Apache Kafka's Next-Gen Rebalance Protocol: Towards More Stable and Scalable ...
Apache Kafka's Next-Gen Rebalance Protocol: Towards More Stable and Scalable ...HostedbyConfluent
 
Using Kafka at Scale - A Case Study of Micro Services Data Pipelines at Evern...
Using Kafka at Scale - A Case Study of Micro Services Data Pipelines at Evern...Using Kafka at Scale - A Case Study of Micro Services Data Pipelines at Evern...
Using Kafka at Scale - A Case Study of Micro Services Data Pipelines at Evern...HostedbyConfluent
 
Rule Based Asset Management Workflow Automation at Netflix
Rule Based Asset Management Workflow Automation at NetflixRule Based Asset Management Workflow Automation at Netflix
Rule Based Asset Management Workflow Automation at NetflixHostedbyConfluent
 
Scalable E-Commerce Data Pipelines with Kafka: Real-Time Analytics, Batch, ML...
Scalable E-Commerce Data Pipelines with Kafka: Real-Time Analytics, Batch, ML...Scalable E-Commerce Data Pipelines with Kafka: Real-Time Analytics, Batch, ML...
Scalable E-Commerce Data Pipelines with Kafka: Real-Time Analytics, Batch, ML...HostedbyConfluent
 
Indeed Flex: The Story of a Revolutionary Recruitment Platform
Indeed Flex: The Story of a Revolutionary Recruitment PlatformIndeed Flex: The Story of a Revolutionary Recruitment Platform
Indeed Flex: The Story of a Revolutionary Recruitment PlatformHostedbyConfluent
 
Forecasting Kafka Lag Issues with Machine Learning
Forecasting Kafka Lag Issues with Machine LearningForecasting Kafka Lag Issues with Machine Learning
Forecasting Kafka Lag Issues with Machine LearningHostedbyConfluent
 
Getting Under the Hood of Kafka Streams: Optimizing Storage Engines to Tune U...
Getting Under the Hood of Kafka Streams: Optimizing Storage Engines to Tune U...Getting Under the Hood of Kafka Streams: Optimizing Storage Engines to Tune U...
Getting Under the Hood of Kafka Streams: Optimizing Storage Engines to Tune U...HostedbyConfluent
 
Maximizing Real-Time Data Processing with Apache Kafka and InfluxDB: A Compre...
Maximizing Real-Time Data Processing with Apache Kafka and InfluxDB: A Compre...Maximizing Real-Time Data Processing with Apache Kafka and InfluxDB: A Compre...
Maximizing Real-Time Data Processing with Apache Kafka and InfluxDB: A Compre...HostedbyConfluent
 
Accelerating Path to Production for Generative AI-powered Applications
Accelerating Path to Production for Generative AI-powered ApplicationsAccelerating Path to Production for Generative AI-powered Applications
Accelerating Path to Production for Generative AI-powered ApplicationsHostedbyConfluent
 
Optimize Costs and Scale Your Streaming Applications with Virtually Unlimited...
Optimize Costs and Scale Your Streaming Applications with Virtually Unlimited...Optimize Costs and Scale Your Streaming Applications with Virtually Unlimited...
Optimize Costs and Scale Your Streaming Applications with Virtually Unlimited...HostedbyConfluent
 
Don’t Let Degradation Bring You Down: Automatically Detect & Remediate Degrad...
Don’t Let Degradation Bring You Down: Automatically Detect & Remediate Degrad...Don’t Let Degradation Bring You Down: Automatically Detect & Remediate Degrad...
Don’t Let Degradation Bring You Down: Automatically Detect & Remediate Degrad...HostedbyConfluent
 
Go Big or Go Home: Approaching Kafka Replication at Scale
Go Big or Go Home: Approaching Kafka Replication at ScaleGo Big or Go Home: Approaching Kafka Replication at Scale
Go Big or Go Home: Approaching Kafka Replication at ScaleHostedbyConfluent
 
What's in store? Part Deux; Creating Custom Queries with Kafka Streams IQv2
What's in store? Part Deux; Creating Custom Queries with Kafka Streams IQv2What's in store? Part Deux; Creating Custom Queries with Kafka Streams IQv2
What's in store? Part Deux; Creating Custom Queries with Kafka Streams IQv2HostedbyConfluent
 
A Trifecta of Real-Time Applications: Apache Kafka, Flink, and Druid
A Trifecta of Real-Time Applications: Apache Kafka, Flink, and DruidA Trifecta of Real-Time Applications: Apache Kafka, Flink, and Druid
A Trifecta of Real-Time Applications: Apache Kafka, Flink, and DruidHostedbyConfluent
 
From Raw Data to an Interactive Data App in an Hour: Powered by Snowpark Python
From Raw Data to an Interactive Data App in an Hour: Powered by Snowpark PythonFrom Raw Data to an Interactive Data App in an Hour: Powered by Snowpark Python
From Raw Data to an Interactive Data App in an Hour: Powered by Snowpark PythonHostedbyConfluent
 
Beyond Monoliths: Thrivent’s Lessons in Building a Modern Integration Archite...
Beyond Monoliths: Thrivent’s Lessons in Building a Modern Integration Archite...Beyond Monoliths: Thrivent’s Lessons in Building a Modern Integration Archite...
Beyond Monoliths: Thrivent’s Lessons in Building a Modern Integration Archite...HostedbyConfluent
 
Exactly-Once Semantics Revisited: Distributed Transactions across Flink and K...
Exactly-Once Semantics Revisited: Distributed Transactions across Flink and K...Exactly-Once Semantics Revisited: Distributed Transactions across Flink and K...
Exactly-Once Semantics Revisited: Distributed Transactions across Flink and K...HostedbyConfluent
 

More from HostedbyConfluent (20)

Build Real-time Machine Learning Apps on Generative AI with Kafka Streams
Build Real-time Machine Learning Apps on Generative AI with Kafka StreamsBuild Real-time Machine Learning Apps on Generative AI with Kafka Streams
Build Real-time Machine Learning Apps on Generative AI with Kafka Streams
 
When Only the Last Writer Wins We All Lose: Active-Active Geo-Replication in ...
When Only the Last Writer Wins We All Lose: Active-Active Geo-Replication in ...When Only the Last Writer Wins We All Lose: Active-Active Geo-Replication in ...
When Only the Last Writer Wins We All Lose: Active-Active Geo-Replication in ...
 
Apache Kafka's Next-Gen Rebalance Protocol: Towards More Stable and Scalable ...
Apache Kafka's Next-Gen Rebalance Protocol: Towards More Stable and Scalable ...Apache Kafka's Next-Gen Rebalance Protocol: Towards More Stable and Scalable ...
Apache Kafka's Next-Gen Rebalance Protocol: Towards More Stable and Scalable ...
 
Using Kafka at Scale - A Case Study of Micro Services Data Pipelines at Evern...
Using Kafka at Scale - A Case Study of Micro Services Data Pipelines at Evern...Using Kafka at Scale - A Case Study of Micro Services Data Pipelines at Evern...
Using Kafka at Scale - A Case Study of Micro Services Data Pipelines at Evern...
 
Rule Based Asset Management Workflow Automation at Netflix
Rule Based Asset Management Workflow Automation at NetflixRule Based Asset Management Workflow Automation at Netflix
Rule Based Asset Management Workflow Automation at Netflix
 
Scalable E-Commerce Data Pipelines with Kafka: Real-Time Analytics, Batch, ML...
Scalable E-Commerce Data Pipelines with Kafka: Real-Time Analytics, Batch, ML...Scalable E-Commerce Data Pipelines with Kafka: Real-Time Analytics, Batch, ML...
Scalable E-Commerce Data Pipelines with Kafka: Real-Time Analytics, Batch, ML...
 
Indeed Flex: The Story of a Revolutionary Recruitment Platform
Indeed Flex: The Story of a Revolutionary Recruitment PlatformIndeed Flex: The Story of a Revolutionary Recruitment Platform
Indeed Flex: The Story of a Revolutionary Recruitment Platform
 
Forecasting Kafka Lag Issues with Machine Learning
Forecasting Kafka Lag Issues with Machine LearningForecasting Kafka Lag Issues with Machine Learning
Forecasting Kafka Lag Issues with Machine Learning
 
Getting Under the Hood of Kafka Streams: Optimizing Storage Engines to Tune U...
Getting Under the Hood of Kafka Streams: Optimizing Storage Engines to Tune U...Getting Under the Hood of Kafka Streams: Optimizing Storage Engines to Tune U...
Getting Under the Hood of Kafka Streams: Optimizing Storage Engines to Tune U...
 
Maximizing Real-Time Data Processing with Apache Kafka and InfluxDB: A Compre...
Maximizing Real-Time Data Processing with Apache Kafka and InfluxDB: A Compre...Maximizing Real-Time Data Processing with Apache Kafka and InfluxDB: A Compre...
Maximizing Real-Time Data Processing with Apache Kafka and InfluxDB: A Compre...
 
Accelerating Path to Production for Generative AI-powered Applications
Accelerating Path to Production for Generative AI-powered ApplicationsAccelerating Path to Production for Generative AI-powered Applications
Accelerating Path to Production for Generative AI-powered Applications
 
Optimize Costs and Scale Your Streaming Applications with Virtually Unlimited...
Optimize Costs and Scale Your Streaming Applications with Virtually Unlimited...Optimize Costs and Scale Your Streaming Applications with Virtually Unlimited...
Optimize Costs and Scale Your Streaming Applications with Virtually Unlimited...
 
Don’t Let Degradation Bring You Down: Automatically Detect & Remediate Degrad...
Don’t Let Degradation Bring You Down: Automatically Detect & Remediate Degrad...Don’t Let Degradation Bring You Down: Automatically Detect & Remediate Degrad...
Don’t Let Degradation Bring You Down: Automatically Detect & Remediate Degrad...
 
Streaming is a Detail
Streaming is a DetailStreaming is a Detail
Streaming is a Detail
 
Go Big or Go Home: Approaching Kafka Replication at Scale
Go Big or Go Home: Approaching Kafka Replication at ScaleGo Big or Go Home: Approaching Kafka Replication at Scale
Go Big or Go Home: Approaching Kafka Replication at Scale
 
What's in store? Part Deux; Creating Custom Queries with Kafka Streams IQv2
What's in store? Part Deux; Creating Custom Queries with Kafka Streams IQv2What's in store? Part Deux; Creating Custom Queries with Kafka Streams IQv2
What's in store? Part Deux; Creating Custom Queries with Kafka Streams IQv2
 
A Trifecta of Real-Time Applications: Apache Kafka, Flink, and Druid
A Trifecta of Real-Time Applications: Apache Kafka, Flink, and DruidA Trifecta of Real-Time Applications: Apache Kafka, Flink, and Druid
A Trifecta of Real-Time Applications: Apache Kafka, Flink, and Druid
 
From Raw Data to an Interactive Data App in an Hour: Powered by Snowpark Python
From Raw Data to an Interactive Data App in an Hour: Powered by Snowpark PythonFrom Raw Data to an Interactive Data App in an Hour: Powered by Snowpark Python
From Raw Data to an Interactive Data App in an Hour: Powered by Snowpark Python
 
Beyond Monoliths: Thrivent’s Lessons in Building a Modern Integration Archite...
Beyond Monoliths: Thrivent’s Lessons in Building a Modern Integration Archite...Beyond Monoliths: Thrivent’s Lessons in Building a Modern Integration Archite...
Beyond Monoliths: Thrivent’s Lessons in Building a Modern Integration Archite...
 
Exactly-Once Semantics Revisited: Distributed Transactions across Flink and K...
Exactly-Once Semantics Revisited: Distributed Transactions across Flink and K...Exactly-Once Semantics Revisited: Distributed Transactions across Flink and K...
Exactly-Once Semantics Revisited: Distributed Transactions across Flink and K...
 

Recently uploaded

Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...
Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...
Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...Will Schroeder
 
Building AI-Driven Apps Using Semantic Kernel.pptx
Building AI-Driven Apps Using Semantic Kernel.pptxBuilding AI-Driven Apps Using Semantic Kernel.pptx
Building AI-Driven Apps Using Semantic Kernel.pptxUdaiappa Ramachandran
 
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPA
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPAAnypoint Code Builder , Google Pub sub connector and MuleSoft RPA
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPAshyamraj55
 
Videogame localization & technology_ how to enhance the power of translation.pdf
Videogame localization & technology_ how to enhance the power of translation.pdfVideogame localization & technology_ how to enhance the power of translation.pdf
Videogame localization & technology_ how to enhance the power of translation.pdfinfogdgmi
 
How Accurate are Carbon Emissions Projections?
How Accurate are Carbon Emissions Projections?How Accurate are Carbon Emissions Projections?
How Accurate are Carbon Emissions Projections?IES VE
 
20230202 - Introduction to tis-py
20230202 - Introduction to tis-py20230202 - Introduction to tis-py
20230202 - Introduction to tis-pyJamie (Taka) Wang
 
Empowering Africa's Next Generation: The AI Leadership Blueprint
Empowering Africa's Next Generation: The AI Leadership BlueprintEmpowering Africa's Next Generation: The AI Leadership Blueprint
Empowering Africa's Next Generation: The AI Leadership BlueprintMahmoud Rabie
 
The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...
The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...
The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...Aggregage
 
OpenShift Commons Paris - Choose Your Own Observability Adventure
OpenShift Commons Paris - Choose Your Own Observability AdventureOpenShift Commons Paris - Choose Your Own Observability Adventure
OpenShift Commons Paris - Choose Your Own Observability AdventureEric D. Schabell
 
Igniting Next Level Productivity with AI-Infused Data Integration Workflows
Igniting Next Level Productivity with AI-Infused Data Integration WorkflowsIgniting Next Level Productivity with AI-Infused Data Integration Workflows
Igniting Next Level Productivity with AI-Infused Data Integration WorkflowsSafe Software
 
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...DianaGray10
 
Basic Building Blocks of Internet of Things.
Basic Building Blocks of Internet of Things.Basic Building Blocks of Internet of Things.
Basic Building Blocks of Internet of Things.YounusS2
 
UiPath Platform: The Backend Engine Powering Your Automation - Session 1
UiPath Platform: The Backend Engine Powering Your Automation - Session 1UiPath Platform: The Backend Engine Powering Your Automation - Session 1
UiPath Platform: The Backend Engine Powering Your Automation - Session 1DianaGray10
 
Nanopower In Semiconductor Industry.pdf
Nanopower  In Semiconductor Industry.pdfNanopower  In Semiconductor Industry.pdf
Nanopower In Semiconductor Industry.pdfPedro Manuel
 
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdf
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdfUiPath Solutions Management Preview - Northern CA Chapter - March 22.pdf
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdfDianaGray10
 
VoIP Service and Marketing using Odoo and Asterisk PBX
VoIP Service and Marketing using Odoo and Asterisk PBXVoIP Service and Marketing using Odoo and Asterisk PBX
VoIP Service and Marketing using Odoo and Asterisk PBXTarek Kalaji
 
IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdf
IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdfIaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdf
IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdfDaniel Santiago Silva Capera
 
UiPath Community: AI for UiPath Automation Developers
UiPath Community: AI for UiPath Automation DevelopersUiPath Community: AI for UiPath Automation Developers
UiPath Community: AI for UiPath Automation DevelopersUiPathCommunity
 

Recently uploaded (20)

Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...
Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...
Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...
 
Building AI-Driven Apps Using Semantic Kernel.pptx
Building AI-Driven Apps Using Semantic Kernel.pptxBuilding AI-Driven Apps Using Semantic Kernel.pptx
Building AI-Driven Apps Using Semantic Kernel.pptx
 
20150722 - AGV
20150722 - AGV20150722 - AGV
20150722 - AGV
 
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPA
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPAAnypoint Code Builder , Google Pub sub connector and MuleSoft RPA
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPA
 
Videogame localization & technology_ how to enhance the power of translation.pdf
Videogame localization & technology_ how to enhance the power of translation.pdfVideogame localization & technology_ how to enhance the power of translation.pdf
Videogame localization & technology_ how to enhance the power of translation.pdf
 
How Accurate are Carbon Emissions Projections?
How Accurate are Carbon Emissions Projections?How Accurate are Carbon Emissions Projections?
How Accurate are Carbon Emissions Projections?
 
20230202 - Introduction to tis-py
20230202 - Introduction to tis-py20230202 - Introduction to tis-py
20230202 - Introduction to tis-py
 
Empowering Africa's Next Generation: The AI Leadership Blueprint
Empowering Africa's Next Generation: The AI Leadership BlueprintEmpowering Africa's Next Generation: The AI Leadership Blueprint
Empowering Africa's Next Generation: The AI Leadership Blueprint
 
The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...
The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...
The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...
 
OpenShift Commons Paris - Choose Your Own Observability Adventure
OpenShift Commons Paris - Choose Your Own Observability AdventureOpenShift Commons Paris - Choose Your Own Observability Adventure
OpenShift Commons Paris - Choose Your Own Observability Adventure
 
Igniting Next Level Productivity with AI-Infused Data Integration Workflows
Igniting Next Level Productivity with AI-Infused Data Integration WorkflowsIgniting Next Level Productivity with AI-Infused Data Integration Workflows
Igniting Next Level Productivity with AI-Infused Data Integration Workflows
 
201610817 - edge part1
201610817 - edge part1201610817 - edge part1
201610817 - edge part1
 
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...
 
Basic Building Blocks of Internet of Things.
Basic Building Blocks of Internet of Things.Basic Building Blocks of Internet of Things.
Basic Building Blocks of Internet of Things.
 
UiPath Platform: The Backend Engine Powering Your Automation - Session 1
UiPath Platform: The Backend Engine Powering Your Automation - Session 1UiPath Platform: The Backend Engine Powering Your Automation - Session 1
UiPath Platform: The Backend Engine Powering Your Automation - Session 1
 
Nanopower In Semiconductor Industry.pdf
Nanopower  In Semiconductor Industry.pdfNanopower  In Semiconductor Industry.pdf
Nanopower In Semiconductor Industry.pdf
 
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdf
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdfUiPath Solutions Management Preview - Northern CA Chapter - March 22.pdf
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdf
 
VoIP Service and Marketing using Odoo and Asterisk PBX
VoIP Service and Marketing using Odoo and Asterisk PBXVoIP Service and Marketing using Odoo and Asterisk PBX
VoIP Service and Marketing using Odoo and Asterisk PBX
 
IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdf
IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdfIaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdf
IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdf
 
UiPath Community: AI for UiPath Automation Developers
UiPath Community: AI for UiPath Automation DevelopersUiPath Community: AI for UiPath Automation Developers
UiPath Community: AI for UiPath Automation Developers
 

Confluent Private Cloud | Rohit Bakhshi, Staff Product Manager

  • 1. GitOps with Confluent and Kubernetes Rohit Bakhshi, Confluent Kafka Summit 2021
  • 2. What is GitOps? Continuous Deployment for cloud native applications ● Declarative Specs (Infrastructure as Code) in Git ● Automation to deploy and maintain to match the declarative spec
  • 3. An API-driven Control Plane to deploy and manage Confluent ● Declarative API (CRDs) for Confluent Platform Services ● Declarative API (CRDs) for topics and RBAC policies Integrates with Cloud-Native ecosystem for Security, Reliability, DevOps Automation ● Complete automated secure setup ● Secrets Management with Kubernetes Secrets and Hashicorp Vault ● One command scale up ● Fault tolerance to zone and node failure Introducing Confluent for Kubernetes
  • 4. An API-driven Control Plane to deploy and manage Confluent ● Declarative API (CRDs) for Confluent Platform Services ● Declarative API (CRDs) for topics and RBAC policies Integrates with Cloud-Native ecosystem for Security, Reliability, DevOps Automation ● Complete automated secure setup ● Secrets Management with Kubernetes Secrets and Hashicorp Vault ● One command scale up ● Fault tolerance to zone and node failure Introducing Confluent Operator 2.0
  • 5. Confluent + Kubernetes => GitOps 5 Confluent Operator Component (Custom Resource) Credentials kafka.yaml --- kafka: spec: listeners: internal: authentication: type: plain jaasConfig: secretRef: credential Helm K8s Secrets K8s API
  • 6. Continuous Deployment on Kubernetes Flux CD Operator Git Repo Kafka CR Zookeeper CR ControlCenter CR Confluent Operator KafkaTopic CRs /cluster/prod /cluster/qa
  • 7. LDAP Demo QA - SASL/Plain authentication - TLS encryption with automatically generated certs Prod - SASL/Plain authentication - TLS encryption with custom certs - Granular Role Based Access Control Control Center ZooKeeper x 3 Confluent Operator Kubernetes Kafka Broker x 3 Schema Registry Connect KsqlDB
  • 8. Try it and Learn More! Try Confluent for Kubernetes cnfl.io/download Confluent Blog cnfl.io/blog Community cnfl.io/meetups