In this session we are going to Introduce Cluster API, a Kubernetes subproject that allows you to manage Kubernetes clusters lifecycle running anywhere using only Kubernetes YAML files. Let’s see how Azure Arc GitOps approach improves and simplify the day-2 operations of these clusters, where your Git repo is now the source of truth. Do you have problems managing identities and Network connection for your current CI/CD process? You don’t know how to manage multiple Kubernetes clusters in production? Then this talk is for you!
WordPress Websites for Engineers: Elevate Your Brand
Manage your kubernetes cluster with cluster api, azure and git ops
1.
2.
3. Cloud Computing
Science - From
Abstraction to Invention
changing the way we
communicate, learn,
solve problems and do
business
youtube.com/AzureTar AzureTar.com @AzureTar
4.
5. Overview
A retailer with 100s of stores would like to move all in-
store applications to containers running on a K8s
clusters
They are faced with the challenge of how to uniformly
deploy, configure and manage their containerized
applications across multiple locations
Business requirements
• Bootstrap a new store to fully run with the applications
and configuration that this store requires
• Enable IT to apply and monitor at scale governance
across all stores
• Monitor the state of applications and configuration in
all stores
• Integrate DevOps and Safe Deployment Practices for
applications running in stores
6.
7. Azure Arc
Azure Arc-enabled infrastructure
Connect and operate hybrid resources
as native Azure resources
Azure Arc-enabled services
Deploy and run Azure services outside of
Azure while still operating it from Azure
Multi-cloud Datacenter Edge
11. PUBLIC PREVIEW
Provide Azure services and users secure access to Arc-enabled Kubernetes clusters
AAD RBAC Cluster Connect Custom Locations
12. Key benefits from Azure Arc
• Asset organization and inventory with a unified
view in the Azure Portal across all locations
• GitOps-based model for deploying configuration
as code to one or many clusters
• Application deployment and update at scale
• Source control based Safe Deployment
Procedures when rolling new applications and
configurations
• Developer tooling agnostic—use the tools they
want
Azure Management
(Azure Resource Manager, Azure Policy,
Azure Portal, API, CLI…)
13. Git as the source of
truth for a system
Git as the single place where
we operate
(create, change, and delete)
All changes are
observable
https://www.weave.works/technologies/gitops/
System state described
declaratively
State declaration versioned in
source control
Approved changes are
applied automatically
Agents enforce desired
state
14. Arc Connected
Kubernetes Cluster
GitOps
Configurations
git
Repository
Flux Operator +
Helm Operator
Application
Changes
git
merge
Flux
pickup changes
Application V1
(Desired State)
Google Kubernetes
Engine (GKE)
Elastic Kubernetes
Service (EKS)
Rancher K3s
Azure Kubernetes
Service on HCI
1 2 3
4
Application
Deployment
5
6
7
Application V2
(New Desired State)
Application
Rolling Update 8
Any Kubernetes,
any Infrastructure
15.
16.
17.
18. AKS CAPI Control
plane
(capi-controlplane)
AKS fully
managed
GitOps
config
GitOps
config
GitOps
config
Self-managed
Self-managed
CAPIZ
Legend:
CAPIZ – Azure CAPI Provider
CAPI – Cluster API
Flux - fluxcd.io
git
Repository
Flux
Application
Changes
git
merge
Flux
Flux
19.
20.
21. azuretar/clusterapi-gitops: This repo stores configuration to Kubernetes clusters management (github.com)
Concepts - The Cluster API Book (k8s.io)
Cluster API Azure Provider | Azure Arc Jumpstart
Quick Start - The Cluster API Book (k8s.io)
Guide To GitOps (weave.works)
kubernetes-sigs/cluster-api (crds.dev)
kubernetes-sigs/cluster-api-provider-azure@v0.4.13 (crds.dev)
kubernetes-sigs/image-builder: Cross provider Kubernetes image building utility. (github.com)
Azure/azure-capi-cli-extension: Kubernetes Cluster API support in the Azure CLI (github.com)
Tutorial: Deploy configurations using GitOps on an Azure Arc enabled Kubernetes cluster - Azure Arc | Microsoft Docs
Azure RBAC for Azure Arc-enabled Kubernetes clusters - Azure Arc | Microsoft Docs
Use Cluster Connect to connect to Azure Arc-enabled Kubernetes clusters - Azure Arc | Microsoft Docs
Monitor Azure Arc enabled Kubernetes clusters - Azure Monitor | Microsoft Docs
Built-in policy definitions for Azure Kubernetes Service - Azure Kubernetes Service | Microsoft Docs
Built-in policy definitions for Azure Arc-enabled Kubernetes - Azure Arc | Microsoft Docs
Azure/arc-k8s-demo: Artifacts for Arc For Kubernetes Demo (github.com)
Azure Arc-enabled Kubernetes - YouTube
23. • Provide a “zero to hero” scenarios for multiple environments and
deployment type using as much automation as possible.
• Create a ”supermarket” experience by being able to take “off the
shelf” scenarios and implement it.
• Meeting Azure Arc customers and partners where they are.
• Agile, “startup-like” team.
• No detail is too small.
• Ready to go technical demos
• Jumpstart ArcBox is a sandbox environment that allows users to
explore all the major capabilities of Azure Arc in a click of a
button.
• Jumpstart Lighting is a show where people come to share their
Azure Arc/Jumpstart/Hybrid experience.
24. aka.ms/arc-introvideo
Introducing Azure Arc
aka.ms/arc-compete
Azure Arc compete deck
aka.ms/azurearcpricing
Azure Arc pricing page
aka.ms/arc-techcommunity
Deep dives on Azure Arc, best practices and more
aka.ms/arc-customerstories
Learn how customers are implementing Azure Arc
https://aka.ms/arc-feedback
Public Q&A forum
aka.ms/AzureArcJumpstart
Azure Arc Jumpstart
aka.ms/AzureArcJumpstartDemos
Azure Arc Jumpstart demos
aka.ms/arc-blog
Azure Arc: Extending Azure management to any
infrastructure
aka.ms/arc-k8svideo
Kubernetes—Managing K8 clusters outside of
Azure with Azure Arc
aka.ms/arc-serversvideo
Server management—Organize all your servers
outside of Azure with Azure Arc
aka.ms/arc-serversdocs
Documentation for Azure Arc
enabled servers
aka.ms/arc-k8sdocs
Documentation for Azure Arc
enabled Kubernetes
aka.ms/arc-datablog
Run Azure data services on-premises, at
the edge, and multi-cloud with Azure Arc
aka.ms/arc-data-mechanicsvideo
Azure Arc-enabled data services demos
including SQL and PostgreSQL Hyperscale
aka.ms/arc-ignite-video
Ignite 2021: Innovate across hybrid and
multicloud with Azure Arc
aka.ms/arc-datadocs
Documentation for Azure Arc-enabled
data services
Azure Arc complete overview Azure Arc-enabled
Kubernetes & servers
Azure Arc-enabled
data services
Hinweis der Redaktion
So now let’s talk about build cloud native apps anywhere, at scale, another core use case enabled by Azure Arc.
We are working with a retailer who have 100s of stores and have in-store applications. They’d like to move all these applications to containers running on Kubernetes clusters but they are not sure how they can uniformly deploy and configure these applications across multiple locations. They need to be able to onboard a new store that can run with the specific applications that this store needs, apply governance, monitor these clusters as well as integrate DevOps practices. Many customers need this today.
So what exactly is Azure Arc and how does it work?
Azure Arc is a set of technologies that unlocks new hybrid scenarios for customers by extending Azure services and management to any infrastructure so that customers can build, operate, and manage all of their resources for traditional, cloud-native and distributed edge applications in a consistent way across the entire IT estate.
This means that you can now manage and operate all of your existing and new IT resources consistently and at-scale, wherever they reside, from Azure.
To unpack this a bit more, Azure Arc-enabled infrastructure enables you to connect your resources, which live outside of Azure today, and operate them as if they where native Azure resources, using the same management tools and services that Azure provides.
And with Azure Arc-enabled services, you have the flexibility to deploy fully managed Azure services anywhere – on-premises or in other public clouds so you can take advantage of cloud benefits everywhere, such as scalability, fast deployment, and always up-to-date cloud innovation. What’s awesome is that you can initiate and manage these deployments, right from the Azure Portal.
One of the goals of Azure Arc is to meet you where you are with your existing investments. As a developer, you get tremendous flexibility and convenience with Azure Arc enabled infrastructure and Arc enabled services. It doesn’t matter where your applications are running, they could be on servers, VMs or on Kubernetes. You can manage across all of these with Azure Arc.
If you have already container-based applications, you could easily deploy, secure and monitor all your deployments with Azure Arc enabled Kubernetes.
OR you could use Azure platform services like Azure App Services, Functions or data services like SQL Managed Instance for a more managed experience and deploy them in the cloud and in hybrid and multi cloud environment.
The important thing to note is that the way you deploy does not change. You could continue using your existing tools and practices and benefit from a seamless, consistent experience.
So, just want to summarize Azure Arc-enabled Kubernetes for you.
Again, similar to Arc-enabled servers, we offer a lot of flexibility to you based on your specific needs. We support a wide range of Kubernetes distributions with flavors from different vendors – as you can see on the slide. You can connect all these clusters to Azure and start deploying applications to these clusters using a GitOps-based model.
Additionally, you can enable cluster health monitoring with Azure Monitor for Containers. Another powerful capability is the integration with Azure Policy that can ensure compliance with the organization’s security baselines.
With the new Cluster Extensions feature, you get a modern management experience on your Arc-enabled Kubernetes clusters. Users can now deploy and configure services like Azure Monitor and Azure Defender via the Azure Portal, CLI and APIs. Previously, these add-ons could be only be deployed manually via Helm Charts.
Azure Monitor Container Insights
The first experience we are enabling is Azure Monitor Container Insights. Monitoring your containers is critical, especially when you're running a production cluster, at scale, with multiple applications. Azure Monitor for Containers has been available for AKS, ARO as well as self managed clusters hosted using AKS-Engine but we can now extend this easily to any Kubernetes cluster, even one running on AWS or GKE!
Container insights delivers a comprehensive monitoring experience across the full stack with workload monitoring encompassing collection of metrics and logs that are sent to Log Analytics resource in the customer’s tenant and subscription. You can get rich live telemetry on cluster health, node/pod status and container performance and correlate these metrics/logs across the App & Infra layers for full stack diagnostics. Container Insights also offers rich integration with the Open Source Ecosystem with support for metrics from Prometheus, Grafana and OpenTelemetry.
Azure Defender
Azure Defender can now be easily extended to clusters that live outside of Azure through the Azure Defender extension for Arc-enabled Kubernetes clusters. This can be easily enabled through the Azure Portal or CLI and supports multiple Kubernetes distributions across on-premises and multi-cloud. You can get a single pane of glass view in Azure to easily monitor the security posture of all your Kubernetes clusters, no matter where they are deployed and detect threats across these clusters using advanced analytics.
Once deployed, the extension collected Kubernetes data and sends it to the Azure Defender backend in the cloud for further analysis. Azure Defender continuously analyzes the Kubernetes cluster for potential threats based on collected data and reports threats and malicious activity detected as Alerts in Azure Security Center.
More new extensions for Azure Policy (Gatekeeper) and Open Service Mesh are coming soon.
Azure Arc-enabled data services will also be deployable as an extension.
AAD RBAC: The Kubernetes native way of defining authorization checks involves creation of ClusterRoleBindings and RoleBinding objects in the cluster. The AAD RBAC feature instead allows for usage of Azure role assignments as the single source of truth for all authorization checks happening on the cluster. Any requests sent to the API server of the cluster are checked with the Azure authorization service to see if the entity making the request (user or service principal) is allowed (or not allowed) to access the resource of concern. This feature allows for a single place of audit on all the role assignments made on any resource within any of the Arc-enabled Kubernetes clusters.
Note: This feature is only applicable for those self-managed Kubernetes clusters where the apiserver of the cluster is accessible by the customer. As a result, this feature is not applicable for cloud provider managed K8s clusters like GKE and EKS. On AKS, this feature is available natively and Arc onboarding of the cluster is not required for the same.
Cluster Connect: Cluster Connect feature of Azure Arc-enabled Kubernetes provides connectivity to the apiserver of the cluster without requiring any additional inbound communication to be enabled. This is achieved by mapping a Hybrid Connections resource on the Azure service side to every Arc-enabled Kubernetes cluster where a reverse proxy agent is able to securely initiate a session with hybrid connection in an outbound manner. This feature allows your developers to access the clusters from anywhere for interactive development and debugging. If you already have a lot of investments in terms of paid pipeline concurrency for Azure Pipelines or GitHub Actions or any other hosted CI/CD provider, you can now reuse the same to deploy against even on-prem clusters without requiring self hosted agents (VMs) on-prem.
Custom Locations: In Azure, every resource is created in a specific location such as eastus or westeurope. This location maps to an Azure region. Custom location allows for extension of this concept beyond the boundaries of Azure to allow customers to define their own Kubernetes clusters (on-prem or hybrid) as targets for running Azure PaaS services. This allows for consistent developer experience across Azure and off-Azure environments.
This is where Azure Arc-enabled Kubernetes comes into play. You can project the Kubernetes clusters to Azure, so you can organize and view all your clusters in Azure (similar to Azure Arc-enabled servers) but you can also configure them uniformly, called zero touch configuration. This practice is called GitOps, which is a Kubernetes operating model. In GitOps, the configurations are declared and stored in a Git repo and our Arc agents running on the cluster continuously monitor this repo for updates or changes and automatically pulls down these changes to the cluster. This in turn enables safe deployment practices as the agents ensure that the cluster conforms to the ‘desired state’ as declared by the organization. Any deviation from this desired state will result in an automated rollback.
Azure Arc-enabled Kubernetes adopts a GitOps methodology, so customers define their applications and cluster configuration in source control. This means changes to apps and configuration are versioned, enforced, and logged across any number of clusters.
Let’s explore some of the principles of GitOps :
Single source of truth – Git
All actions taken by developers/admins – create, change and delete happens in Git
All changes are visible to project teams and can be tracked
Declarative Configuration: All resources managed through a GitOps process must be expressed declaratively.
Version controlled, immutable storage: The declarative descriptions are stored in a repository that supports immutability, versioning and version history. For example, git.
Automated delivery: Delivery of the declarative descriptions, from the repository to runtime environment, is fully automated.
Software Agents: Reconcilers deploy and maintain the resources described in the declarative configuration. Actions are performed on divergence between the version controlled declarative configuration and the actual state of the target system.
So what does the GitOps flow look like in the context of Arc-enabled Kubernetes?
We are starting with Kubernetes clusters outside of Azure
Azure Arc Kubernetes connected cluster is created
User creates cluster’s GitOps configurations
Flux operator gets deployed on the cluster, and starts ”listening” to the git repository with the user’s application code
Flux operator initiates user’s application deployment on the cluster, representing the current desired state
User is updating the application (creating a new app version) and merge changes to the repository
Flux pickup a change to the git repository
Flux operator initiates a new user’s application version deployment on the cluster while removing old version application pods, resulting in a new Desired State
Azure Arc-enabled Kubernetes adopts a GitOps methodology, so customers define their applications and cluster configuration in source control. This means changes to apps and configuration are versioned, enforced, and logged across any number of clusters.
Let’s explore some of the principles of GitOps :
Single source of truth – Git
All actions taken by developers/admins – create, change and delete happens in Git
All changes are visible to project teams and can be tracked
Declarative Configuration: All resources managed through a GitOps process must be expressed declaratively.
Version controlled, immutable storage: The declarative descriptions are stored in a repository that supports immutability, versioning and version history. For example, git.
Automated delivery: Delivery of the declarative descriptions, from the repository to runtime environment, is fully automated.
Software Agents: Reconcilers deploy and maintain the resources described in the declarative configuration. Actions are performed on divergence between the version controlled declarative configuration and the actual state of the target system.