The Kubernetes Cluster API (CAPI) sub-project aims to simplify and unify the Kubernetes cluster lifecycle management. With almost a dozen provider implementations, we decided to add our own provider implementation for Apache CloudStack. But how simple was it? With several CRDs, controllers, API versions, and even experimental features, and e2e tests to develop, it’s easy to get lost in the details and can seem quite overwhelming. Join us to discover how we did it! We detail our journey, from building an initial prototype over a weekend to a full-fledged, production-ready application, and deployed in an environment with over 10s of 1000s of hosts! We share the pitfalls, hacks, learnings, and key takeaways from our journey to develop CAPC – the CAPI Provider for Apache CloudStack.
David is a CloudStack committer, member of the Kubernetes Org and Software Engineer at ShapeBlue.
-----------------------------------------
CloudStack Collaboration Conference 2022 took place on 14th-16th November in Sofia, Bulgaria and virtually. The day saw a hybrid get-together of the global CloudStack community hosting 370 attendees. The event hosted 43 sessions from leading CloudStack experts, users and skilful engineers from the open-source world, which included: technical talks, user stories, new features and integrations presentations and more.
5. What is Cluster API Provider [CAPI] ?
Plethora of
Providers
Tailored Services
Differing
Implementations
Standardize
Default
Implementations
Unified Cluster
Management
Individual Cluster
Management
Custom Distros
Commonality
Problem
Solution
6. What is Cluster API Provider [CAPI] ?
How it solves the problem :
Defines Common Operations
Create, Scale, Upgrade, Delete
Default Implementation
Declarative
clusterctl
7. How does CAPI work ?
CAPI
Provider
Cluster
Definition Kubernetes
Cluster
11. Concepts:
How does CAPI work ?
Management Cluster
● Manages Lifecycle of
Workload Clusters
● CAPI
● Providers
● Resources, State
● Runs Anywhere
Workload Cluster
● Managed by MC
● Runs on your infra
● End user
Infrastructure
Provider
● Interface to the
underlying infra
● Pod on MC
Custom Resource
Definitions
● Extension of K8s API
● Define CAPI & infra
specific components
● Cluster, Machine
Eg: CloudStack Cluster,
AWS Machine
12. How does CAPI work ?
clusterctl generate
clusterctl describe
Management Cluster
Cluster
Config
Workload Clusters
clusterctl init
13. How does CAPI work ?
Cluster
Definition
CAPI
ACS
Provider
Management
Cluster
14. How does CAPI work ?
Cluster
Definition
CAPI
ACS
Provider
Management
Cluster
Cluster Resources
● Network
● Public IP
● Load Balancer
15. How does CAPI work ?
Cluster
Definition
CAPI
ACS
Provider
Management
Cluster
Cluster Resources
● Network
● Public IP
● Load Balancer
Cluster Nodes
● Deploy VMs
● Bootstrap : cloud-init
16. How does CAPI work ?
Cluster
Definition
CAPI
ACS
Provider
Management
Cluster
Cluster Resources
● Network
● Public IP
● Load Balancer
Cluster Nodes
● Deploy VMs
● Bootstrap : cloud-init
Cleanup Resources
17. CAPC Features
Cross Platform
Unified Cluster
Management
Customizable Independent Plugin
Specialized Features :
● Host Affinity
● Cross Zone
● Additional Disk
● Custom Health Check
● VM Details
19. How we did it ?
The Code :
20XX
Specialized Features
Multi Zone - Shared
Network
20XX
Advanced Features
Affinity - VM Details -
Custom Health Checks
20XX
Core Functionality
Create - Delete
20XX
Basic Functionality
Scale - Upgrade
20XX
Integrations
EKS - A
20. How we did it ?
The Fun Part !
Research Meetings Support Propose
Bonus :
Great People !
21. How we did it ?
The Fun Part !
Cluster
API
Image Builder
Contributions
Sponsor
Membership
22. How we did it ?
Repo Donation
Container Registry
Github Integrations
Documentation
Home Stretch :
23. How we did it ?
v0.4.8 Release
~80% Test Coverage
EKS-A Integrated
How its going :
Image Builder
- EL 7 / 8
- Ubuntu 20 / 22
- Suse
24. Learnings
Be Curious
But Satisfaction brought it back
Ask
Everyone loves to be a hero
Do your research
Avoid Surprises
Community > Code
Win - win - Win
26. The Future
And Many More …
Watch this space for more !
Additional OS Support
CloudStack aware of CAPC
Clusters
Unmanaged Clusters
CloudStack aware of CAPC Clusters
27. We Want You !
https://github.com/kubernetes-sigs/cluster-api-provider-cloudstack
https://kubernetes.slack.com/messages/cluster-api-cloudstack
https://cluster-api-cloudstack.sigs.k8s.io/
https://www.youtube.com/watch?v=Du1pGriJzo8