Diese Präsentation wurde erfolgreich gemeldet.
Wir verwenden Ihre LinkedIn Profilangaben und Informationen zu Ihren Aktivitäten, um Anzeigen zu personalisieren und Ihnen relevantere Inhalte anzuzeigen. Sie können Ihre Anzeigeneinstellungen jederzeit ändern.
Federation of Kubernetes
Clusters ("Übernetes")
Kubecon 2015
Quinton Hoole <quinton@google.com>
Staff Software Engineer - ...
Google has beeeg data centers...
... but you know that already.
Images by Connie Zhou
But we also have rather a lot of them...
Treating these differently can have benefits...
UI
CLI
API
Control Plane Servers
Kubernetes
Users
containers
containers
containers
containers
containers
containers
contai...
UI
All you really care about?
API Containers
UI
CLI
API
Control Plane Clusters
Übernetes
API
Users
Kubernetes on
Kubernetes on
Kubernetes on
Premise
Federation
Why is this interesting?
Reason 1: High Availability
• Cloud providers have outages, yes, but...
• Has one of your application software
upgrades ev...
Reason 2: Application Migration
• Migrating applications between clusters
is tedious and error-prone if done
manually
• Mu...
Reason 3: Policy Enforcement
• Some data must be stored and
processed within specified political
jurisdictions, by law.
• ...
Reason 4: Vendor Lock-in Avoidance
• Make it easy to migrate applications
between cloud providers.
• Run the same app on m...
Reason 5: Capacity Overflow
• Make intelligent placement decisions
• Utilization
• Cost
• Performance Ubernetes
User
On Pr...
"OK, I'm sold. Where's the catch?"
Provider 1
Zone A
Zone B
Federation comes with some challenges...
Provider 2
Zone C
Provider 1
Zone D
● Different bandwidt...
Cross-cluster load balancing
• Geographically aware DNS gets clients to
the "closest" healthy cluster.
• Standard Kubernet...
Cross-cluster service discovery
• DNS + Kubernetes cluster-local service
discovery.
• Can default to cluster-local with fa...
Location affinity
• Strictly coupled pods/applications
• High bandwidth requirements
• Low latency requirements
• High fid...
Cross-cluster monitoring and auditing...
• "Cluster per tab" might suffice for small
numbers of clusters
• Some monitoring...
Cluster Federation - The Implementation...
API Compatible with Kubernetes
• Less new stuff to learn
• Can learn incrementally, as you
need new functionality.
• Analo...
State and control resides in
underlying clusters
(for the most part)
• Better scalability
• Kubernetes scales with
number ...
• Drive current state -> desired state
• But per-cluster state, not per node,
per pod etc.
• Observed state is the truth
R...
Modularity
Loose coupling is a goal everywhere
• simpler
• composable
• extensible
Code-level plugins where possible
Multi...
Federation status & plans
Federation Lite (single cluster, multiple zones)
• In alpha Q4 2015
• Productionized ~Q1 2016
Fe...
I want more!
• Requirements doc - comments welcome
• tinyurl.com/ubernetesv2
• Special interest group
• groups.google.com/...
Nächste SlideShare
Wird geladen in …5
×

Federation of Kubernetes Clusters (a.k.a. "Ubernetes") - KubeCon 2015 slides - Quinton Hoole

2.759 Aufrufe

Veröffentlicht am

Slides from the keynote presentation that I gave on day 1 of KubeCon 2015 (http://kubecon.io).

Veröffentlicht in: Software
  • Loggen Sie sich ein, um Kommentare anzuzeigen.

Federation of Kubernetes Clusters (a.k.a. "Ubernetes") - KubeCon 2015 slides - Quinton Hoole

  1. 1. Federation of Kubernetes Clusters ("Übernetes") Kubecon 2015 Quinton Hoole <quinton@google.com> Staff Software Engineer - Google quinton_hoole@github
  2. 2. Google has beeeg data centers... ... but you know that already. Images by Connie Zhou
  3. 3. But we also have rather a lot of them...
  4. 4. Treating these differently can have benefits...
  5. 5. UI CLI API Control Plane Servers Kubernetes Users containers containers containers containers containers containers containers containers containers containers containers containers containers containers containers Cluster / Data Center / Availability Zone
  6. 6. UI All you really care about? API Containers
  7. 7. UI CLI API Control Plane Clusters Übernetes API Users Kubernetes on Kubernetes on Kubernetes on Premise Federation
  8. 8. Why is this interesting?
  9. 9. Reason 1: High Availability • Cloud providers have outages, yes, but... • Has one of your application software upgrades ever gone terribly wrong? • How about infrastructure upgrades (auth systems? quota? data store?) • How about a fat-fingered config change? • There are several interesting variants: • Multiple availability zones? • Multiple cloud providers? Cross-cluster Load Balancer Your paying customer Cluster 1 Cluster 2 Cluster 3
  10. 10. Reason 2: Application Migration • Migrating applications between clusters is tedious and error-prone if done manually • Much like software upgrades, you *can* script them, but (K)ubernetes just does it quicker/safer/better. • Now with rollback too! • On-premise ↔ Cloud • Amazon ↔ Google :-) • ... Ubernetes UI On-Premise Cluster In-Cloud Cluster Migrate: On Premise→Cloud Different Cloud Provider
  11. 11. Reason 3: Policy Enforcement • Some data must be stored and processed within specified political jurisdictions, by law. • Some software/data must be on premise and air-gapped, by company policy. • Some business units get to use the expensive gear, some don't. • Auditing is also a big deal, so funnelling all operations through a central control point makes this easier. Ubernetes UI U.S. Cloud Cluster E.U Cloud Cluster On-premise Cluster
  12. 12. Reason 4: Vendor Lock-in Avoidance • Make it easy to migrate applications between cloud providers. • Run the same app on multiple cloud providers and choose the best one for your: • workload characteristics • budget • performance requirements • availability requirements Ubernetes UI Kubernetes on GCE Kubernetes on AWS Kubernetes On-Premise
  13. 13. Reason 5: Capacity Overflow • Make intelligent placement decisions • Utilization • Cost • Performance Ubernetes User On Premise Cluster Other Cloud Provider Preferred Cloud Provider Run my stuff
  14. 14. "OK, I'm sold. Where's the catch?"
  15. 15. Provider 1 Zone A Zone B Federation comes with some challenges... Provider 2 Zone C Provider 1 Zone D ● Different bandwidth charges/latency/through- put/reliability ● Different service discovery (but DNS!) ● Consolidated monitoring & alerting
  16. 16. Cross-cluster load balancing • Geographically aware DNS gets clients to the "closest" healthy cluster. • Standard Kubernetes service load balancing within each cluster. • New L7 LB's available soon. • Can be extended to divert traffic away from "healthy-but-saturated" clusters.
  17. 17. Cross-cluster service discovery • DNS + Kubernetes cluster-local service discovery. • Can default to cluster-local with failover to remote clusters.
  18. 18. Location affinity • Strictly coupled pods/applications • High bandwidth requirements • Low latency requirements • High fidelity requirements • Cannot easily span clusters • Loosely coupled • Opposite of above • Relatively easily distributed across clusters • Preferentially coupled • Strongly coupled but can be migrated piecemeal.
  19. 19. Cross-cluster monitoring and auditing... • "Cluster per tab" might suffice for small numbers of clusters • Some monitoring solutions provide stronger integration and global summarization
  20. 20. Cluster Federation - The Implementation...
  21. 21. API Compatible with Kubernetes • Less new stuff to learn • Can learn incrementally, as you need new functionality. • Analogous argument applies to existing automation systems (PAAS etc). • These can be ported to Ubernetes relatively easily. • All Kubernetes entities are "federatable". Ubernetes or Kubernetes Client Applications Applications Applications Run my stuff
  22. 22. State and control resides in underlying clusters (for the most part) • Better scalability • Kubernetes scales with number of nodes per cluster (<10,000) • Ubernetes scales with number of clusters (~100) • Beter fault isolation • Kubernetes clusters fail independently of Ubernetes Kubernetes Cluster Kubernetes Cluster Ubernetes API APIRepl. Ctrl etc State API APIRepl. Ctrl etc State API APIRepl. Ctrl etc State
  23. 23. • Drive current state -> desired state • But per-cluster state, not per node, per pod etc. • Observed state is the truth Recurring pattern in the system Examples: • ReplicationController • Service observe diff act Similar Control loops to Kubernetes
  24. 24. Modularity Loose coupling is a goal everywhere • simpler • composable • extensible Code-level plugins where possible Multi-process where possible Isolate risk by interchangeable parts Examples: • MigrationController • Scheduler
  25. 25. Federation status & plans Federation Lite (single cluster, multiple zones) • In alpha Q4 2015 • Productionized ~Q1 2016 Federation Proper (multiple clusters, federated) • Alpha Q1 2016 Google Container Engine (GKE) • hosted Federation too • GKE Federation Lite ~Q1-Q2 2016 PaaSes and Distros • RedHat OpenShift, CoreOS Tectonic, RedHat Atomic... • ... watch this space...
  26. 26. I want more! • Requirements doc - comments welcome • tinyurl.com/ubernetesv2 • Special interest group • groups.google.com/forum/kubernetes-sig-federation • quinton@google.com • quinton_hoole@github Kubernetes Cluster Kubernetes Cluster Ubernetes API APIRepl. Ctrl etc State API APIRepl. Ctrl etc State API APIRepl. Ctrl etc State

×