Diese Präsentation wurde erfolgreich gemeldet.
Wir verwenden Ihre LinkedIn Profilangaben und Informationen zu Ihren Aktivitäten, um Anzeigen zu personalisieren und Ihnen relevantere Inhalte anzuzeigen. Sie können Ihre Anzeigeneinstellungen jederzeit ändern.

GitOps - Modern best practices for high velocity app dev using cloud native tools

8.558 Aufrufe

Veröffentlicht am

Alexis Richardson, Weaveworks CEO, recently presented this slide deck at the KubeCon + CloudNativeCon event. He covers GitOps - modern best practices for developing apps faster using cloud native tools.

Veröffentlicht in: Technologie

GitOps - Modern best practices for high velocity app dev using cloud native tools

  1. 1. GitOps Modern best practices for high velocity app dev using cloud native tools Alexis Richardson & William Denniss CEO, Weaveworks PM, Google Kubernetes Engine TOC Chair, CNCF K8s Conformance WG Lead @monadic @williamdenniss December 2017
  2. 2. Meet Qordoba ● SF based team use machine learning to create ”local” marketing UX for big brands ● Rapid iteration while obeying SOC2 compliance ● Google Cloud – Kubernetes Engine & Container Builder ● Weave Cloud – single cont. delivery & observability pipeline
  3. 3. Over 30 releases per day, up from 1-2 per week 1) Estimated time needed to fix prod software bugs ~60% less time 2) Estimated time to respond to customer requests ~43% less time 3) Uptime 99% → 100% Impact
  4. 4. GitOps builds on DevOps with Git as a single source of truth for the whole system. Over the few years at Weaveworks we learned that success came down to getting 3 things right: 1. Have a complete automated pipeline 2. Operating a fast paced business 24/7 requires monitoring and observability baked into the beginning. Security is of critical importance 3. Everything has to be version controlled and stored in a single source of truth from which you can recover Introducing GitOps
  5. 5. Image credit: Helen Beal, Ranger4 Build on a decade of DevOps best practices
  6. 6. Add declarative infrastructure
  7. 7. • We use declarative infrastructure ie. Kubernetes, Docker, Terraform, & more • Our entire system including code, config, monitoring rules, dashboards, is described in GitHub with full audit trail • We can roll our major or minor changes as pull requests, and automatically check for diffs if system diverges from the desired “source of truth” in Git How did Weaveworks rebuild our systems in 45 mins?
  8. 8. • Config is code • Code must be version controlled • Config must be version controlled too GitOps follows the Logic of DevOps
  9. 9. GitOps follows the Logic of DevOps • Config is code • Code must be version controlled • Config must be version controlled too • What can be described can be automated • Describe everything: code, config, monitoring & policy; and then keep it in version control
  10. 10. GitOps • Git as a source of truth for desired state of whole system • Compare desired with actual state to fire diff alerts • Make ops changes by pull request
  11. 11. What this gets us • Any developer can use GitHub • Anyone can join team and ship a new app or make changes easily • All changes can be triggered, stored, audited and validated in Git And we didn’t have to do anything very new or clever ☺
  12. 12. Three pillars of GitOps Pipelines Observability Control
  13. 13. Three pillars of GitOps Pipelines Observability Control Joined Up Pipelines Continuous Integration Continuous Deployment Release Automation >>> Git as source of truth for desired system state
  14. 14. First you must join up these components into a pipeline Deployment App Dev Build (CI) Containers Execution (CD + Release Automation)
  15. 15. Service deployments are controlled using operator pattern Git Config Kubernetes Cluster Deployment Service Deploy Operator
  16. 16. Service deployments are controlled using operator pattern ○ Config is code & everything is config (‘declarative infra’) ○ Code (& config!) must be version controlled ○ Anything that does not record changes in version control is harmful
  17. 17. Extracting Declarative Config From Running Clusters • What if you’re already running a cluster setup and don’t have the resource configuration available? • `kubectl get <object> -o yaml --export` can be used to extract the existing configuration and bootstrap your configuration repository • Be sure to manually review the output, “--export” intentionally drops information that it sees as transient, but it’s not perfect.
  18. 18. ❯ GitOps operational mindset, all k8s applications stored in Git. ❯ Securely automate & share secrets publicly ❯ Asymmetric (public key) cryptography ❯ Encrypt data up to (and inside) K8s cluster Bitnami: Encrypt Kubernetes SecretsSealed Secrets
  19. 19. Structure of a GitOps repository ● 1 repository per application/service ● Use a separate branch per environment (maps to a Kubernetes namespace, or cluster) ● Push changes such as the image name, health checks, etc to staging (or feature) branches first. ● Rolling out to production involves a merge. (use `git merge -s ours branchname` to skip a set of staging-only changes). ● Use protected branches to enforce code review requirements.
  20. 20. Example GitOps Pipeline Git Code Git Config Container Registry Build Container (CI) Update image in staging config 1/ Code change 2/ Merge Staging to Prod Config Updater Kubernetes Cluster Deployment Service Deploy Operator
  21. 21. Three pillars of GitOps Pipelines Observability Control Observability Monitoring Logging Tracing & Visualization >>> Holistic view of the real system state ‘right now’
  22. 22. Implication: each service PR has to be observed & measured
  23. 23. Service level view, from UX to health to deep diagnostics • End user happiness is all • Integrate GitOps CD pipeline with tools to observe results of PRs • Developers have to correlate UX to operational concepts like monitoring, tracing, logs • Like doctors, we must be able to validate health as well as diagnose problems
  24. 24. In GitOps, every service should have a unified interactive dash (eg. at Lyft, metrics + events + actions)
  25. 25. Three pillars of GitOps Pipelines Observability Control Control 1) Everything via Git (updates, policy, security) 2) Orchestration 3) Diff & Sync >>> Automation is Convergence
  26. 26. Every service has two sources of truth Desired State In Git Observed State In Real System Same?
  27. 27. Control means CONVERGENCE ● Orchestration: Kubernetes & CD operators (Weave Flux) ● When orchestration fails use Diff & Sync tools ● When sync fails make changes via Git not kubectl
  28. 28. Examples
  29. 29. Kubernetes Monitoring Tracing Logging… Control & Operations Git Diff GitOps Service Lifecycle
  30. 30. Kubernetes Monitoring Tracing Logging… Control & Operations Git Diff GitOps Service Lifecycle ”release driven” Observe Orient Decide Act Release
  31. 31. ONLY what can be described and observed can be automated and controlled and accelerated Fundamental Theorem of GitOps
  32. 32. Use declarative configuration to define your application and services. All changes need to go through your git review process – noone should be using kubectl directly Use an operator in the cluster to drive the observed cluster state to the desired state, as declared by your configuration in git. The 3 core principles of GitOps
  33. 33. GitOps is for Developers ● If you can merge a pull request on GitHub then you are an app developer ● If you use Cloud Native tools you can describe the whole stack & app in Git ● GitOps is the logical conclusion for operations Image: GitHub
  34. 34. What’s new vs DevOps and “infra as code”? How is this different from PaaS? Does GitOps work with … ● ChatOps? ● Secrets? ● Helm (non-compositional templates) ● Istio (How to do Canary rollouts and “partial” updates?) What else can be declarative? Policy & Security & Dashboards & … Any Questions?

×