Diese Präsentation wurde erfolgreich gemeldet.
Wir verwenden Ihre LinkedIn Profilangaben und Informationen zu Ihren Aktivitäten, um Anzeigen zu personalisieren und Ihnen relevantere Inhalte anzuzeigen. Sie können Ihre Anzeigeneinstellungen jederzeit ändern.

Cloud Native Apps with GitOps

988 Aufrufe

Veröffentlicht am

More and more businesses are requiring developers to own end to end delivery, including operational ownership. Weaveworks will share with you what GitOps means, and how easy it is to create cloud native applications, CICD pipelines, integrate operations and more, using GitOps.

Inherited from best practices going back 10-15 years, cloud native is making these practices more relevant today. At Weaveworks, they implement these principles in their product, Weave Cloud. This not only helps customers ship apps faster, it also helps them run their own cloud native stack. This presentation will show how Weaveworks does this, identify best practices and tools, and showcase some of Weaveworks’ use cases.

For the video of this presentation at Cloud Native London visit: https://skillsmatter.com/skillscasts/10506-keynote-by-alexis-richardson

To learn more about Weaveworks: www.weave.works

Veröffentlicht in: Technologie

Cloud Native Apps with GitOps

  1. 1. More, Better, Faster! Cloud Native Apps with GitOps Alexis Richardson CEO, Weaveworks & TOC Chair, CNCF September 27, 2017
  2. 2. WTF I want to show you how Weaveworks learned to apply technology to make our business go much faster, using cloud native and continuous delivery
  3. 3. WTF GitOps – empowering developers to do operations
  4. 4. How did we get here
  5. 5. Remember this?
  6. 6. 1994
  7. 7. 1995
  8. 8. 1996
  9. 9. 2003
  10. 10. 2004
  11. 11. 2005
  12. 12. (1999-) 2008
  13. 13. Netflix wanted to improve streaming service for all, fast Must Read!https://www.slideshare.net/AmazonWebServices/dmg206
  14. 14. 2009-2010 – cloud native appears as a set of requirements https://www.slideshare.net/AmazonWebServices/dmg206 Must Read!
  15. 15. Investors pile in because: velocity equals big bucks
  16. 16. We are all developers now ● If you can merge a pull request on GitHub then you are a developer ● The industry has learnt how to connect GitHub to running applications via CICD pipelines, enabling Git to drive Ops… GitOps
  17. 17. Frontend (Nginx) The Interwebs RDS Instance Frontend (Nginx) Frontend (Nginx) Authenticatin g Proxy WeaverangCollection Service AWS DynamoDB Query Service WeaverangControl Service AWS SQS WeaverangPipe Service AWS S3 Consul Cluster NATS Cluster Static Content Users Service Core Services Scope Services AWS DynamoDB AWS S3 WeaverangDistribution Service WeaverangIngester Service Consul Cluster Cortex ServicesMonitoring & Management Services Grafana Prometheus Weave Scope WeaverangNode Exporter WeaverangScope Probes Kubediff One time, when we blew up Weave Cloud…
  18. 18. Before After
  19. 19. 45 minutes from complete and total wipeout back to happy
  20. 20. “I want to go faster” Then Adopt continuous delivery & align business and tech to work as one team Tech teams are empowered to act quickly upon business needs, so they must be multi-skilled and “own the system, own the changes”.  more operations roles in “dev” teams And Use the right Cloud Native technologies Automation means getting fewer errors and scaling safely  Git, CICD, containers, orchestration GitOps = Cloud Native + Continuous Delivery
  21. 21. Continuous Delivery
  22. 22. Fundamental Theorem of DevOps What can be described, can be automated and accelerated
  23. 23. But most businesses release infrequently
  24. 24. DevOps while ubiquitously known has not yet delivered magic velocity & continuous everything
  25. 25. 1. New cloud Native apps and tools  is forcing change 2. Accelerate all the things 3. Automation phase shift – adapting to many releases per day Summer is coming
  26. 26. Cloud Native
  27. 27. “Curate and promote tools for cloud native” Microservices Containerised apps Dynamically orchestrated
  28. 28. Industry has bought in
  29. 29. Kubernetes is a platform for cloud native apps “orchestration”  Means that it runs containerised apps the way Linux runs processes Powerful but ”low level” – will have many simplifying tools in future Important: It is declarative automated infrastructure Kubernetes
  30. 30. We can store Kubernetes config in Git and validate it
  31. 31. The key to velocity
  32. 32. Weave Kubernetes gRPC Docker Fluentd OpenTracing Prometheus *** CUNNING SUBLIMINAL ADVERT ***
  33. 33. GitOps
  34. 34. • We use declarative infrastructure ie. Kubernetes, Docker, Terraform, & more • Our entire system including code, config, monitoring rules, dashboards, is described in GitHub with full audit trail • We can roll our major or minor changes as pull requests, and automatically check for diffs if system diverges from the desired “source of truth” in Git How did Weaveworks rebuild our systems in 45 mins?
  35. 35. • Config is code • Code must be version controlled • Config must be version controlled too GitOps follows the Logic of DevOps
  36. 36. GitOps follows the Logic of DevOps • Config is code • Code must be version controlled • Config must be version controlled too • What can be described can be automated • Describe everything: code, config, monitoring & policy; and then keep it in version control
  37. 37. GitOps • Git as a source of truth for desired state of whole system • Compare desired with actual state to fire diff alerts • Make ops changes by pull request
  38. 38. What this gets us • Any developer can use GitHub • Anyone can join team and ship a new app or make changes easily • All changes can be triggered, stored, audited and validated in Git And we didn’t have to do anything very new or clever 
  39. 39. The future is joined up • DevOps is evolving to accommodate the potential of cloud native tools to get more joined up CICD and release automation at a much higher quality • GitOps shows us how to join up workflows and action oriented dashboards in ways that make sense for developers doing more ops
  40. 40. GitOps Deep Dive
  41. 41. GitOps journey • Day 0 – push first app on first cluster & validate that it works • Day 1 – add CICD updates & rollbacks via Git PRs • Day 2 – observing and controlling a production system • Day 3 – scale up – eg. better service routing (mesh) & security policy
  42. 42. The three pillars of GitOps CompletePipeline Observability Security
  43. 43. Pipelines – ABCDE pattern Deployment App Dev Build (CI) Containers Any Cluster Any Cloud Execution Push app to cloud
  44. 44. Pipelines – ABCDE pattern Deployment App Dev Build (CI) Containers Any Cluster Any Cloud Execution Push app to cloud
  45. 45. GitOps - do CD right • Config is code & everything is config (‘declarative infra’) • Code (& config!) must be version controlled • CD tools that do not record changes in version control are harmful
  46. 46. Continuous Delivery/Deployment The GitOps Pipeline – automate releases, sync with Git Image Repo OrchestratorDeploy Synchronizer Config change Manual deployment Git Code change Git Update Hint Continuous Integration Deploy Automator CI Pipeline
  47. 47. Takeaways • Pushing apps & changes is the fundamental operation • GitOps needs complete pipelines that join up CI, CD and Release Automation in one flow • The right tools must be used – they coordinate between Git, CI, and the services running in the cluster, enabling sophisticated deployment policies
  48. 48. Observability
  49. 49. GitOps & Observability • If a change is released and no- one is around to see it, then did it really work?
  50. 50. Read the whole thing – https://twitter.com/mipsytipsy/status/911711540008628224
  51. 51. Observability – understanding whole system wellness • In GitOps we want to get developers comfortable with operational concepts like monitoring, tracing, and incident handling • Like doctors, we must be able to validate health as well as diagnose problems, using a common language and a coherent set of tools
  52. 52. Observability – understanding whole system wellness monitor log trace visualize troubleshoot debugdiffs alerts validate Incident management & response
  53. 53. BigCos pioneer integrated solutions to spur faster action
  54. 54. Observability Asking hard questions of the system is not the same as setting up a load of alerts
  55. 55. Diffs & auto sync are really great Bake in metrics end to end and full stack from the start For alerts, use RED metrics  focus on services You can’t avoid some instrumentation – but that’s ok since all in Git Visuals in Git – grafanalib Policy & Rules in Git (traffic, incident management) Automate (autogenerate) per-service screens & keep in Git Some lessons we learnt running Weave Cloud
  56. 56. Digression
  57. 57. • Observability is a way to verify that our system is in the desired state as specified in Git eg. diffs & alerts & more • An observable system is one that can be controlled, via a feedback control loop that drives continuous improvement A bit of theory
  58. 58. The GitOps Pipeline is really driving a CONTROL LOOP… GitOps loop Deployment App Dev Build (CI) Containers Any Cluster Any Cloud Execution Release ObserveOperate
  59. 59. Runtime System Monitoring Tracing Logging… Control System Git Diff People Release Observe Operate
  60. 60. System Actual State Observable State Understanding Desired State (in Git) Observe Orient Decide Act Release Diff ROODA Loop
  61. 61. Fundamental Theorem of GitOps What can be described and observed can be automated and controlled and accelerated
  62. 62. Takeaways ● Observability is fundamental to automation and understanding ● It is holistic and encompasses any question you could ask about the difference between desired and observed state ● You must bake it in from start, using monitoring, tracing, diff tools …
  63. 63. Security (& Policy)
  64. 64. Recap… • Day 0 – push first app on first cluster & validate that it works • Day 1 – add CICD updates & rollbacks via Git PRs • Day 2 – observing and controlling a production system • Day 3 – scale up – eg. better service routing (mesh) & security policy
  65. 65. Who sees what Who talks to whom  Matters more as you scale Based on rules Routing, Firewalls, ACLs, Rollouts  Declarative? Store them in Git Security
  66. 66. ● By using diffs, we can immediately and automatically enforce convergence to a correct (desired) system state ● SOX: Git repos control which developers touch the system, which via GitOps CICD tooling can be mapped directly into running clusters ● Secrets? Security: some examples we have seen @ Weaveworks
  67. 67. Parting Thoughts
  68. 68. ● A much easier way to deliver and manage better apps, faster ● Works anywhere! ● Much more resilient – 45 mins to recover from total system wipeout What we got
  69. 69. ● Git is a source of truth *for everything* in cloud native era ● GitOps ROODA loop improves velocity & collaboration ● Focus on the 3 pillars: pipelines, observability & security central This is leading us to new insights, new tools, new dashboards today Key takeaways
  70. 70. Observe Orient Decide Act Release GitOps is social Understand Team Control System
  71. 71. Why GitOps The need for speed! Business expects tech to be super responsive  consolidation of dev & ops skills in the most agile teams Automation: a phase shift is coming If we want to go from 1 release per MONTH to 1 or more release per DAY then we need to automate the complete lifecycle New app types will accelerate change DevOps and cloud adoption have arrived. New application types are emerging. Many use tools like Kubernetes & Docker which support “everything as code” and practices that deliver a complete automated & accelerated lifecycle
  72. 72. Where ops happens Powered by Kubernetes