Diese Präsentation wurde erfolgreich gemeldet.
Die SlideShare-Präsentation wird heruntergeladen. ×

OSDC 2019 | Automated patch management with Ansible and Rundeck by Andreas Lehr and Rico Spießberger

Anzeige
Anzeige
Anzeige
Anzeige
Anzeige
Anzeige
Anzeige
Anzeige
Anzeige
Anzeige
Anzeige
Anzeige
Wird geladen in …3
×

Hier ansehen

1 von 25 Anzeige

OSDC 2019 | Automated patch management with Ansible and Rundeck by Andreas Lehr and Rico Spießberger

Herunterladen, um offline zu lesen

In our On Premise hosting environment we still run a lot of applications on traditional stacks without using containers. In order to run them in a secured way we created a mature patch automation. Thanks to ansible, rundeck, icinga and a bunch of other opensource tools we are able to update and reboot most of our systems without our customers noticing. We do that throughout the day on a regular base using rundeck, or even on short notice if another “heartbleed” occurs.

In our On Premise hosting environment we still run a lot of applications on traditional stacks without using containers. In order to run them in a secured way we created a mature patch automation. Thanks to ansible, rundeck, icinga and a bunch of other opensource tools we are able to update and reboot most of our systems without our customers noticing. We do that throughout the day on a regular base using rundeck, or even on short notice if another “heartbleed” occurs.

Anzeige
Anzeige

Weitere Verwandte Inhalte

Ähnlich wie OSDC 2019 | Automated patch management with Ansible and Rundeck by Andreas Lehr and Rico Spießberger (20)

Aktuellste (20)

Anzeige

OSDC 2019 | Automated patch management with Ansible and Rundeck by Andreas Lehr and Rico Spießberger

  1. 1. AUTOMATED PATCH MANAGEMENT WITH ANSIBLE AND RUNDECK Schwarz IT KG - @crsp & @shakalandy
  2. 2. ABOUT US 2 Andreas Lehr @shakalandy Rico Spiesberger @crsp “Hosting and Domain Services” department - lidl.de - lidl-reisen.de/.at/.ch/... - lidl-shop.nl/.be/.cz/.pl/... - mobile app backend (30 countries)
  3. 3. ABOUT SCHWARZ IT ➔ Central IT of the Schwarz Group (Lidl, Kaufland, PreZero, GreenCycle,...) ➔ ~ 3000 employees ➔ HQ in Weinsberg/Heilbronn - Location in Berlin ➔ We have Jobs - https://jobs.schwarz 3
  4. 4. WHAT’S WRONG HERE? 4 WTF?
  5. 5. AUTOMATED PATCHING 5 WHY WE’VE DONE IT HOW WE’VE DONE IT LIVE-DEMO (sort of)
  6. 6. WHY AUTOMATED PATCHING? 6 manual patching takes too much valuable time
  7. 7. WHY AUTOMATED PATCHING? 7 Make security and auditors happy
  8. 8. WHY AUTOMATED PATCHING? 8 Have a mature and reliable process
  9. 9. HOW WE’VE DONE IT c 9 Ansible and Rundeck • 1 week cycle for DEV/TEST/QA • 4 week cycle for PROD • Emergency stuff can be patched without prior information • Target: no manual process, but fully automated
  10. 10. PATCHING WORKFLOW 10 Set Monitoring Downtime
  11. 11. PATCHING WORKFLOW 11 Create VMWare Snapshot
  12. 12. PATCHING WORKFLOW 12 Send Notifications
  13. 13. PATCHING WORKFLOW 13 Host Preparation
  14. 14. PATCHING WORKFLOW 14 finally: upgrade time
  15. 15. PATCHING WORKFLOW 15 reboot if needs-restarting
  16. 16. PATCHING WORKFLOW c 16 ● remove old kernels ● patching date > /etc/last_patching (Monitoring, motd, ansible CMDB) ● activate Loadbalancer health checks ● clean up (yum clean up, etc) ● update “patchlist” documentation (For auditors and POs) ● remove downtime ● remove snapshot (3 days later) after reboot tasks
  17. 17. IMPEDIMENTS AND RECOMMENDATIONS ON AUTOMATED PATCHING 17 have fixed timeslots
  18. 18. IMPEDIMENTS AND RECOMMENDATIONS ON AUTOMATED PATCHING 18 Delete Snapshots automatically
  19. 19. IMPEDIMENTS AND RECOMMENDATIONS ON AUTOMATED PATCHING 19 Rebooting HW servers takes some time…..
  20. 20. IMPEDIMENTS AND RECOMMENDATIONS ON AUTOMATED PATCHING 20 preload packages
  21. 21. IMPEDIMENTS AND RECOMMENDATIONS ON AUTOMATED PATCHING 21 have enough space in /var/yum and /tmp
  22. 22. IMPEDIMENTS AND RECOMMENDATIONS ON AUTOMATED PATCHING 22 parallel patching: ansible forks=20+ and strategy: free
  23. 23. LIVE-DEMO!?!?
  24. 24. QUESTIONS?
  25. 25. Thanks. Don’t forget - https://jobs.schwarz 25

×