Diese Präsentation wurde erfolgreich gemeldet.
Wir verwenden Ihre LinkedIn Profilangaben und Informationen zu Ihren Aktivitäten, um Anzeigen zu personalisieren und Ihnen relevantere Inhalte anzuzeigen. Sie können Ihre Anzeigeneinstellungen jederzeit ändern.

Automated Server Administration for DevSecOps

54 Aufrufe

Veröffentlicht am

Talk given at 3rd DevSecOps Forum Mar 12th in Berne, Switzerland

Veröffentlicht in: Software
  • Als Erste(r) kommentieren

  • Gehören Sie zu den Ersten, denen das gefällt!

Automated Server Administration for DevSecOps

  1. 1. VSHN - The DevOps Company Automated Server Administration for DevSecOps Aarno Aukia, CTO @ VSHN - The DevOps Company DevSecOps Forum 12.3.2019
  2. 2. VSHN - The DevOps Company ● Who is Aarno and VSHN - The DevOps Company ? ● Managing 1200 servers ○ Challenges ○ Our implementation & user interface ○ Demo ○ Lessons learned 22 Agenda
  3. 3. VSHN - The DevOps Company @aarnoaukia http://about.me/aarno aarno.aukia@vshn.ch ETH → Google → Atrila → VSHN VSHN - The DevOps Company Since 2014, currently 35 VSHNeers in Zürich, Switzerland Helping Developers run applications on any infrastructure making both visitors happy with stability and developers happy with agility 33 About Aarno & VSHN.ch
  4. 4. VSHN - The DevOps Company 5 Operations = Firefighting-as-a-Service ? 5
  5. 5. VSHN - The DevOps Company Capability Maturity Model Integration (CMMI) 66 Operations 2014 How to get to this level?
  6. 6. VSHN - The DevOps Company DevOps: CMMI Level 5: People, Processes & Tools 77
  7. 7. VSHN - The DevOps Company ● Interface Dev <-> Ops ○ DevOps, CI/CD, Containers, etc ● Interface Ops <-> Infra ○ Hypervisors, APIs, Metrics, etc ● Manage VMs on any infrastructure ● Service Monitoring, Alerting, Metrics, Logs ● Updates, Backups, Configuration Management 88 VSHN Managed Service
  8. 8. VSHN - The DevOps Company ● you don’t like proper (software) release management ○ or else you’ll be in dependency hell with 200 modules in 200 environments ● you practice “don’t touch a running system” ○ you’ll be updating all servers all the time - both good and bad ● standardizing/optimizing ○ no more handcrafted snowflake pets, lots and lots of cattle ● you don’t actively develop monitoring/checks ○ or else you’ll get alert fatigue from noisy checks 99 Don’t do automatic server management if...
  9. 9. VSHN - The DevOps Company ● Web-based self-service for customers ● CRUD server definitions -> CMDB 1010 VSHN Control Panel
  10. 10. VSHN - The DevOps Company 11 Demo 11
  11. 11. VSHN - The DevOps Company 1212 Infrastructure as Code Single source of truth: Configuration YAML in GIT --- classes: - profile_certificates - profile_webhosting packages: - nagios-plugins-wordpress - nagios-plugins-wordpress-config profile_php::php_settings: date.timezone: 'Europe/Zurich' acmetool::use_production_endpoint: true profile_php::php_version: '7.0' profile_php::extensions: - mysql - gd - mbstring - curl profile_icinga2::hiera_services: check_wordpress_dev: check_command: 'wordpress_updates' display_name: 'Wordpress Updates test.vshn.net' vars: wordpress_instance: 'vshn_dev' wordpress_db_name: 'vshn_dev' wordpress_db_user: 'vshn_dev' wordpress_db_pass: 'replaced' check_wordpress_prod: check_command: 'wordpress_updates' display_name: 'Wordpress Updates vshn.net' vars: wordpress_instance: 'vshn_prod' wordpress_db_name: 'vshn_prod' wordpress_db_user: 'vshn_prod' wordpress_db_pass: 'replaced' profile_mariadb::db_root_pw: 'replaced' profile_webhosting::sites: vshn_dev: stack_type: 'php' db_type: 'mariadb' db_password: 'replaced' php_admin_values: memory_limit: '300M' post_max_size: '20M' upload_max_filesize: '20M' manage_tls: letsencrypt
  12. 12. VSHN - The DevOps Company ● who ● when ● what ● tests successful? ● why 1313 GIT: changelog
  13. 13. VSHN - The DevOps Company 1414 Logic # https check if $manage_tls and $manage_tls != 'false' { ::profile_icinga2::resources::check { "${name}_nginx-${main_domain}-https": check_command => 'http', display_name => "nginx https ${name}", check_zone => $monitoring_check_zone, notes => 'https check on port 443, supports authentication and URI', notes_url => 'http', production_level => $monitoring_production_level, vars => merge({ 'http_address' => $::fqdn, 'http_vhost' => $main_domain, 'http_ssl' => true, 'http_uri' => $monitor_path, 'http_sni' => true, }, $_auth_var, $monitor_extra_check_vars), } } # https certificate check if ($manage_tls and $manage_tls != 'false' and $manage_tls != 'snakeoil') { if $manage_tls == 'trusted' { $_days = 30 } else { $_days = 20 } ::profile_icinga2::resources::check { "${name}_nginx-${main_domain}-certificate": check_command => 'http', display_name => "nginx certificate ${main_domain}", notes => "https certificate check, checks days (${_days}) left until expire", notes_url => 'https_certificate', production_level => $monitoring_production_level, check_interval => '2h', vars => { 'http_address' => $::fqdn, 'http_vhost' => $main_domain, 'http_tls' => true, 'http_certificate' => $_days, 'http_sni' => true, }, } } } e.g. automatically monitor all HTTPS sites & certificates
  14. 14. VSHN - The DevOps Company ● Trade-off between multi-tenancy (separation of customers/environments) and common code base development -> Tools & Software-Release-Processes (e.g. https://github.com/vshn/crmngr) ● Monitoring & Backup by default: value > cost ● Hierarchical CMDB: configuration for policy groups ● Staged/canary rollouts (e.g. for intrusive things like firewall changes) ● Local tests (pacco: puppet-in-docker-in-vagrant: https://github.com/vshn/puppet-in-docker) ● Good and robust process for regular & frequent updates 1515 Lessons learned
  15. 15. Come visit us for a coffee! VSHN AG - Neugasse 10 - CH-8005 Zürich - +41 44 545 53 00 - https://vshn.ch/ - info@vshn.ch https://vshn.ch/kontakt/ Follow us on Twitter! @vshn_ch 16

×