Diese Präsentation wurde erfolgreich gemeldet.
Wir verwenden Ihre LinkedIn Profilangaben und Informationen zu Ihren Aktivitäten, um Anzeigen zu personalisieren und Ihnen relevantere Inhalte anzuzeigen. Sie können Ihre Anzeigeneinstellungen jederzeit ändern.
SecDevOps
securing DevOps
Area41
10.6.2016 Zürich
Aarno Aukia
VSHN AG - The DevOps Company
10.6.2016 VSHN AG | http://vshn.ch 2
About me
●
MSc Computer Science ETH
●
Security Operations @ Google
●
Co-Founder & CTO...
10.6.2016 VSHN AG | http://vshn.ch 3
Agenda
●
DevOps ?
●
Where is the security ?
●
Customer example 1
●
Customer example 2...
10.6.2016 VSHN AG | http://vshn.ch 4
DevOps?
●
Collaboration: Development (Dev) and Operations (Ops)
●
Bring agile softwar...
10.6.2016 VSHN AG | http://vshn.ch 5
Dev + Ops collaboration
●
Bring together Developers & Operations
●
Practice agile Ope...
10.6.2016 VSHN AG | http://vshn.ch 6
Infrastructure as code
●
Change from hand-groomed servers to Operations
Engineering (...
10.6.2016 VSHN AG | http://vshn.ch 7
Infrastructure tools
●
Packaging code & dependencies for atomic
deployment/rollback
–...
10.6.2016 VSHN AG | http://vshn.ch 8
Infrastructure testing
●
Bring software engineering best practice to operations
●
Lar...
10.6.2016 VSHN AG | http://vshn.ch 9
Infrastructure feedback
●
Collect all logs in ELK (Elasticsearch, Logstash & Kibana)
...
10.6.2016 VSHN AG | http://vshn.ch 10
Software Delivery Automation
10.6.2016 VSHN AG | http://vshn.ch 11
Where is the security ?
10.6.2016 VSHN AG | http://vshn.ch 12
Developers
●
Duh!
●
Education, education, education
●
Concept/architecture/code audi...
10.6.2016 VSHN AG | http://vshn.ch 13
Configuration management
●
Declare target state
●
Enforce state every x minutes, e.g...
10.6.2016 VSHN AG | http://vshn.ch 14
Logging
●
Audit logging (who changed what when)
●
Application/request log
●
As WORM ...
10.6.2016 VSHN AG | http://vshn.ch 15
Service Monitoring
●
Layer 7: HTTP, SMTP, etc
●
Layer 6: SSL, certificates, protocol...
10.6.2016 VSHN AG | http://vshn.ch 16
Backup
●
As WORM as feasible
– Restricted admin access, no access for customer staf
...
10.6.2016 VSHN AG | http://vshn.ch 17
Version Management
●
Everything is in version management = GIT
– Customer code
– Con...
10.6.2016 VSHN AG | http://vshn.ch 18
Continuous integration
●
Trigger Build/Package/Test/Deploy on each commit
– Targets ...
10.6.2016 VSHN AG | http://vshn.ch 19
Automated testing
●
All code is tested automatically
– Customer code
– Config manage...
10.6.2016 VSHN AG | http://vshn.ch 20
Databases & Backends
●
Growing list of 'standard software' needed as backends for
cu...
10.6.2016 VSHN AG | http://vshn.ch 21
Web & Application Servers
●
Growing list of application servers
– PHP, Python, Ruby,...
10.6.2016 VSHN AG | http://vshn.ch 22
Customer case 1
●
Server stack (Puppet)
– Nginx, Varnish
– PHP versions 5.6 and 7
– ...
10.6.2016 VSHN AG | http://vshn.ch 23
Case1
10.6.2016 VSHN AG | http://vshn.ch 24
Customer case 2
●
OpenShift: PaaS Plattform as a Service
●
Docker, Kubernetes (Googl...
10.6.2016 VSHN AG | http://vshn.ch 25
OpenShift
10.6.2016 VSHN AG | http://vshn.ch 26
About VSHN
●
Swiss DevOps & Ops Company, 17 people in Zürich
●
Building the tools an...
10.6.2016 VSHN AG | http://vshn.ch 28
Thanks
●
Questions ?
●
We're hiring System and Software Engineers @vshn_ch !
●
Get i...
Nächste SlideShare
Wird geladen in …5
×

Securing DevOps

477 Aufrufe

Veröffentlicht am

Presented at Area41.io (Defcon Switzerland Conference) 10.6.2016

Veröffentlicht in: Präsentationen & Vorträge
  • Als Erste(r) kommentieren

Securing DevOps

  1. 1. SecDevOps securing DevOps Area41 10.6.2016 Zürich Aarno Aukia VSHN AG - The DevOps Company
  2. 2. 10.6.2016 VSHN AG | http://vshn.ch 2 About me ● MSc Computer Science ETH ● Security Operations @ Google ● Co-Founder & CTO @ Atrila (Security Operations) ● Co-Founder & CTO @ VSHN (DevOps) ● Spare Time: Event Networks/WiFi at Area41 ● @aarnoaukia ● http://about.me/aarno
  3. 3. 10.6.2016 VSHN AG | http://vshn.ch 3 Agenda ● DevOps ? ● Where is the security ? ● Customer example 1 ● Customer example 2 ● Discussion
  4. 4. 10.6.2016 VSHN AG | http://vshn.ch 4 DevOps? ● Collaboration: Development (Dev) and Operations (Ops) ● Bring agile software engineering methods to operations – Automation: infrastructure as a code, versioning/rollback – Testing: continuous integration/testing/deployment ● Bring operations engineering experience to developers – Scalability: independent microservices – Production insight: monitoring/logging/metrics ● Together: make the application's owner happier
  5. 5. 10.6.2016 VSHN AG | http://vshn.ch 5 Dev + Ops collaboration ● Bring together Developers & Operations ● Practice agile Operations Engineering ● Counter fear of change with (automated) testing ● Provide developer and development infrastructure – Tools for developers, preferably self-served
  6. 6. 10.6.2016 VSHN AG | http://vshn.ch 6 Infrastructure as code ● Change from hand-groomed servers to Operations Engineering (from pets to cattle) ● Speed & reliability ● Versioning & rollback ● Prerequisite for self-service – Give each developer a full stack – No manual changes in production – As many testing instances as needed
  7. 7. 10.6.2016 VSHN AG | http://vshn.ch 7 Infrastructure tools ● Packaging code & dependencies for atomic deployment/rollback – Deb/rpm, Docker ● Infrastructure state management (configuration mgmt) – Puppet, Salt, Chef, Ansible ● Continuous Integration/Testing/Deployment – Jenkins/TravisCI/GitlabCI/Atlassian Bamboo ● Self-Service – Vagrant/Docker or through Continuous Deployment
  8. 8. 10.6.2016 VSHN AG | http://vshn.ch 8 Infrastructure testing ● Bring software engineering best practice to operations ● Large complex infrastructure (as code) → many moving parts – Unit testing each module (webserver setup, database setup, cache setup, etc) – Functional end-to-end testing of full stack (request to cache delivers content from database) ● Basically the same thing as production service monitoring but for each change
  9. 9. 10.6.2016 VSHN AG | http://vshn.ch 9 Infrastructure feedback ● Collect all logs in ELK (Elasticsearch, Logstash & Kibana) – Let the developers search for prod error root cause – No sudo/root access to production needed – Added value: merged & indexed ● Collect Server & Application Metrics – Correlate with deployments & site traffic
  10. 10. 10.6.2016 VSHN AG | http://vshn.ch 10 Software Delivery Automation
  11. 11. 10.6.2016 VSHN AG | http://vshn.ch 11 Where is the security ?
  12. 12. 10.6.2016 VSHN AG | http://vshn.ch 12 Developers ● Duh! ● Education, education, education ● Concept/architecture/code audits ● Use proven libraries ● ...
  13. 13. 10.6.2016 VSHN AG | http://vshn.ch 13 Configuration management ● Declare target state ● Enforce state every x minutes, e.g. 15min ● Establish baseline system security – Services enabled/disabled – System (admin) users, groups, keys, hashes, sudoers – AAA (AD/LDAP) for 'normal users' – Host firewall (e.g. iptables) – Installed software
  14. 14. 10.6.2016 VSHN AG | http://vshn.ch 14 Logging ● Audit logging (who changed what when) ● Application/request log ● As WORM as feasible for the customer – Generally read-only for 'normal users' – Restricted admin access ● ELK-Stack – Transport, parsing, ingest: Logstash – Storage & Indexing: Elasticsearch – Querying & Dashboard: Kibana
  15. 15. 10.6.2016 VSHN AG | http://vshn.ch 15 Service Monitoring ● Layer 7: HTTP, SMTP, etc ● Layer 6: SSL, certificates, protocols, ciphers, etc ● System parameters ● Updates ● Backup ● Tool: Icinga2
  16. 16. 10.6.2016 VSHN AG | http://vshn.ch 16 Backup ● As WORM as feasible – Restricted admin access, no access for customer staf – Only new data can be pushed ● Servers are enrolled automatically by configuration management – Enforcing the backup target will not be in the same location/infrastructure ● Data encrypted at source server using multiple keys ● Control connections use SSL/TLS ● Continuously monitored, regularly restore-tested
  17. 17. 10.6.2016 VSHN AG | http://vshn.ch 17 Version Management ● Everything is in version management = GIT – Customer code – Configuration management code & config ● Changes/commits feed into audit log ● Shared or dedicated service – Shared: github.com, bitbucket.com, gitlab.com – Dedicated: Atlassian Bitbucket, Gitlab ● AAA through AD/LDAP ● Since all Devs have offline copies: no credentials in code !
  18. 18. 10.6.2016 VSHN AG | http://vshn.ch 18 Continuous integration ● Trigger Build/Package/Test/Deploy on each commit – Targets configurable per repository, branch, tag – Manual 'promote' e.g. of production release – Feed into audit log – Store completed build/package artifacts ● Artifactory ● (private) docker registry ● Deb/RPM repository ● Feed back status to Git-GUI, dashboard, monitoring ● AAA through AD/LDAP
  19. 19. 10.6.2016 VSHN AG | http://vshn.ch 19 Automated testing ● All code is tested automatically – Customer code – Config management code & parameters ● Testing depth depends on customer... – Syntax, coding style (lint), static code analysis – Unit tests ● 'Does this module do what it is supposed to do?' – Functional tests ● 'Does the application behave correctly end-to-end?' ● detect changes in nikto/sqlmap output ?
  20. 20. 10.6.2016 VSHN AG | http://vshn.ch 20 Databases & Backends ● Growing list of 'standard software' needed as backends for customer applications – MySQL/MariaDB/Galera/MaxScale, PostgreSQL, Redis, MongoDB, RabbitMQ, Memcached, Solr, Elasticsearch, NFS/DRBD, Ceph ● All services automatically deployed by configuration management ● Provide each service with sane config, clustering, credential management, firewall config, backup config, monitoring config
  21. 21. 10.6.2016 VSHN AG | http://vshn.ch 21 Web & Application Servers ● Growing list of application servers – PHP, Python, Ruby, Java/Tomcat, Java/Wildfly, Java/Play, Coldfusion, Docker ● Provide each service with sane config, firewall config, backup config, monitoring config ● Provide backend credentials through environment variables – http://12factor.net ● Other standard components: – Apache, Nginx, Varnish, mod_security, HAproxy, OpenVPN, iptables, pacemaker, keepalived
  22. 22. 10.6.2016 VSHN AG | http://vshn.ch 22 Customer case 1 ● Server stack (Puppet) – Nginx, Varnish – PHP versions 5.6 and 7 – MySQL/MariaDB-Galera-Cluster – Memcached/Redis/Solr/Elasticsearch ● Application Deployment/update (Ansible/SSH) ● Bundle know-how (settings, tunings, etc.) in common module, override if necessary per customer through YAML-File in Git- Repository ● Docker image for local testing/developing
  23. 23. 10.6.2016 VSHN AG | http://vshn.ch 23 Case1
  24. 24. 10.6.2016 VSHN AG | http://vshn.ch 24 Customer case 2 ● OpenShift: PaaS Plattform as a Service ● Docker, Kubernetes (Google), Openshift (Redhat) ● 100% Opensource, enterprise support available ● Swiss public PaaS: appuio.ch ● EU/US public: AWS ● Dedicated/private available worldwide – AWS – Enterprise on-premises
  25. 25. 10.6.2016 VSHN AG | http://vshn.ch 25 OpenShift
  26. 26. 10.6.2016 VSHN AG | http://vshn.ch 26 About VSHN ● Swiss DevOps & Ops Company, 17 people in Zürich ● Building the tools and workflows for self-service ● Managing web applications in any cloud – We are cloud-agnostic: we run on AWS, MSA, GCE, DO, Hetzner, OVH, SafeSwissCloud, Cloudscale, Exoscale and on any on-premises Enterprise private cloud ● We work for Amazee Labs, Liip, Mercedes Benz Switzerland,Migros, SaltCinema, SIX Group, Sherpany, Sobrado, Starticket, Suisa, Taskfleet, zurichopenair.ch, etc. ● How can we help YOU?
  27. 27. 10.6.2016 VSHN AG | http://vshn.ch 28 Thanks ● Questions ? ● We're hiring System and Software Engineers @vshn_ch ! ● Get in touch with @aarnoaukia, @tobruzh or @vshnemanuel

×