Diese Präsentation wurde erfolgreich gemeldet.
Wir verwenden Ihre LinkedIn Profilangaben und Informationen zu Ihren Aktivitäten, um Anzeigen zu personalisieren und Ihnen relevantere Inhalte anzuzeigen. Sie können Ihre Anzeigeneinstellungen jederzeit ändern.
Wird geladen in …3
×
47 von 51

CI/CD with Docker, DC/OS, and Jenkins

8

Teilen

Herunterladen, um offline zu lesen

Companion slide from Velocity NY 2-day training.
Labs: https://github.com/mesosphere/training/blob/master/velocity-training-09-2016/

Ähnliche Bücher

Kostenlos mit einer 30-tägigen Testversion von Scribd

Alle anzeigen

Ähnliche Hörbücher

Kostenlos mit einer 30-tägigen Testversion von Scribd

Alle anzeigen

CI/CD with Docker, DC/OS, and Jenkins

  1. 1. © 2016 Mesosphere, Inc. All Rights Reserved. 1 CI/CD WITH DOCKER, DC/OS, AND JENKINS ( Velocity NY 2016 Edition )
  2. 2. © 2016 Mesosphere, Inc. All Rights Reserved. 2 KARL WHO? Where? - Mesosphere - Pivotal What? - DC/OS - Kubernetes (on Mesos) - CloudFoundry - BOSH - github.com/karlkfi - twitter.com/karlkfi - linkedin.com/in/karlkfi - karl.isenberg.us
  3. 3. © 2016 Mesosphere, Inc. All Rights Reserved. 3 WHO ARE YOU?
  4. 4. © 2016 Mesosphere, Inc. All Rights Reserved. 4 INTEGRATION Build/Test Manual Review/Merge Build/Test Test all the pieces together DELIVERY Integration Test Release/Deployment Manual Acceptance Prod Release/Deployment Integration branch is always deployable DEPLOYMENT Integration Validation Release/Deployment Integration branch is validated and deployed automatically CONTINUOUS EVERYTHING
  5. 5. © 2016 Mesosphere, Inc. All Rights Reserved. 5 SOURCE Branched > Developed > Compiled > Tested > Reviewed > Merged > Tagged PACKAGES Built > Tested > Versioned > Released > Promoted > Installed/Vendored SERVICES Deployed > Tested > Accepted/Validated > Promoted RELEASE vs. DEPLOY
  6. 6. © 2016 Mesosphere, Inc. All Rights Reserved. 6 PAIN - Expensive manual merges - Independent refactoring - Incompatible dependencies - Works on my machine - Different environments - Who owns integration? - Mismatched Interfaces - Large upfront planning & communication WHY CONTINUOUS INTEGRATION? RELIEF - Small change sets - Feature branches - Package extraction - Interface extraction - Provisionable environment - Centralized testing - Automatic build/test triggers - Configuration as code - Integration package
  7. 7. © 2016 Mesosphere, Inc. All Rights Reserved. 7 PAIN - Manual, undocumented release process - Siloed, tribal knowledge - Bus factor of one - Poor process communication - Infrequent end-to-end testing - Bugs caught right before/after release/deployment - Slow onboarding - Manual regression testing WHY CONTINUOUS DELIVERY? RELIEF - Programmatic dependency management - Automated package management - Automated release building - Automated deployment - Automated promotion - Secrets management - Unit/integration/acceptance tests - Test/UAT/Acceptance environment
  8. 8. © 2016 Mesosphere, Inc. All Rights Reserved. 8 PAIN - Unhappy customers - Infrequent/late validation - Many/large simultaneous changes - Making decisions without data - Unhappy developers - Process limiting innovation - Infrequent feedback - Long development cycles - Large upfront planning WHY CONTINUOUS DEPLOYMENT? RELIEF - Automated front-end tests - Automated end-to-end tests - Programmatic definition of acceptance - Acceptance driven development - Milestone releases, not release milestones - Release train - Decouple feature promises from release numbers - Automatic semantic versioning
  9. 9. © 2016 Mesosphere, Inc. All Rights Reserved. 9 TOOLS
  10. 10. © 2016 Mesosphere, Inc. All Rights Reserved. 10 DOCKER ( Container Runtime, Image Format, and Repository )
  11. 11. © 2016 Mesosphere, Inc. All Rights Reserved. 11 DOCKER CONTAINER isolated process tree isolated filesystem optional volumes optional network routing optional resource constraints
  12. 12. © 2016 Mesosphere, Inc. All Rights Reserved. 12 RUNTIME The docker-engine runs on linux machines and manages containers and container images. IMAGE A container image is a set of file system layers that can be shipped around as a unit and is used as a template for creating containers. REPOSITORY A docker image repository is a service that manages and store docker images and image layers. CONTAINER ...
  13. 13. © 2016 Mesosphere, Inc. All Rights Reserved. 13
  14. 14. © 2016 Mesosphere, Inc. All Rights Reserved. 14 BUILD ONCE, RUN ANYWHERE - Portable runtime environment - Language independent - Dependency vendoring - Packaging format - Dependency isolation - More developer control - Portable across operating systems - Fast development cycles HAPPY DEVELOPERS & OPERATORS CONFIGURE ONCE, RUN ANYTHING - Predictability - Repeatable deployment - Consistent, reliable results - Low overhead, low cost - Fast, efficient deployment - Image Layer caching - Developer awareness of operational concerns - Separation of concerns
  15. 15. © 2016 Mesosphere, Inc. All Rights Reserved. 15 CONTAINER IMAGE LIFECYCLE Image Image Image Layer Machine 1 Machine 2Image Repository Shared Layer Layer Layer Push Image Pull Dockerfile Build Container Run
  16. 16. © 2016 Mesosphere, Inc. All Rights Reserved. 16 DOCKER LAB 1 - 2
  17. 17. © 2016 Mesosphere, Inc. All Rights Reserved. 17 DC/OS ( Container Orchestration )
  18. 18. © 2016 Mesosphere, Inc. All Rights Reserved. 18 IN THE BEGINNING... Sombrero Galaxy by ESA/Hubble - CC BY 4.0
  19. 19. © 2016 Mesosphere, Inc. All Rights Reserved. Hardware Operating System Application 19 COMPUTERS
  20. 20. © 2016 Mesosphere, Inc. All Rights Reserved. Hardware Operating System Web Application 20 INTERNET - Remote Users!
  21. 21. © 2016 Mesosphere, Inc. All Rights Reserved. Web App Hardware Operating System 21 DISTRIBUTION - Horizontal Scale - Fault Tolerance - Availability - Load Balancing Operating System Operating System Hardware Hardware Web App Web App
  22. 22. © 2016 Mesosphere, Inc. All Rights Reserved. Service Service Service Web App Web App Web App Hardware Operating System 22 SERVICE- ORIENTED ARCHITECTURE - Separation of concerns - Optimization of bottlenecks - Smaller teams - API Contracts - Data replication - Complicated provisioning - Dependency management Operating System Operating System Hardware Hardware
  23. 23. © 2016 Mesosphere, Inc. All Rights Reserved. Service Service Service Web App Web App Web App Machine Operating System 23 HARDWARE VIRTUALIZATION - Fast provisioning - Isolation - Portability - Utilization - Configuration Management - Virtual Networking - Credential management Operating System Operating System Infrastructure Machine Machine
  24. 24. © 2016 Mesosphere, Inc. All Rights Reserved. Operating System Operating System Operating System ServiceApp ServiceServiceAppApp 24 MICROSERVICES - Polyglot - Single Responsibility - Smaller Teams - Utilization - Machine types/groups - Dependency hell Machine Infrastructure Machine Machine ServiceService ServiceServiceServiceService
  25. 25. © 2016 Mesosphere, Inc. All Rights Reserved. ServiceApp ServiceServiceAppApp OS 25 CONTAINERS - Rapid deployment - Dependency vendoring - Container image repositories - Spreadsheet scheduling OS OS Machine Infrastructure Machine Machine Container Runtime Container Runtime Container Runtime ServiceService ServiceServiceServiceService
  26. 26. © 2016 Mesosphere, Inc. All Rights Reserved. Service Container Runtime Container Runtime Container Runtime OS 26 CONTAINER ORCHESTRATION OS OS Container Orchestration Machine Infrastructure Machine Machine App ServiceServiceAppApp
  27. 27. © 2016 Mesosphere, Inc. All Rights Reserved. 27 CONTAINER ORCA-WHAT-NOW? Scheduling, Coordination, & Management of Containerized Processes as Higher Order Abstractions (e.g., Jobs, Services, Applications, Pods, etc.)
  28. 28. © 2016 Mesosphere, Inc. All Rights Reserved. Orchestration 28 Machine Infrastructure Web Apps & Services Scheduling Resource Management Container Runtime Machine & OS Service Management CONTAINER ORCHESTRATION Machine & OS Machine & OS Container Runtime Container Runtime Expanded
  29. 29. © 2016 Mesosphere, Inc. All Rights Reserved. 29 SCHEDULING - Placement - Replication/Scaling - Resurrection - Rescheduling - Rolling Deployment - Upgrades - Downgrades - Collocation RESOURCE MANAGEMENT - Memory - CPU - GPU - Volumes - Ports - IPs - Images/Artifacts SERVICE MANAGEMENT - Labels - Groups/Namespaces - Dependencies - Load Balancing - Readiness Checking FUNCTIONAL ORCHESTRATION CAPABILITIES
  30. 30. © 2016 Mesosphere, Inc. All Rights Reserved. 30 DC/OS LAB 1 - 2
  31. 31. © 2016 Mesosphere, Inc. All Rights Reserved. 31 DC/OS ( The Datacenter Operating System )
  32. 32. © 2016 Mesosphere, Inc. All Rights Reserved. 32 SCALABILITY Performance, Responsiveness, Efficiency AVAILABILITY Fault Tolerance, Robustness, Reliability, Resilience, Disaster Recovery FLEXIBILITY Format Support, Interoperability, Extensibility, Container Runtimes USABILITY Familiarity, Maintainability, Compatibility, Debuggability PORTABILITY Host OS, Cloud, Bare-Metal, Hybrid SECURITY Auditability, Secrets Management, Encryption, Isolation NON-FUNCTIONAL PLATFORM QUALITIES
  33. 33. © 2016 Mesosphere, Inc. All Rights Reserved. 33 PLATFORM A base of technologies on which other technologies or processes are built. PLATFORM AS A SERVICE A managed, multitenant platform. CLOUD PLATFORM A hosted platform as a service on shared infrastructure. PLATFORM SPECTRUM Rainbow by Susanne Nilsson - CC BY-SA 2.0
  34. 34. © 2016 Mesosphere, Inc. All Rights Reserved. INFRASTRUCTURE PLATFORM e.g., OpenStack, VMware vSphere CONTAINER PLATFORM e.g., Kubernetes, DC/OS, Rancher APPLICATION PLATFORM e.g., Cloud Foundry, Red Hat OpenShift, Deis 34 PLATFORMS Oil Platform by Berardo62 - CC BY-SA 2.0
  35. 35. © 2016 Mesosphere, Inc. All Rights Reserved. CLOUD INFRASTRUCTURE PLATFORM e.g., Amazon Elastic Compute Cloud, Microsoft Azure, Google Compute Engine CLOUD CONTAINER PLATFORM e.g., Google Container Engine, Azure Container Services, Amazon EC2 Container Service CLOUD APPLICATION PLATFORM e.g., Heroku, Google App Engine, Pivotal Web Services, IBM Bluemix 35 CLOUD PLATFORMS
  36. 36. © 2016 Mesosphere, Inc. All Rights Reserved. APPLICATION DEVELOPMENT LIFECYCLE Source Repo, Continuous Integration, Artifact Repo CONTAINER ORCHESTRATION Scheduling, Resource Management, Service Management DAY 2 OPERATIONS Debugging, Maintenance 36 CONTAINER OPERATIONS
  37. 37. © 2016 Mesosphere, Inc. All Rights Reserved. 37 APPLICATION DEVELOPMENT LIFECYCLE CONTAINER OPERATIONS Code Build, Test, Package Publish Deploy
  38. 38. © 2016 Mesosphere, Inc. All Rights Reserved. Applications & Services 38 APPLICATION DEVELOPMENT LIFECYCLE CONTAINER OPERATIONS Source Control Continuous Integration Artifact Repository Orchestration
  39. 39. © 2016 Mesosphere, Inc. All Rights Reserved. 39 APPLICATION DEVELOPMENT LIFECYCLE CONTAINER OPERATIONS Applications & Services Source Control Continuous Integration Artifact Repository Orchestration
  40. 40. © 2016 Mesosphere, Inc. All Rights Reserved. 40 DEBUGGING - Logging & Metrics - Emitting - Aggregation - Search - Reporting - Alerting - Dashboards - Shell Access MAINTENANCE - Package Management - Upgrading - Cluster Resizing - Application Autoscaling - Capacity Planning - Software Defined Networking - Backups - Disaster Recovery DAY 2 OPERATIONS CONTAINER OPERATIONS
  41. 41. © 2016 Mesosphere, Inc. All Rights Reserved. System Space Orchestration User Space Source Control Continuous Integration Artifact Repository 41 CONTAINER OPERATIONS Load Balancer Package Manager Security Networking Logging & Metrics Storage
  42. 42. © 2016 Mesosphere, Inc. All Rights Reserved. Kernel 42 OPERATING SYSTEM System Space Hardware User Space
  43. 43. © 2016 Mesosphere, Inc. All Rights Reserved. 43 DISTRIBUTED OPERATING SYSTEM Hardware Hardware Hardware User Space Circa 1992 Microkernel System Space Microkernel Microkernel
  44. 44. © 2016 Mesosphere, Inc. All Rights Reserved. 44 Machine Infrastructure User Services Orchestration Container Runtime System Services DISTRIBUTED OPERATING SYSTEM Container Runtime Container Runtime Revised OS OS OS Machine Machine Machine DistributedOS
  45. 45. © 2016 Mesosphere, Inc. All Rights Reserved. 45 DC/OS - Resource Management - Universal Container Runtime - Container Orchestration - Pluggable Schedulers - Jobs & Services - Distributed System Services - Virtual Networking - Package Management - Services Catalog & Application Ecosystem - Day 2 Operational Focus ENTERPRISE DC/OS - E2E Encryption - Identity & Access Management - Secrets Management - Support
  46. 46. © 2015 Mesosphere, Inc. All Rights Reserved. 46 HYPERSCALE OPERATIONS ULTIMATE INFRASTRUCTURE FLEXIBILITY FOR MODERN APPLICATIONS • Many “snowflakes” • Management nightmare • Lengthy cycles to deploy code • Low utilization Mesosphere DCOS MESOSPHERE DCOS APPROACH Container Apps (All) Big Data Analytics (All) PaaS (All) Stateful Service (All) • High performance and resource isolation • Easy scalability and multi-tenancy • Fault tolerant and highly available • Highly efficient with highest utilization Proven in mission-critical deployments Deploys on-prem or in cloud TRADITIONAL APPROACH PaaS 1 Container App 1 Big Data Analytics 1 Big Data Analytics 2 PaaS 2 Container App 2 Stateful Service 1 Stateful Service 2 Complete workload portability
  47. 47. USE, REPRODUCTION, OR DISCLOSURE IS SUBJECT TO RESTRICTIONS SET FORTH IN CONTRACT NUMBER 2014-14031000011 AND SUB CONTRACT NO. MESOSPHERE 01. VPC DC/OS ARCHITECTURE DC/OS CLI DC/OS UI DC/OS Universe Auto-ScalingGroup&SecurityGroup ELB ELB LDAP www AGENT (1..n) 47 Backend Services Spartan Minuteman Mesos DNS Mesos Agent 3DT Admin Router Docker User Tasks Cassandra Spark Kafka HDFS Containers... Apps/Tasks Jenkins Auto-ScalingGroup&SecurityGroup MASTER (1..9) Admin Router Backend Services Spartan Minuteman Mesos DNS Resolv-Conf Exhibitor Zookeeper DC/OS Signal CA Vault Marathon Mesos Master History 3DT Networking API Cosmos Bouncer Frontend Services Metronome
  48. 48. © 2016 Mesosphere, Inc. All Rights Reserved. 48 DC/OS LAB 3 - 5
  49. 49. © 2016 Mesosphere, Inc. All Rights Reserved. 49 JENKINS ( Open Source Automation Server )
  50. 50. © 2016 Mesosphere, Inc. All Rights Reserved. 50 JENKINS LAB 1 - 5
  51. 51. © 2016 Mesosphere, Inc. All Rights Reserved. 51 EXIT SURVEY

×