CI and CD Across the Enterprise with Jenkins (devops.com Nov 2014)

CI and CD Across the Enterprise with Jenkins (devops.com Nov 2014)
CI and CD Across the Enterprise with Jenkins Cyrille Le Clerc, CloudBees ©2014 CloudBees, Inc. All Rights Reserved
©2014 CloudBees, Inc. All Rights Reserved About Me @cyrilleleclerc Cyrille Le Clerc Open Source CTO Product Management DevOps, Infra as Code, Con9nuous Delivery
©2014 CloudBees, Inc. All Rights Reserved Agenda • Con9nuous Delivery & Jenkins Workflow • Scaling Jenkins • Con9nuous Delivery with Jenkins and Puppet and Chef
Continuous Delivery ©2014 CloudBees, Inc. All Rights Reserved
Continuous Delivery ©2014 CloudBees, Inc. All Rights Reserved • “Reliable So+ware Releases through Build, Test, and Deployment Automa=on” • DONE = Shippable into Produc9on
Continuous Integration Steps ©2014 CloudBees, Inc. All Rights Reserved Con$nuous Integra$on Source Code Quality Analysis Compile & Unit Tests Binaries
Continuous Delivery Steps QA ©2014 CloudBees, Inc. All Rights Reserved Con$nuous Delivery DEV DEV OPS Source Code Quality Analysis Compile & Unit Tests Binaries Perfs Environment Func Test Environment UAT Environment Production Environment GO LIVE! Release! Staging Environment
Continuous Delivery Pipeline ©2014 CloudBees, Inc. All Rights Reserved Continuous Delivery Pipeline Compile & Unit Tests Perfs Tests Selenium Tests Deploy UAT Sonar Analysis Deploy Staging Deploy Production
Jenkins: Hub of Continuous Delivery ©2014 CloudBees, Inc. All Rights Reserved • Connect the dots • It is the process Source Code Perfs Environment Func Test Environment UAT Environment Binaries Quality Analysis Compile & Unit Tests
CD Pipeline è Workflow ©2014 CloudBees, Inc. All Rights Reserved Continuous Delivery Pipeline Compile & Unit Tests Perfs Tests Selenium Tests Deploy UAT Sonar Analysis Deploy Staging Deploy Production
©2014 CloudBees, Inc. All Rights Reserved What we need • Complex pipelines involving mul9ple stages • Non-‐sequen$al logic such as loops, forks … • Long-‐running builds must survive outages • Interac$ons with humans including pauses, input • Restartable builds in case of transient errors • Reusable defini$ons to avoid duplica9on • Comprehensible scripts with one clear defini9on
Workflow until today ©2014 CloudBees, Inc. All Rights Reserved • Job chaining • Build Flow Plugin • Build Pipeline Plugin • Build Delivery Pipeline
Workflow until today • Many atomic jobs ©2014 CloudBees, Inc. All Rights Reserved • Hard to share variables/state between jobs • Limited logic • Mix build triggers, parameterized build …
Build Flow Plugin • Did have scriptability and extensibility ©2014 CloudBees, Inc. All Rights Reserved • Did not address configura9on “sprawl” • Disjointed view of what really ran • No ability to survive restarts • Almost good enough but could not go further
New Workflow Syntax ©2014 CloudBees, Inc. All Rights Reserved
©2014 CloudBees, Inc. All Rights Reserved Demo time! https://github.com/cyrille-leclerc/spring-petclinic/blob/petclinic-1.0.0-clc/src/main/jenkins/workflow.groovy
CD Pipeline è Workflow ©2014 CloudBees, Inc. All Rights Reserved Continuous Delivery Pipeline Compile & Unit Tests Perfs Tests Selenium Tests Deploy UAT Sonar Analysis Deploy Staging Deploy Production
©2014 CloudBees, Inc. All Rights Reserved Key features • En9re flow is one concise Groovy script – For loops, try-‐finally, fork-‐join … • Can restart Jenkins while flow is running • Allocate slave nodes and workspaces – As many as you want, when you want • Stages throYle concurrency of builds • Human input/approval integrated into flow • Standard project concepts: SCM, ar9facts, plugins
Groovy DSL vs. Graphical Workflow • Familiar control flow construc9on ©2014 CloudBees, Inc. All Rights Reserved • Familiar “tools” for building abstrac9ons Func9ons, classes, variables, … • Workflow in version control • As opposed to: Graphical workflow designer
©2014 CloudBees, Inc. All Rights Reserved Project setup • One workflow is defined as a job • Single script for all steps • Build triggers & parameters like regular projects • Each workflow execu9on is a regular Jenkins build displayed in regular Jenkins views • Graphical visualiza9on of actual build possible Not of visualiza9on job defini9on but of build execu9on
Non-sequential logic try {} catch {} ©2014 CloudBees, Inc. All Rights Reserved
Interaction with humans ©2014 CloudBees, Inc. All Rights Reserved
Restartable build / checkpoint ©2014 CloudBees, Inc. All Rights Reserved
©2014 CloudBees, Inc. All Rights Reserved Visualization
©2014 CloudBees, Inc. All Rights Reserved Stages • Semaphore • Visualiza9on
Comprehensible scripts ©2014 CloudBees, Inc. All Rights Reserved
Scaling with Jenkins ©2014 CloudBees, Inc. All Rights Reserved
Problem Statement: Scale! • How do I scale Jenkins from 10 to 1000 jobs? Projects? Users? • How do I comply with corporate rules? • How do I keep Jenkins flexible for project teams? ©2014 CloudBees, Inc. All Rights Reserved
Typical Jenkins installations Let’s think about this for a few minutes ©2014 CloudBees, Inc. All Rights Reserved
The top down “structured” and “lets grow vertically” approach ©2014 CloudBees, Inc. All Rights Reserved 5 projects 0 Slaves 25 projects 5 Slaves 75 projects 50 Slaves 150 projects 400 Slaves
Vertical Scaling issues When is the next failure? Single point of failure ©2014 CloudBees, Inc. All Rights Reserved
The “organic” agile growth – “let each group do what they want to” approach Horizontally… virally ©2014 CloudBees, Inc. All Rights Reserved
Horizontal Issues: I Would Like to… • Share executors between masters – Not possible today ©2014 CloudBees, Inc. All Rights Reserved Master-‐a Master-‐b Master-‐c
Horizontal Issues: I Would Like to… • Share security information – Security realm and roles • Push configurations & upgrades ©2014 CloudBees, Inc. All Rights Reserved Master-‐a Master-‐b Master-‐c
oc.cloudbees.com Jenkins Opera9ons Center Sample Topology ©2014 CloudBees, Inc. All Rights Reserved Master-‐a Master-‐b Master-‐c Shared Roles , Security Realm and versions AYached-‐master client AYached-‐master client AYached-‐master client pushed to client master pushed to client master
©2014 CloudBees, Inc. All Rights Reserved Demo time!
Continuous Delivery traceability with Jenkins and puppet and Chef ©2014 CloudBees, Inc. All Rights Reserved xxx Environment
Puppet environments Prod style environment: puppet apply ©2014 CloudBees, Inc. All Rights Reserved – Master & agents petclinic.war Dev Environment petclinic.war Puppet Agent Application Server war file + puppet manifests & modules “Production style” Environment Puppet Master Puppet Agent war file + puppet manifests & modules Application Server Dev environment: – Standalone agents + “puppet apply”
Test environment with Puppet and Vagrant Dev Environment petclinic.war Puppet Agent Application Server vagrant provision puppet apply war file + puppet manifests ©2014 CloudBees, Inc. All Rights Reserved & modules Vagrant to create “clean” puppet managed servers
CI with Jenkins and Vagrant and Puppet petclinic.war vagrant provision puppet apply Jenkins builds war file and puppet modules and deploys via vagrant ©2014 CloudBees, Inc. All Rights Reserved puppet deployment module java code Dev Environment Puppet Agent Application Server war file + puppet manifests & modules
CI with Jenkins and Vagrant and Puppet petclinic.war notification vagrant provision puppet apply Jenkins builds war file and puppet modules and deploys via vagrant ©2014 CloudBees, Inc. All Rights Reserved puppet deployment module java code Dev Environment Puppet Agent Application Server war file + puppet manifests & modules
Jenkins – Puppet Integration ©2014 CloudBees, Inc. All Rights Reserved
©2014 CloudBees, Inc. All Rights Reserved Demo time! https://github.com/CloudBees-community/vagrant-puppet-petclinic/tree/vagrant-puppet-petclinic-0.1.0 jenkins-workflow-with-vagrant-and-puppet.groovy: https://gist.github.com/cyrille-leclerc/7f621c341e193622c844
Jenkins & Puppet • Jenkins Deployment No9fica9on Plugin – hYps://wiki.jenkins-‐ci.org/display/JENKINS/Deployment+No9fica9on+Plugin ©2014 CloudBees, Inc. All Rights Reserved • Puppet module for Jenkins tracking – hYps://github.com/jenkinsci/puppet-‐jenkinstracking • Puppet Labs plugin for Jenkins – hYps://wiki.jenkins-‐ci.org/display/JENKINS/Puppet+Plugin • CloudBees DevOps page – hYps://developer.cloudbees.com/bin/view/Main/DevOps • This demo – hYps://github.com/CloudBees-‐community/vagrant-‐puppet-‐petclinic – Jenkins Workflow with Vagrant and Puppet: hYps://gist.github.com/cyrille-‐leclerc/7f621c341e193622c844
©2014 CloudBees, Inc. All Rights Reserved Jenkins & Chef? • Similar concepts • Jenkins Deployment No9fica9on Plugin – hYps://wiki.jenkins-‐ci.org/display/JENKINS/Deployment+No9fica9on+Plugin • Puppet module for Jenkins tracking – hYps://github.com/jenkinsci/puppet-‐jenkinstracking • Jenkins Chef Tracking – hYps://wiki.jenkins-‐ci.org/display/JENKINS/Chef+Tracking+Plugin • CloudBees DevOps page – hYps://developer.cloudbees.com/bin/view/Main/DevOps
©2014 CloudBees, Inc. All Rights Reserved Conclusion • Jenkins workflow to build Con9nuous Delivery pipelines • Jenkins Opera9ons Center by CloudBees to scale Jenkins • Puppet & Chef deployments can be integrated in Jenkins
©2014 CloudBees, Inc. All Rights Reserved

Check these out next

CI and CD Across the Enterprise with Jenkins (devops.com Nov 2014)

Recomendados

Más contenido relacionado

Presentaciones para ti(20)

Similar a CI and CD Across the Enterprise with Jenkins (devops.com Nov 2014)(20)

Más de CloudBees(20)

Último(20)

CI and CD Across the Enterprise with Jenkins (devops.com Nov 2014)