Jenkins as a Service - Code all the way down
•Als PPTX, PDF herunterladen•
2 gefällt mir•2,300 views
This presentation walks through a Jenkins as Code approach that aims to fully automate and describe the creation of Infrastructure, Application and Configuration as Code. We treat our applications with a strong 'as code' approach, but often forget about the critical operational tools. This presentation shows how it is possible to create a code first approach to creating and managing a Jenkins Service. Working code repository is available at https://bitbucket.org/stevemac/dockerfiles
Empfohlen
Empfohlen
Weitere ähnliche Inhalte
Was ist angesagt?
Was ist angesagt? (20)
Ähnlich wie Jenkins as a Service - Code all the way down
Ähnlich wie Jenkins as a Service - Code all the way down (20)
Mehr von Steve Mactaggart
Kürzlich hochgeladen
Kürzlich hochgeladen (20)
Jenkins as a Service - Code all the way down
- 1. Jenkins as a Service Code all the way down Steve Mactaggart Application Delivery Evangineer
- 2. The moving parts Infrastructure (AWS / ECS) Application (Jenkins) Configuration (Jenkins Jobs)
- 3. The moving parts - AS CODE Infrastructure (AWS / ECS) Application (Jenkins) Configuration (Jenkins Jobs) CloudFormation Docker containers Groovy / Jenkins DSL
- 4. Everything as code • Consistently repeatable process • Enables a strong self service workflow • Highly visible traceability of change • Roll forward / rollback
- 5. The moving parts - Infrastructure Infrastructure (AWS / ECS) Application (Jenkins) Configuration (Jenkins Jobs) CloudFormation Docker containers Groovy / Jenkins DSL
- 6. Infrastructure • Infrastructure captured as Code • Designed for easy re-use per team • Deployable into existing VPCs • Common location to share learned lessons Infrastructure / AWS
- 7. EC2 Infrastructure - Simplified ap-southeast-2bap-southeast-2a ECS agent auto-scaling group ECS master auto-scaling group EC2 EC2 Infrastructure / AWS / CloudFormation ECS configuration Master ECR Agent ECR AWS ECS CloudWatch logs
- 8. EC2 Infrastructure ap-southeast-2bap-southeast-2a ECS agent auto-scaling group ECS master auto-scaling groupECS configuration Master ECR Agent ECR Legend ECS ScaleDown function ECS Memory Reservation Alarm Termination Event AWS ECS EC2 EC2 Query for tasks running on host Continue the ASG lifecycle hook Publish intent to terminate Watch for ScaleUp / ScaleDown alarm Consume message Containers download configuration ECSLaunches AgentTask Launch / Terminate EC2 Instance Jenkins Schedules new Task via ECS JenkinsAgent phoneshometo Mastercontainer Direct TCP call S3 download AWS API Call Retrieve Master container from ECR Retrieve Agent container from ECR EC2instance registerswith ECS Agentcontainer registerswithmaster ALB routes web traffic Infrastructure / AWS / CloudFormation CloudWatch logs EC2 / ECS logs shipped to CloudWatch
- 9. Infrastructure • Example CloudFormation • https://bitbucket.org/stevemac/dock erfiles/src/282408c04239/cloudforma tion/?at=master • 2 stacks • Infrastructure (2x ECS clusters) • Cluster - Jenkins Master Service
- 10. The moving parts - Application Infrastructure (AWS / ECS) Application (Jenkins) Configuration (Jenkins Jobs) CloudFormation / Terraform Docker containers Groovy / Jenkins DSL
- 11. EC2 Jenkins in Docker ap-southeast-2bap-southeast-2a ECS agent auto-scaling group ECS master auto-scaling group EC2 EC2 ECS configuration Master ECR Agent ECR AWS ECS CloudWatch logs Jenkins Application / Docker containers Jenkins Master Jenkins Agents
- 12. The Jenkins Master and its Agents • Jenkins is an Application • We understand the Docker application workflow • Immutable Applications increase confidence of changes • Based of official JenkinsCI base containers • From jenkinsci/Jenkins • From jenkinsci/slave • Extended to include specific build and deployment tools • Python / AWSCli / etc Jenkins Application / Docker containers
- 13. Jenkins master - Dockerfile Jenkins Application / Docker containers
- 14. Jenkins agent - Dockerfile Jenkins Application / Docker containers
- 15. Jenkins in Docker • Many plugins in-use: • build-pipeline-plugin • amazon-ecs • delivery-pipeline-plugin • build-monitor-plugin • gitlab-merge-request-Jenkins • Etc… • State persisted through Docker volume mount • Example Master and Agent Docker containers • https://bitbucket.org/stevemac/dockerfiles/src/
- 16. The moving parts - Configuration Infrastructure (AWS / ECS) Application (Jenkins) Configuration (Jenkins Jobs) CloudFormation / Terraform Docker containers Groovy / Jenkins DSL
- 17. Configuration as Code - setup • Start of the configuration delivered in Dockerfiles • Configuration and ssh keys downloaded from S3 bucket • Supplied to container as Environment variable Configuration / Groovy
- 18. Configuration as Code - setup • Runtime configuration automated through use of custom Groovy initialisation scripts within the container • Groovy is used to configure: • Jenkins credentials • ECS Cluster configuration • Jenkins Bootstrap job • Executed as part of Dockerfile entrypoint Configuration / Groovy
- 19. Pipeline as Code - runtime • Using the Jenkins DSL pipeline • https://bitbucket.org/jenkinspipeline/jenkins-dsl-examples • Using the power of Groovy code to create reusable elements • Version controlled through Git • Repeatable / reliable / extensible • See https://github.com/jenkinsci/job-dsl-plugin/wiki/Tutorial---Using-the-Jenkins-Job-DSL • Automatic documentation available within each Jenkins instance Configuration / Jenkins DSL Lets talk about a Jenkinsfile approach over a beer
- 20. Pipeline as Code – groovy constructs Configuration / Groovy / Jenkins DSL
- 21. Pipeline as Code - example https://bitbucket.org/jenkinspipeline/jenkins-dsl-examples 1. build_pipeline.groovy Creation of example build jobs - CI 2. delivery_pipeline.groovy Creation of example jobs for a candidate delivery pipeline - CD 3. views.groovy Creation of some standard pipeline views Configuration / Groovy / Jenkins DSL
- 22. Bringing it together Let our codebases combine
- 23. Executed within ECS Jenkins service Result of infrastructure deploy Performed by operator Delivery workflow Build and Publish Docker Containers Launch Stack Publish keys to S3 bucket Create ECS Service Retrieve latest Docker image from ECR Launch ECS Task Download keys from S3 Clone groovy config repo Launch Jenkins Master Process Groovy config executed Bootstrap job generates pipelines
- 24. Followup discussions 1. Migrating to a Jenkins-as-code approach 2. Jenkins DSL vs Jenkinsfiles 3. All about Jenkins State 4. Security in a stateless world 5. Advanced Jenkins DSL patterns
- 25. Thanks ! Steve Mactaggart • Application Delivery Evangineer at Cevo • @stevemac Example working code available at: https://bitbucket.org/stevemac/dockerfiles