Presentation for Singapore Devops meetup

1. Jenkins
automation
Using Python to automate Jenkins

2. About me
● Work with builds for 8 years
● Was Java architect and developer for 7
years before that
● Originally from Russia, but lived and work in
US, UK, Denmark and now in Singapore
● Follow me on Github: @lechat

3. Questions
● How many jobs your Jenkins has?
● Do you run parallel build pipelines?
● Do you automate Jenkins?

4. Agenda
● Save Jenkins jobs in a safe place
● Making config.xml easier to handle
● Making branch builds
● Making and running build pipelines

5. I want to save my (Jenkins) job!
Jenkins backup
● Jobs
of course, they only exist in Jenkins and handcrafted to perfection
● Builds
a history of (not so) useful information
● Keys
usually forgotten essential bit of information

6. I want to save my (Jenkins) job!
Jenkins backup
● Jobs
Trivial approach - make shell script that will put files somewhere else
Another trivial approach - use Jenkins job to save Jenkins as build artefact
● Useful only as a disaster recovery
Restoring just one job requires Jenkins restart
(only in latest version there is an option to reload job definition)

7. I want to save my (Jenkins) job!
But why not keep job definition with your source code?
Use jenkinsapi library to save/restore jobs!
What jenkinsapi can do?
● Query Jenkins about jobs, builds, views, nodes
● Create/update jobs, views, nodes (only JNLP nodes for
now)
● Execute jobs

8. I want to save my (Jenkins) job!
Example 1:

9. Making easier to handle config.xml
Editing job’s config.xml is not easy
● You need to know plugin/component internal name
● XML is too verbose
● Editing does not scale

10. Making easier to handle config.xml

11. Making easier to handle config.xml
Use OpenStack jenkins-job-builder
● Job definition is in YAML
● Good defaults - you only need to specify overrides
● Less stuff to edit

12. Making easier to handle config.xml

13. Making easier to handle config.xml
Jenkins-job-builder features
● Create templates for jobs
● Create variables and expand them when job is generated
● Create default values used across multiple jobs
● Define macros for repeated sections

14. Making branch builds
Example 3: Create jobs for each git branch

15. Making job pipelines
Making job pipelines is not trivial in Jenkins

16. Making job pipelines
Making good job pipelines is not trivial in Jenkins
● No visualization when making jobs dependencies
● Have to use “Join” plugin
● Hard to run part of the pipeline

17. Making job pipelines
Use jenkinsflow library to make and run pipelines

18. Making job pipelines

19. Making job pipelines
With jenkinsflow
No dependencies between jobs in Jenkins
You can create dynamic pipelines
You can execute pipelines outside Jenkins
good for building automated Git flows

20. Summary
All in your source code repository:
● Job definitions
● Job pipelines
● Build automation
No worries about losing Jenkins instance - it can be re-
created in minutes

21. Reference
jenkinsapi
https://github.com/salimfadhley/jenkinsapi
OpenStack jenkins-job-builder
https://github.com/openstack-infra/jenkins-job-builder
jenkinsflow
https://github.com/lhupfeldt/jenkinsflow

22. Reference
Job creators + editors (summary)
https://gist.github.com/mika/5237127
XML templates - Hudson job creator
https://github.com/hudson2-plugins/jobcreator-tool
Source code for this presentation
https://github.com/lechat/devops-python-jenkins