Slide deck from Jenkins User Conference Tel Aviv 2018.
Talking about suggested (best?) practices, tips and tricks, using Jenkins pipeline scripts with shared libraries, managing shared libraries, using docker compose, and more.
WSO2Con2024 - WSO2's IAM Vision: Identity-Led Digital Transformation
Taking Jenkins Pipeline to the Extreme
1. Taking Jenkins Pipeline to the Extreme
Yinon Avraham
Tech Lead @ eBay
Copyright @ 2018 JFrog - All rights reserved | juc-il.jfrog.com
2. AGENDA
● Why
● What
● How
● WOW!
Copyright @ 2018 JFrog - All rights reserved | juc-il.jfrog.comUser Conference
3. WHY
Copyright @ 2018 JFrog - All rights reserved | juc-il.jfrog.comUser Conference
4. MICROSERVICES
Copyright @ 2018 JFrog - All rights reserved | juc-il.jfrog.comUser Conference
Service
Service
Service Service
SQL
DB
Service
SQL
DB
Service
NoSQL
DB
HTTPHTTP
HTTP HTTP
3rd Party
Service
6. IMPROVED CI/CD PROCESS
CI: For each active branch:
1. Build
2. Unit Tests
3. Publish Artifacts
4. Integration Tests
Copyright @ 2018 JFrog - All rights reserved | juc-il.jfrog.comUser Conference
M
F RF D
CI
M
F RF D
CI
Deploy & TestDeploy & Test
Deploy to
Production
Deploy to
Production
Repo 2Repo 1
Deploy to Acceptance Environment & Test
15. TIPS & TRICKS
● Do not load global libraries implicitly by default.
This is sometimes redundant and it is not visible
● Do load global libraries explicitly with specific stable development branch, e.g. “1.x”.
This way you get fixes and enhancements transparently, but keep the option for “2.x”
● Do treat your global pipeline library as any other library you write (e.g. versioning,
testing, etc.)
Copyright @ 2018 JFrog - All rights reserved | juc-il.jfrog.comUser Conference
Global Pipeline Library Management
16. TIPS & TRICKS
Copyright @ 2018 JFrog - All rights reserved | juc-il.jfrog.comUser Conference
Global Pipeline Library Management - Testing
#!groovy
@Library('buildtools@1.x') _
import static com.ebay.tools.test.PipelineTestUtils.*
withMavenDockerNode {
pipelineTests(this) {
testSuite('withBuildTools, shEx') {
test("withBuildTools - Test build tools with JDK8") {
withBuildTools(jdk: 'JDK8') { buildTools ->
def result = shEx(script: 'java -version')
assertStringContains expected: 'version "1.8.',
actual: result.get('err'),
message: 'Java version not as expected'
assertEquals expected: 0,
actual: result.get('status'),
message: "Return status not as expected"
}
}
...
}
}
}
17. TIPS & TRICKS
● Do prototype, test and play using the online script editor
● Do use the “Replay” functionality
● Do echo, echo, echo – write decisions and state to the console
● Trick: interactiveShell
Copyright @ 2018 JFrog - All rights reserved | juc-il.jfrog.comUser Conference
Pipeline Script Development and Debugging
def call() {
timeout(time: 10, unit: 'MINUTES') {
echo "[Shell] ***** Starting interactive shell *****"
String cmd = ''
while (cmd != 'exit') {
cmd = input(id: 'cmd', message: 'Command:', parameters: [
[$class: 'TextParameterDefinition', name: 'cmd',
description: 'Enter shell command to run, "exit" to stop the shell']
])
def ret = sh(script: cmd, returnStatus: true)
echo "[Shell] Return Code: $ret"
}
echo "[Shell] Bye!"
}
}
18. TIPS & TRICKS
● Do organize in stages to reflect your pipeline flow, think visibility
● Do echo, echo, echo (yes, I already wrote it, it’s important)
● Do control the result status to reflect the actual result
● Do write your global library to express your DSL
Copyright @ 2018 JFrog - All rights reserved | juc-il.jfrog.comUser Conference
Increase Visibility and Readability
19. Any Questions?
THANK YOU!
Yinon Avraham
yavraham@ebay.com
@yinonavraham
Copyright @ 2018 JFrog - All rights reserved | juc-il.jfrog.comUser Conference
Hinweis der Redaktion
Questions:
Who’s using Jenkins pipeline (i.e. Jenkinsfile)?
Who’s using global shared libraries?
Who’s using docker in their pipeline scripts?
Some background – what was our trigger to do it
Working with microservices
Currently ~15 different services, each has ~9 instances
Communicating sync (HTTP REST API) & async (Kafka)
Working with various databases (various SQL, neo4j, etc.)
Working with 3rd party services
Each service is maintained in its own git repository
We use git-flow – we have many active branches
What we needed to have in order to achieve our plans
Quite obvious, but still – this is not where we started…
We are not there yet
In this PPT we focus on the CI part
Requirements:
Build configuration changes together with the application code, and without affecting other branches
Git-flow – give quick feedback for feature branches
Needs to be quick and easy to see what failed and why
Many services in many repositories, common build process.
Concurrent builds should not affect one another, no need to statically maintain databases for each application / branch …
As less manual Jenkins configuration as possible. It should be contained with the build configuration as much as possible.
Answers:
Simple: Jenkinsfile
Simple: Multibranch Pipelines, trigger on push to GitHub
Simple: The native Jenkins Pipeline UI is very good: which stage failed, quick access to console output, etc.
Focus on: Jenkins Pipeline Shared Libraries
Focus on: Use docker-compose to spin up “sidecar” services
Focus on: Push as much configuration as possible to the Jenkinsfile (timestamps, parameters, pods, etc.)
Some examples of how we implemented it
Some tips and tricks we collected along the way
This is somewhat simplified and hides implementation.
Reuse: forcePublishParam, discardOldBuildsProperty
DSL: withMavenDockerNode, failAsUnstable, withDockerEx
Tip: echo, echo, echo
Go to live Jenkins to show how a successful job looks like and how an echoed message looks like…
Jenkins Master – Orchestrates and drives the builds
Jenkins Slave – Executes the build steps
Kafka “Sidecar” – an ad-hoc composition started per build, controlled by the build, isolated from other builds
withDockerEx – Creates the dockerEx handle, closes it when finished (successfully or not)
startKafkaSidecar – Starts the kafka composition, checks it is up, and returns its IP
DockerEx keeps all started compositions in order to destroy them implicitly once done
DockerExCompose – Give a nice fluent and familiar DSL. Underline it is just calling the docker-compose CLI.
inspectGateway – Using docker network inspection CLI with its built-in formatting capabilities to extract the composition’s network gateway.
Push as much job configuration to the global pipeline script or the project specific script – where it fits
Use podTemplate to define the Jenkins slave image you use, instead of Jenkins Configuration
Print information on your build environment – it might be quite handy when the build fails…
Show example of the console output of printing tools version