Slides for the talk on Commit Conf 2018 about "Multilanguage Pipelines with Jenkins, Docker and Kubernetes"

1. SPAIN
ATCINFINITE
POSSIBILITIES
MULTILANGUAGE
PIPELINES WITH
JENKINS, DOCKER
AND KUBERNETES
Jorge Hidalgo @_deors @deors
Toni de la Prieta @tldpa @antonio-de-la-prieta-antoli
COMMIT CONF NOVEMBER 2018

2. WHO WE ARE
Toni de la Prieta @tldpa
@antonio-de-la-prieta-antoli
Director
Accenture Technology Spain
Spain Custom Software Engineering Lead
Jorge Hidalgo @_deors
@deors
Senior Technology Architect
Accenture Technology Spain
Spain ATC Custom & Architecture Lead
Honorable mentions:
Vicente González, Julio Palma and the whole
Architecture & DevOps Squad at Spain ATC

3. TL;DR
All code and pipeline examples can be found online
in these repositories:
https://github.com/deors/deors-demos-petclinic
https://github.com/deors/deors-demos-python-pipeline
Don’t hesitate to ask, log issues or send your pull
requests – HACKERS WELCOMED!

4. AT THE BEGINNING...
...ONLY PAIN AND BREAKING BUILDS

5. ARE THESE FAMILIAR TO YOU?
Code change paralysis
Production environments
hard to observe
Lack of automation Inadequate code analysis
Obsolete processes (waste)
Brittle, tightly coupled
applications
Limited test environments Stop-the-world deployments

6. 50 DEPLOYMENTS PER DAY IS POSSIBLE
Netflix proactively identify
and resolve issues before
they impact customers
Amazon deploys new
features to production
every 11 seconds
Uber deploys more
than 500 times per
day

7. WHAT ‘AWESOME’ LOOKS LIKE
(THE VALUE PROPOSITION OF MODERN ENGINEERING)
Provision environments dynamically
Spawn a new environment in under 5 minutes
Have “eagle eyes” to observe everything
Precise, real-time metrics on KPIs of interest
No manual effort in repetitive tasks
All common tasks are automated
Eliminate regression and security defects
Achieving 0 regression defects is feasible
Deliver at the pace of business
From weeks to hours or minutes
Build resilient, self-healing apps
No downtime, no unexpected errors

8. MODERN ENGINEERING
IS A KEY ENABLER FOR
BUSINESS AGILITY
COST OF
INEFFICIENCY
COST OF
DELAY
“In the new world, it is not the big fish
which eats the small fish, it's the fast
fish which eats the slow fish.”
-- Klaus Schwab

9. AGILE
DEVOPS
CLOUD-NATIVE
ARCHITECTURE
VALUE PRIORITISED
DELIVER–REFLECT–IMPROVE
DECOUPLED & DISTRIBUTED
RESILIENT BY DESIGN
AUTOMATED PIPELINES
CROSS-FUNCTIONAL TEAMS
PRODUCT AND CUSTOMER CENTRIC
BUILT-IN QUALITY
DECENTRALISED EXECUTION
ELASTIC SUPPLY OF RESOURCES
M
O
D
E
R
N
E
N
G
I
N
E
E
R
I
N
G

10. AGILE & DEVOPS – BRIEF DEFINITIONS
Agile & DevOps are sets of principles for iterative development, promoting lean processes,
communication and collaboration, adaptation to changes, and continuous improvement, to
speed up the TIME TO MARKET and the CONTINUOUS DELIVERY of business value
Agile & DevOps both focus on nurturing SELF-ORGANISING, CROSS-FUNCTIONAL
teams, with all the skills and capabilities needed to achieve the PRODUCT VISION
IDEATION VALUE
Adapted image from Wikimedia – https://commons.wikimedia.org/wiki/File:Devops-toolchain.svg
Agile focuses on the interactions between the
CUSTOMER (business) and DEVELOPMENT
DevOps focuses on the interactions between
DEVELOPMENT and OPERATIONS

11. AGILE & DEVOPS DRIVE
A NEW WAY OF
WORKING TOGETHER
AS TEAMS

12. BY DOING…
OWNERSHIP & COMMITMENT
BREAKING DOWN SILOS
COLLABORATING
LEARNING TOGETHER
LOOKING FOR SOLUTIONS
SHARING RESPONSIBILITIES
CRAFTMANSHIP

13. CREATIVE THINKING
PROBLEM SOLVING
PRODUCT VISION
CUSTOMER EXPERIENCE
DESIGN & BUILD
CONTINUOUS INTEGRATION
CONTINUOUS TESTING
CONTINUOUS DEPLOYMENT
CONTINUOUS MONITORING
HUMAN + MACHINE

14. IMPROVING
ONE BY ONE

15. BUT ALSO…
IMPROVING
EVERYONE

16. CAN WE REALLY
MANUFACTURE
‘UNICORNS’?
https://en.wikipedia.org/wiki/Unicorn_(finance)
https://www.linkedin.com/pulse/5-signs-youre-unicorn-employee-ryan-holmes/

17. STANDARDIZATION
FAST
PREDICTABLE
FAMILIAR

18. THE THREE AMIGOS

19. PIPELINES
AS
CODE
THE THREE AMIGOS

20. UNIVERSAL
APPLICATION
PACKAGING
THE THREE AMIGOS

21. UNIVERSAL
RUNTIME
PLATFORM
THE THREE AMIGOS

22. K8S – HAVE IT YOUR WAY

23. SKETCH THE
EXEMPLAR
PIPELINE IN
ARCHETYPES
AND TEMPLATES

24. EVERYTHING
TOGETHER
AND IN
VERSION
CONTROL

25. JENKINS
LIBRARIES
TO REUSE
COMMON
TASKS

26. FREEDOM TO
OVERRIDE
WHATEVER
IS NEEDED

27. AND LET THE
PLATFORM
CARE OF
ANYTHING ELSE

28. INCLUDING
MANAGEMENT
OF BUILDER
NODES
(NO NEED TO CARE
ABOUT SLAVES/SWARMS
ANYMORE)

29. STANDARDIZATION
TAKES FORM
OF AN AGREED
PIPELINE SCHEMA
“Behavior-over-tool”

30. WHAT’S IN A CI PIPELINE
ENVIRONMENT
PREPARATION
COMPILE UNIT TESTS MUTATION
TESTS
PACKAGE BUILD DOCKER
IMAGE
RUN DOCKER
IMAGE
INTEGRATION
TESTS
PERFORMANCE
TESTS
DEPENDENCY
VULNERABILITY
TESTS
CODE
INSPECTION
PUSH DOCKER
IMAGE
“THE BRIEF HISTORY OF A COMMIT”

31. • Prepare environment for pipeline execution
• Builder image with basic stuff: JDK, Maven, npm, Node.js,
Python…
• Install all needed dependencies
• Python: pip install
• Node.js: npm install
• Java: Maven, either explicit or along the pipeline
ENVIRONMENT PREPARATION
ENVIRONMENT
PREPARATION
COMPILE UNIT TESTS MUTATION
TESTS
PACKAGE BUILD DOCKER
IMAGE
RUN DOCKER
IMAGE
INTEGRATION
TESTS
PERFORMANCE
TESTS
DEPENDENCY
VULNERABILITY
TESTS
CODE
INSPECTION
PUSH DOCKER
IMAGE

32. • Transform source code to binaries
• In some cases compilation is replaced by other activities like
uglyfication, minification, transpilation…
• For interpreted languages, when viable, check for syntax
errors
COMPILE
ENVIRONMENT
PREPARATION
COMPILE UNIT TESTS MUTATION
TESTS
PACKAGE BUILD DOCKER
IMAGE
RUN DOCKER
IMAGE
INTEGRATION
TESTS
PERFORMANCE
TESTS
DEPENDENCY
VULNERABILITY
TESTS
CODE
INSPECTION
PUSH DOCKER
IMAGE

33. • Run project unit tests
• And never forget: gather code coverage metrics
• Coverage will highlight which code is not being exercized
• “EXERCIZED” does not imply “TESTED”
UNIT TESTS
ENVIRONMENT
PREPARATION
COMPILE UNIT TESTS MUTATION
TESTS
PACKAGE BUILD DOCKER
IMAGE
RUN DOCKER
IMAGE
INTEGRATION
TESTS
PERFORMANCE
TESTS
DEPENDENCY
VULNERABILITY
TESTS
CODE
INSPECTION
PUSH DOCKER
IMAGE

34. • In a nutshell: validates whether your tests are actually testing
anything
• Mutation tests will uncover:
• Untested use cases
• Test cases without assertions
• Also, test cases with insufficient or wrong assertions
MUTATION TESTS
ENVIRONMENT
PREPARATION
COMPILE UNIT TESTS MUTATION
TESTS
PACKAGE BUILD DOCKER
IMAGE
RUN DOCKER
IMAGE
INTEGRATION
TESTS
PERFORMANCE
TESTS
DEPENDENCY
VULNERABILITY
TESTS
CODE
INSPECTION
PUSH DOCKER
IMAGE

35. • Package all application resources:
• Binaries
• Static resources
• Application configuration which does not depend on the environment, e.g.
literals, error codes…
• Environment specific configuration must be externalized (package once run
everywhere)
PACKAGE
ENVIRONMENT
PREPARATION
COMPILE UNIT TESTS MUTATION
TESTS
PACKAGE BUILD DOCKER
IMAGE
RUN DOCKER
IMAGE
INTEGRATION
TESTS
PERFORMANCE
TESTS
DEPENDENCY
VULNERABILITY
TESTS
CODE
INSPECTION
PUSH DOCKER
IMAGE

36. • Put together all pieces which constitute our application
• Base image
• Build artifacts
• Dependencies
• Base images have their own pipeline
• Typically maintained by someone else (vendor, community, other team…)
BUILD DOCKER IMAGE
ENVIRONMENT
PREPARATION
COMPILE UNIT TESTS MUTATION
TESTS
PACKAGE BUILD DOCKER
IMAGE
RUN DOCKER
IMAGE
INTEGRATION
TESTS
PERFORMANCE
TESTS
DEPENDENCY
VULNERABILITY
TESTS
CODE
INSPECTION
PUSH DOCKER
IMAGE

37. • The purpose of this step is to ensure that our application runs
and to make it available for tests
• If application depends on other applications (database,
message queue, other services…) they will be executed all
together (docker compose, application.yml…)
• The test environment is not lasting — will be available only to
tests during the life of the pipelines
• Should allow for concurrent execution (fast feedback on a
continuous integration build-per-commit approach)
RUN DOCKER IMAGE
ENVIRONMENT
PREPARATION
COMPILE UNIT TESTS MUTATION
TESTS
PACKAGE BUILD DOCKER
IMAGE
RUN DOCKER
IMAGE
INTEGRATION
TESTS
PERFORMANCE
TESTS
DEPENDENCY
VULNERABILITY
TESTS
CODE
INSPECTION
PUSH DOCKER
IMAGE

38. • Run tests requiring the application running and its
dependencies
• Integration tests may include: UI tests, API tests…
• Should be user-oriented whenever possible
• Test data preparation is performed at this step too
• For web applications use vulnerability scan agent/proxy
• And remember: gather code coverage metrics
INTEGRATION TESTS
ENVIRONMENT
PREPARATION
COMPILE UNIT TESTS MUTATION
TESTS
PACKAGE BUILD DOCKER
IMAGE
RUN DOCKER
IMAGE
INTEGRATION
TESTS
PERFORMANCE
TESTS
DEPENDENCY
VULNERABILITY
TESTS
CODE
INSPECTION
PUSH DOCKER
IMAGE

39. • Exercise the application under load
• Response times are important but will also uncover errors due
to concurrency usage
• Quality gate should be asserting both metrics
• Should be user-oriented too — reuse integration tests
whenever possible
• Use of dynamic performance monitor agents is also advised
PERFORMANCE TESTS
ENVIRONMENT
PREPARATION
COMPILE UNIT TESTS MUTATION
TESTS
PACKAGE BUILD DOCKER
IMAGE
RUN DOCKER
IMAGE
INTEGRATION
TESTS
PERFORMANCE
TESTS
DEPENDENCY
VULNERABILITY
TESTS
CODE
INSPECTION
PUSH DOCKER
IMAGE

40. • Assessing application security is not negotiable
• But usually we forget that most vulnerabilities, if not all, come
from external dependencies
• Some tools for this step include:
• OWASP Dependency Check
• Snyk
• Black Duck
• Nexus Lifecycle
• PyUp Safety
DEPENDENCY VULNERABILITY TESTS
ENVIRONMENT
PREPARATION
COMPILE UNIT TESTS MUTATION
TESTS
PACKAGE BUILD DOCKER
IMAGE
RUN DOCKER
IMAGE
INTEGRATION
TESTS
PERFORMANCE
TESTS
DEPENDENCY
VULNERABILITY
TESTS
CODE
INSPECTION
PUSH DOCKER
IMAGE

41. • Static code analysis:
• Adherence to coding standards
• Possible bugs
• Code smells and best practices
• Security vulnerabilities
• Documentation and complexity metrics
• Code duplication
• This step usually happens at the same time that the overall quality
gate (SonarQube)
• Ideally this should happen earlier in the pipeline — however we
assume that this is happening before code is pushed and/or
merged
CODE INSPECTION
ENVIRONMENT
PREPARATION
COMPILE UNIT TESTS MUTATION
TESTS
PACKAGE BUILD DOCKER
IMAGE
RUN DOCKER
IMAGE
INTEGRATION
TESTS
PERFORMANCE
TESTS
DEPENDENCY
VULNERABILITY
TESTS
CODE
INSPECTION
PUSH DOCKER
IMAGE

42. • After the final quality gate, if all pass, the image is pushed to a
shared registry
• Once pushed it is available to be used by:
• Other applications that depends on this image
• To be promoted to a stage environment
• All the way up to production
PUSH DOCKER IMAGE
ENVIRONMENT
PREPARATION
COMPILE UNIT TESTS MUTATION
TESTS
PACKAGE BUILD DOCKER
IMAGE
RUN DOCKER
IMAGE
INTEGRATION
TESTS
PERFORMANCE
TESTS
DEPENDENCY
VULNERABILITY
TESTS
CODE
INSPECTION
PUSH DOCKER
IMAGE

43. GOT ANY
QUESTIONS?
LET’S TALK!