A presentation originally intended for internal learning purposes, it is an introduction to DevOps practices and CI/CD pipelines, with an example of our current implementation and an optimal one.

1. DEVOPS INTRODUCTION
AHMED ABDULAZIZ

2. AGENDA
D e v O p s
C I / C D
To o l s O v e r v i e w
O u r S e t u p
O p t i m a l S e t u p
2

3. DEVOPS
W H A T , W H Y A N D H O W
3

4. WHAT IS DEVOPS?
4
D e v O p s i s a s e t o f p ra c t i c e s t h a t
c o m b i n e s s o f t w a re d e v e l o p m e n t
( D e v ) a n d I T o p e ra t i o n s ( O p s ) .
I t a i m s t o s h o r t e n t h e sy s t e m s
d e v e l o p m e n t l i fe c y c l e a n d
p ro v i d e c o n t i n u o u s d e l i v e r y w i t h
h i g h s o f t w a re q u a l i t y.
S i m p l y e x p l a i n e d , D e v O p s ’s ro l e
i n t h e p ro j e c t s t a r t s j u s t a f t e r
t h e c o d e l a n d s fo r m t h e
d e v e l o p e r ’s l a p t o p a l l t h e w ay t o
t h e p ro d u c t i o n d e p l o y m e n t .

5. WHAT IS DEVOPS?
A merge between
the software
development
team, the testing
team and the
operations team.
Intertwined with
Agile
methodologies of
continuously
releasing
software.
Automation,
automation and
more automation
is the backbone
of DevOps.
5

6. DEVOPS CULTURE
• The DevOps team consists of people who
don’t identify as a software developer, system
administrator, testing engineer, security
engineer, or even application support
engineers, although they might come from
either backgrounds.
• DevOps culture dictates that the DevOps
team is formed of small, multidisciplinary
teams, who work autonomously and take
collective accountability for how actual users
experience their software.
6
• As we can see all of the above engineers can be DevOps ones.
Although they come from different backgrounds their
accountability will be a collective one.
• In reality, this is not always applied, and the DevOps role can
just be concerned with automated operations engineering.

7. WHY DEVOPS?
Higher Speed of development,
faster delivery of fixes and
features and faster reaction.
7
Reliability is improved due to
automatic enforcement of
quality standards.
Scalability is easier to perform
with automated deployments
and cluster management.
Higher Security as different
security standards are
automated (DevSecOps).
Collaboration in the team is
improved with automated
alerts and messaging.
Revenue is increased with less
time to market and by saving
developers time.

8. CI/CD
C O N T I N U O U S I N T E G R A T I O N
A N D D E P L O Y M E N T
8

9. WHAT IS CI/CD
CI (Continuous
Integration) is the
practice of merging
all developers'
working copies to a
shared repo
frequently and
continuously.
CD (Continuous
Deployment/Delivery)
is the practice of
delivering software
frequently and
continuously through
automated
deployments.
The difference
between Deployment
and Delivery is the
former deploys on
production while the
latter delivers to
pre-production (UAT,
dev, test and so on)
9

10. CI/CD PIPELINE
10
A C I / C D p i p e l i n e i s a s e r i e s o f s t e p s
t h a t i s p e r f o r m e d t o d e l i v e r a n e w
v e r s i o n o f s o f t w a r e .
T h e m i n i m u m f u l l C I / C D p i p e l i n e i s :
B u i l d , Te s t , R e l e a s e ( U p l o a d ) ,
D e p l o y, Va l i d a t e .
T h e m o r e D e v O p s a n d A g i l e p r a c t i c e s
a r e a p p l i e d t h e m o r e r o b u s t a n d
m a t u r e t h e p i p e l i n e w i l l h a v e t o b e .

11. TOOLS
OVERVIEW
T O O L S T O S U C C E E D W I T H D E V O P S
11

12. I T ’ S T H E A R C H I T E C T ’ S D E C I S I O N
I will try to cover at least a tool for each part of a
DevOps environment. However note that there will
always be alternatives that can be used as well.

13. COLLABORATION
Communication
tools include
Slack, Microsoft
Team, Zoom.
App life cycle
management
there is JIRA,
Trello and TFS
and more.
Docs sharing
tools include
Confluence,
XWiki, GitHub
page and OpenAPI

14. SOURCE CONTROL
• Source control refers to tracking and managing changes
to code. This ensures that developers are always working
on the right version of source code.
• Enables easier collaborations, it’s not specific to
developers and is used by many non-software oriented
users to keep track of things like shared document.
• The most famous one is Git, alternatives include TFS
(Team Foundation Server) for Microsoft based
environments, Mercurial and Apache Subversion.
• Popular products that support git includes GitHub, Gitlab,
Bitbucket and Gitea (pronounced Git + tea ).
• Most of them can support either cloud or self-hosted
servers except for Bitbucket which Atlassian ended their
support for self-hosted version.
14

15. BUILD
• Building tools can vary greatly between each and every language,
framework and environment. They can be package managers too.
• They mostly compile the code to some sort of a packaged version.
• For backend java maven is used, maven is more than a build tool
but in this context it works as one.
• For frontend npm or yarn are used or a cli provided by the
frontend framework (that builds over npm or yarn).
• For android gradle is used.
• For iOS Xcode is used on a MacOS machine.
• Other tools exist for other environments too like PIP for python or
rbenv for Ruby
15

16. VIRTUALIZATION
16
V i r t u a l i z a t i o n i s t h e c re a t i o n o f a
v i r t u a l e n v i ro n m e n t a s o p p o s e d t o
a p hy s i c a l e n v i ro n m e n t .
L i ke c o m p u t e r - g e n e ra t e d v e rs i o n s
o f h a rd w a re , o p e ra t i n g sy s t e m s ,
s t o ra g e d e v i c e s , a n d m o re .
A l l o w s p a r t i t i o n i n g a s i n g l e
m a c h i n e i n t o d i f fe re n t
e n v i ro n m e n t s , i n d e p e n d e n t o f
e a c h o t h e r.
Po p u l a r v i r t u a l i z a t i o n t o o l s a re
V M Wa re , O ra c l e ’s V i r t u a l B ox ,
M i c ro s o f t H y p e r - V a n d m o re .

17. CONTAINERS
• Containerization is a form of operating system, in which
applications are run in isolated spaces called containers,
all using the same shared operating system.
• Containers are useful as they allow us to easily package
any application and isolate it from the underlying system
so we can get the same results from the application on
any environment, without the headache of VMs.
• Docker is the popular choice, there are alternatives like
LXC (the 1st container tech), podman, containerd and RKT.
• Vagrant blurs the line between containers and VMs, it
provisions VMs in a similar manner to containers but
uses VM technology.
• The difference between containers and virtual machines
is that containers use a shared kernel of the underlying
OS while VMs each have an independent OS.
17

18. TEST
• Test tools differs widely just like build tools based on
different tests and different environments.
• Unit testing tools include Junit/TestNG for Java,
Jasmine/Jest/Karma for JavaScript and more.
• Integration testing depends a lot on the environment,
unit tests tools can be used without mocking with
additional configuration for the required test
environment.
• E2E testing will depend on how the system is built, if it is
a UI test we can use Selenium or Cucumber or tools
based on the frontend tech like Protractor for Angular for
API tests we can use SOAP-UI, Insomnia, Postman.
• Load testing can be done with JMeter, Apache ab,
Gatling, bombardier or other tools.
• Static application security testing (SAST) tools analyze the
application’s source code to uncover security issues. SAST
tools include SonarQube, DeepSource, Semgrep and
Gitlab plus linting tools like PMD, Checkstyle or ES/TSLint.
• Dynamic application security testing (DAST) tools test the
application by running it and interacting with it. DAST
tools include OWASP ZAP, Veracode and more.
• Tests are used in the pipeline mostly after building
especially to do regression and smoke tests to ensure the
build is ready. For continuous deployment, this is a must.
18

19. CI
• CI tools are the maestro of the whole process (i.e.
pipeline) like building and testing until deployment is
handed to CD. In a cloud native environment, CI is of
extreme important and practically indispensable tool.
• The CI products are countless, the most famous ones
include Jenkins, Circle CI, Gitlab, Bitbucket, GitHub, Travis
CI, Bitrise, AWS CodePipeline, Azure Pipelines and more.
• Jenkins popularity is due to it being open source and free
of charge plus being extremely powerful and old.
• However, Jenkins is more complicated and harder to
setup and would only be recommended if we need a
completely free product.
19

20. CONFIGURATION
MANAGEMENT
20
I t re fe rs t o t h e p ro c e s s i n
w h i c h t h e e n v i ro n m e n t i s
c o n f i g u re d . I t s t o o l s i n s t a l l
s o f t w a re p a c ka g e s a n d m o d i f y
t h e e n v i ro n m e n t t o m a ke i t
re a d y fo r d e p l o y m e n t .
N o t t o b e c o n f u s e d w i t h
a p p l i c a t i o n c o n f i g ( e . g . s p r i n g
p ro p e r t i e s f i l e ) .
T h e m o s t fa m o u s i s A n s i b l e
t h e n P u p p e t , C h e f a n d S a l t .
A n s i b l e i s p re fe r re d d u e t o
g o o d s u p p o r t a n d e a s e o f u s e .

21. INFRASTRUCTURE
AS CODE
21
I a C ( I n f ra s t r u c t u re a s a C o d e )
t o o l s d e f i n e c l o u d i n f ra s t r u c t u re
( e . g . c l o u d s e r v e r i n s t a n c e s ) a s
c o d e a s i n f i l e c o n f i g u ra t i o n s .
To o l s fo r t h i s i n c l u d e Te r ra fo r m ,
AW S C l o u d F o r m a t i o n a n d a c t u a l l y
c o n f i g u ra t i o n m a n a g e m e n t t o o l s
c a n a l l q u a l i f y a s I a C t o o l s .

22. ARTIFACT REPO
22
A r t i fa c t s a re p a c ka g e s t h a t a re
c re a t e d t h ro u g h o u t t h e
d e v e l o p m e n t p ro c e s s . A n a r t i fa c t
re p o s i t o r y i s a n
a p p l i c a t i o n d e s i g n e d t o s t o re a n d
m a n a g e t h e s e a r t i fa c t s .
I t c a n s t o re e x t e r n a l o r i n t e r n a l
l i b ra r i e s , d o c ke r i m a g e s , XC o d e
p o d s a n d m o re .
I t s t o o l s i n c l u d e N e x u s , N P M ,
M a v e n , A rc h i v a , J F ro g A r t i fa c t o r y
a n d D o c ke r H u b / D o c ke r Re g i s t r y.

23. CONTAINER ORCHESTRATION
• It is the automation of deployment, management,
scaling, and networking of containers.
• It allows a small team of DevOps specialists to manage
thousands of containers distributed worldwide.
• It saw an extremely sharp rise in popularity with the
advent of cloud computing and microservices
architecture software.
• Kubernetes (K8s) is the most famous and the best choice
here. There is also Docker Swarm, HashiCorp Nomad,
Apache Mesos and much more.
• For K8s there is a lot of lighter implementations like K3S
(Kubes), MicroK8s and more. Rancher also uses K8s.
• In the Cloud there is AWS EKS, Google Cloud GKE, Azure
AKS and more.
• On prem is not simple harder as cloud vendors hide bare
metal setup from users, a famous tool to support K8s on
prem is OpenStack from Red Hat.
• Recently, both Google and Amazon announced new
products that’s like GKE and EKS but is made for on prem.
• There is also Rancher which is a great tool as well and can
actually be better for smaller clusters. Other tools include
Kubespray (collection of ansible scripts).
• For service meshes that governs service-to-service
comms with K8s there is Istio, Linkerd and Consul.
23

24. (CONTINUOUS) DEPLOYMENT
• The following tools are used to enable different types of
deployment for both containers and normal apps.
• The tool used will depend a lot on the environment, there
is Gitlab CD, Octopus Deploy, Spinnaker and even SSH.
• Note that CI and CD overlap, it’s up to the architect to
decide whether a specific step should be in CI or CD.
24

25. LOGS MANAGEMENT AND MONITORING
Logs Management
• Mostly logs management tools aim to unify, centralize
and provide helpful tools to search and process logs for
easier troubleshooting.
• The most famous tech for log management is the ELK
stack (ElasticSearch, LogStash and Kibana) there is also
DataDog and Splunk and different variations of ELK stack.
Monitoring
• Monitoring tools aim to keep constant track of
application performance and other required metrics.
• There is Prometheus to collect the monitoring data and
Grafana to visualize it, there is also Zipkin, AppDynamics
and New Relic.
25

26. OUR SETUP
F R O N T E N D , B A C K E N D A N D A N D R O I D
26

27. OVERVIEW
Frontend pipeline
is for Angular
with extra steps
for optimization.
Backend pipeline
is for Java. It has
a build and a
docker push or to
deploy a file.
Android build is
easy but Android
SDKs setup is
complex, so
docker was used.
27

28. FRONTEND
28
Push
Developer Web
hook
Bitbucket
Start pipeline job
Jenkins
Install
packages
If tag or
develop or
master
branch
Code
quality
check
Build, profile based
on tag and branch
CSS Purge
If develop
or master
Fonts
purging
Deploy by
SSH
connection
Post build
actions
pipeline
Always
Send email
of job
status
On Success
upload tag
builds to
Nextcloud

29. BACKEND: SPRING BOOT AND DOCKER
29
Push
Developer Web
hook
Bitbucket Start
pipeline
Jenkins Build Maven
Build Docker
image if master
branch
Upload Image
to Nexus
Run the project
in a container
with Docker
Post action: on
success archive
the .jar file

30. MOBILE: ANDROID
30
Git Tag Created and
Pushed
Developer starts
Jenkins tag build
Pipeline is started in a docker container
(image: “thyrlian/android-sdk”)
From Jenkin’s secrets add
KeyStore.jks to project folder
Build apk
with Gradle
Upload apk
to Nextcloud
On Success send an
email with job status

31. OPTIMAL
SETUP
A F U L L D E V O P S S E T U P
31

32. AGILE & MICROSERVICES
Agile
• Agile methodologies is a must for a well functioning
DevOps culture.
• Planning, designing and developing software in an Agile
environment where the Agile guidelines are followed
correctly will lead to many DevOps practices.
Microservices
• To achieve an actual well-built distributed Microservices
architecture, you must use DevOps or at least a large part
of what constitutes a DevOps or a correctly created
CI/CD.
32

34. HOW TO IMPROVE?
Creating Unit
tests and
integration tests
UI tests or API
tests is probably
the best way we
can improve our
own pipeline.
As we mostly deal
with on-prem
customers with
small clusters we
can use Ansible
with K8s (using
K3s) to deploy
the small cluster.
Documentation
should be
enforced a lot
more and design
decisions should
be planned
beforehand and
enforced as well.
34

35. THANK YOU
A h m e d A b d u l - A z i z + 2 0 1 1 1 4 7 3 4 7 2 3 a h m e d . a b d u l a z i z @ s e g m a t e k . c o m