Transform Agile Development With Practical DevOps

Presented By – Daffodil DevOps Team
1
Transform Agile
Development
With
Practical DevOps
@2018 Daffodil Software Pvt. Ltd.
All trademarks and logo displayed in this presentation are the trademarks of their respective owners and used for informational purposes only. We constitute neither an endorsement nor
a recommendation of those organizations

21
2
3
5
4
6
7
DevOps
Introduction
Understanding
DevOps Lifecycle
DevOps, Automation &
Best Practices
Docker Containers &
Ecosystem
DevOps With
Cloud Services
DevOps
Toolchain for
Best Practices
Infrastructure as
a Code (IaC)

3
Will Also See
Some Tools
in Action

1. DevOps
Introduction
4

5
Tools Process
DevOps
CultureSkill
People
Speed
What is DevOps

6
Release
Deploy
Monitor
Feedback
Code
Review
Build
Test
Plan
Functional Perspective of Dev and Ops Teams
Agile
Development
Dev Ops
Operations

What is DevOps 7
Is a Culture 1
Is about implementing
best practices with the
help of tools
2
Is about
implementing
agility in
Operations
3
Is About
Automation
4

8
Not a Role,
Person or
Organization
Not
Something
only IT
Administrator
do
Not
Something
only
Developer do
Not about
Writing Scripts
or Implementing
Tools only
What is NOT DevOps

9
Why Agile is
Incomplete
Without DevOps ?

10
How I can look into
the system if
everything is
running well ?
Why I am not
getting the
desired result
from Agile Team?
How do we
achieve faster
time to market
with Agile team?
How I can keep my
Production Server
Secure and Stable
with frequent
releases ?

11
Every Team is Happily Working in Silos With Different
Objectives
Customer Business Owner Agile Dev Team OpsTeam
Business
Revenue
Target
Well
Structure
Code
Stable and
Secure
Infra

12
Agile Seems Fine Without DevOps Culture Till
Deployment Comes Into The Picture
My Code is working fine.
We will share the build
with OpsTeam soon ?
My Environment is secure
and ready for production.
I will deploy whatever Dev
team will give us ?
Are we ready to
deploy new
features ?
Yes, Our Agile Team is
all set to deploy new
features.
I have confirmed
to Client that we
are ready to
deploy new
features ?

13
Deployment By Team Working in Silos
Build Handover Between
Teams (Working in Silos) for
Deployment
Leads To
Disaster
Dev Team Ops Team
I don’t know
anything about Prod
Env but I assume it
must be working as
good as my code
I don’t know anything
about Code but I
assume it must be
working as good as my
Prod Env

14
Hunting for The Defaulter Begins With Client’s Escalation
Do you know what might
have caused the failure ?
How would I know,
you wrote the code ?
Do you know what
might have
caused the failure
?
Let me check with the
team. I will update you
about it.
Team, We have to
resolve the issue
ASAP ?

15
Lack of Transparency and Coordination Kick Off the
Blame Game and Left Customer Furious
Our Code was working
fine before it was
deployed in your
environment ?
My Servers were
working well before
you made the new
deployment ?
My business is
getting affected, I
need the
explanation now ?
Entire team is looking
into the issue. I will
update you about it.
Client is now
really frustrated
and I don’t have
any answers ?

16
Agile Development WITHOUT DevOps Culture
Release and Deploy Speed
Mismatch
Unpredictable Issues
Blame Games Lack Of Monitoring and Feedback

17
Agile Development WITH DevOps Culture
Streamlined Deliveries Team Works in Collaboration
Continuous Monitoring & Feedback

2. Understanding
DevOps Lifecycle
18

19
Agile Development
Continuous Integration
Continuous Delivery
Continuous Deployment
DevOps

20
Difference between Continuous Integration,
Continuous Delivery and Continuous Deployment
Build Test
Deploy to
Staging
Acceptance
Test
Deploy to
Production
Smoke
Test
Build Test
Deploy to
Staging
Acceptance
Test
Deploy to
Production
Smoke
Test
Manual
Automated
Continuous
Delivery
Continuous
Deployment
Continuous
Integration

3. Automation &
Best Practices
21

22
We Can Start DevOps Adoption With a 5 Steps Journey
Process
Transformation
2
Cultural
Transformation
1
Technology
Transformation
3
Automate
Everything
4
Adoption of Tools
For Best Practices
5

23
How DevOps Disrupt Project Execution Methodologies ?
Design Code Test Deploy
Design
Code
Test
Design
Code
Test Deploy CodeDesign Test Deploy CodeDesign Test Deploy
Waterfall
Agile
DevOps
DeployDesign TestCode Design TestCode DeployDesign TestCode

24
Reactive and Scalable Infrastructure with Embrace DevOps
Waterfall
Agile
DevOps
Architecture Packaging Infrastructure
Monolithic
N-Tier Decoupled
Microservices
Physical Server InHouse Datacenter
VM
Containers
Hosted
Cloud

25
Culture
CI/CD
Continuous
Feedback
Continuous
Monitoring
Process Cronjobs
System
Scheduling
Build
Test
Release
Deploy
Configure
Monitor
DevOps IT Automation
Automation
in DevOps
Custom Shell
Scripts

26
Monitoring
Build Process
Provisioning
Testing
Deployment
01
02 03
0504
Automate Provisioning -
Infrastructure as Code
Automate Builds –
Continuous Integration
Automate Deployments –
Defined Deployment Pipeline
Automate Testing –
Continuous Testing,
Automated tests after each
deployment
Automate Monitoring &
Metrics – Proper monitors
in place sending alerts
Automate as Many as Processes to Support Faster Deliveries

27
Take Proactive Measure using Continuous Monitoring
Application
Monitoring
Hardware
Monitoring
Cloud
Monitoring
Mobile
Apps
Monitoring
Infrastructure
Monitoring
Logs
Monitoring

29
Improve Quality and Stability by Continuous Feedback

4. DevOps Toolchain
for Best Practices
30

31
Adoption of Tools to Implement Best Practices for
DevOps

32
PROD UAT DEV
Continuous
Delivery
SM DEV 1 DEV 2
CheckInChanges
Fetch Changes
QA
Service
Desk
Monitoring
Feedback
CI/CD
Server
Send To
Backlog
Agile Team
Version Control
Adoption of Tools to Implement Best Practices for DevOps

33
CI/CD
With

34Jenkins for CI & CD

35
Jenkins Dashboard

36
Jenkins Dashboard

37
Monitoring
With

38Zabbix for Continuous Monitoring

39
Monitoring
With

40NetData for RealTime Monitoring

5. Docker Containers
& Ecosystem
41

43
A classical problem in shipping industry was
“How to Transport different goods
having different size, type, shape and
requirement ? ”

44
Packaging Lifting Transportation Management
Liquid, Eatables Electronics Furniture Automobiles
A standard container which is loaded
with virtually any type of goods and
stay sealed until it reaches its
destination
A standard container which is loaded
with virtually any type of goods and
stay sealed until it reaches its
destination

45
Linux
Containers

46
• A modern software system comprises many parts, including binaries,
libraries, configuration files, and dependencies.
• It's hard enough to assemble such diverse components on a single
machine, but it's dramatically more complex when you "ship" that
software.
• Once you ship your software, you need to figure out a way to package
all these things together and put them where they need to run.
“ Container makes it easy to package software, along
with all its dependencies, and ship it to the developer
across the room, to staging or production, or
wherever it needs to run. “

47
Docker
Overview

48
Docker is world's leading container
engine that automates the
deployment of any application as a
lightweight, portable, self-sufficient
container that will run virtually
anywhere.
Developed by "Solomon Hykes"
Initiated in 13 March 2013
Promoted by the company Docker, Inc.

49
Docker is written in Go and takes
advantage of several features of
the Linux kernel to deliver its
functionality.

50
Docker uses following set of technologies
of linux and convert them into a wrapper
known as Container format:
• Namespaces
• Control Groups (CG)
• Union File System (UFS)

51

52

53

54
Why Need Of
Container
Arise in Web Development ?

55
OS
Application - 1
Server - 1
High Procurement Lead Time
Upfront Capex
Ongoing Opex
Under Utilized Servers
Drawbacks of Traditional One Server per Application Approach
Server - 1

56
Guest OS 1
Application -
1
VM - 1
Guest OS 2
Application -
1
VM - 2
Guest OS 3
Application -
1
VM - 3
Server - 1
Hypervisor
Big Overhead on Server
Each VM divides Server Resources
Each OS uses RAM, CPU and Storage
VM Size is Quiet Big to Port
Drawbacks of Multi Applications Per Server with
Multiple VMs Approach
Server - 1
Host OS

57
Application - 1 Application - 2 Application - 3
Server - 1
OS
Very less Overhead on Server
Each container shares the same OS
Very less storage is required by each
container
Fast to launch & port
Benefits of Multi Applications Per Server with Multiple
VMs Approach
Docker Engine

58
Guest OS 1
Application -
1
VM - 1
Guest OS 2
Application -
1
VM - 2
Guest OS 3
Application -
1
VM - 3
Server - 1
Hypervisor
Server - 1
Host OS
Server - 1
OS
Docker Engine
Comparison of Hypervisor vs Container Engine

59
Guest OS 1
Application -
1
VM - 1
Guest OS 2
Application -
1
VM - 2
Guest OS 3
Application -
1
VM - 3
Server - 1
Hypervisor
Server - 1
Host OS
Server - 1
OS
Docker Engine
Comparison of Hypervisor vs Container Engine
64 GB RAM
8 GB RAM
8 GB RAM
8 x 3 GB
RAM
8 GB
EACH
16 GB
EACH

60
Guest OS 1
Applicati
on - 1
VM - 1
Guest OS 2
Applicati
on - 1
VM - 2
Server - 1
Hypervisor
Server - 1
Host OS
Container & VM Together
Docker Engine
Applicati
on - 1
Applicati
on - 1
Docker Engine

61
Quick Comparison of Container vs VM
VS
Integration
Size
Migration Speed
Startup

62
How Containers
Simplify
Application Deployment

63
Dev System InHouse Server VM Cloud VM
Dev QA UAT / Staging Production
Frontend
Components
Backend
Components
Database
Components
Libraries and 3rd Party
Components

64
Dev System InHouse Server VM Cloud VM
Dev QA UAT / Staging Production
Frontend
Components
Backend
Components
Database
Components
Libraries and 3rd Party
Components

65
Docker
Architecture

66
1
Docker
Host
2
Docker Engine /
Daemon
3
Docker
Client
4
Docker
Registry
5
Docker
File
6
Docker
Image
Docker
Terminologies
7
Docker
Container

67
Docker Client
Docker Daemon
Docker Host Linux
Docker Client
Docker Host
OS X
Linux
Docker Daemon
Docker host is a machine that
runs the Docker Engine

68
Docker Engine is the
Brain
Behind the Whole Operation

69
Docker Engine run as a back ground
process known as Docker Daemon and
performs all activities like
How to talk to the kernel,
Makes the system calls to create,
Operate and Manage containers,
which we as a user of docker don’t have to
worry about

70
Docker Client is a
Utility
To run any Docker
Commands

71
When you run any docker command
to perform any action on file, Image
or container, your Docker Client will
Translate that command into http
API call and sends
it to Docker Daemon

72
Docker File is like
Source Code
Of Docker Image

73
How to Create a Docker File
Create a new directory and a new and empty dockerfile inside
that directory.
> mkdir ~/myimages
> cd myimages/
> touch Dockerfile
> vi Dockerfile

74
Keywords of Docker File
FROM:The base image for building a new image. This command must be on top of the
dockerfile.
MAINTAINER: Optional, contains the name of the maintainer of the image..
RUN: Used to execute a command during the build process of the docker image.
ADD: Copy a file from the host machine to the new docker image. There is an option to
use an URL for the file, docker will then download that file to the destination directory.
ENV: Define an environment variable.
CMD: Used for executing commands when we build a new container from the docker
image.
ENTRYPOINT: Define the default command that will be executed when the container
is running.
WORKDIR: This is directive for CMD command to be executed.
USER: Set the user or UID for the container created with the image.
VOLUME: Enable access/linked directory between the container and the host machine.
LABEL: The LABEL instruction adds metadata to an image in a key-value pair

75
Write instructions in DockerFile
#Download base image ubuntu 16.04
FROM ubuntu:16.04.
#Add name of Image owner
MAINTAINER Gaurav Sharma (gaurav.sharma@daffodilsw.com)
# Update Ubuntu Software repository
RUN apt-get update
# install NGINX and PHP FPM
RUN apt-get install -y nginx php7.0-fpm

76
# Volume configuration
VOLUME ["/etc/nginx/sites-enabled", "/etc/nginx/certs",
"/etc/nginx/conf.d", "/var/log/nginx", "/var/www/html"]
#Define the ENV variable
ENV nginx_vhost /etc/nginx/sites-available/default
ENV php_conf /etc/php/7.0/fpm/php.ini
ENV nginx_conf /etc/nginx/nginx.conf

77
# Volume configuration
VOLUME ["/etc/nginx/sites-enabled", "/etc/nginx/certs",
"/etc/nginx/conf.d", "/var/log/nginx", "/var/www/html"]
# Configure Services and Port
COPY start.sh /start.sh
CMD ["./start.sh"]
# Open the port for HTTP and HTTPS
EXPOSE 80 443
# Add LABEL
LABEL description="Excentus API Server for Dev"

78
Docker Image is
Compiled Version
Of Docker File

79
It is a lightweight, stand-alone,
executable package of a piece of
software that includes everything
needed to run it:
Code, Runtime, System tools, System
libraries, Settings

80
Docker File
Each Docker File instruction generates a
new image layer
From ubuntu:latest # Layer 0
Docker Image Layers
Layer 3 (Image 3) – 91e54dfb1179 (1.925 KB)
Layer 2 (Image 2) – d74508fb6632 (1.895 KB)
Layer 1 (Image 1) – c22013c84729 (194.5 KB)
Layer 0 (Base Image) – d3a1f33e8a5a (188.1 MB)
Maintainer GauravSharma # Layer 1
RUN apt-get update # Layer 2
RUN apt-get install –y curl # Layer 3
Layer 4 (Image 3) – 71jn5kefb1179 (0 KB)

81
Docker Container Layer
Layer 3 (Image 3) – 91e54dfb1179 (1.925 KB)
Layer 2 (Image 2) – d74508fb6632 (1.895 KB)
Layer 1 (Image 1) – c22013c84729 (194.5 KB)
Layer 0 (Base Image) – d3a1f33e8a5a (188.1 MB)
Layer 4 (Image 3) – 71jn5kefb1179 (0 KB)
Thin R/W Layer Container Layer (Writable)
Image Layer (Read Only)
Container

82
Docker Registry is an image store,
holding named Docker images,
available in different tagged versions.
Sample image repository: distribution/registry, with tags 2.0
and 2.1
Users can interact with a registry by using docker push and
pull commands.
> docker pull https://hub.docker.com/distribution/registry:2.1

83
DockerHub

84
Docker Container are running
instances of Docker images. If
image is class file then container
can be considered an instance of
image

85

86
Docker
Ecosystem

87
Docker is available in two editions:
• Community Edition (CE) and
• Enterprise Edition (EE).
• Enterprise Edition Basic
• Enterprise Edition Standard
• Enterprise Edition Advanced

89
Docker
Installation on
Ubuntu Machine

90
Install DockerCE on Ubuntu
To install Docker CE, you need the 64-bit version of one of these Ubuntu
versions:
• Zesty 17.04
• Xenial 16.04 (LTS)
• Trusty 14.04 (LTS)
Docker CE is supported on Ubuntu on x86_64, armhf, and s390x (IBM z
Systems) architectures.

91
Verify Repository list on Ubuntu (Prior to 17.03)
To Install Docker Prior to Ubuntu 17.03
> sudo apt-get install –y docker.io
To Install Docker on Ubuntu 17.03 from
https://download.docker.com/linux/ubuntu
> sudo apt-get install docker-ce=<VERSION>

92
Docker on
Windows

93
• Docker for Windows is a Docker Community Edition (CE) app. The
Docker for Windows install package includes everything you need to
run Docker on a Windows system.
• Virtualization must be enabled. Typically, virtualization is enabled by
default. (Note that this is different from having Hyper-V enabled.) For
more detail see Virtualization must be enabled in Troubleshooting.
• The current version of Docker for Windows runs on 64bit Windows
10 Pro, Enterprise and Education (1607 Anniversary Update, Build
14393 or later).
• The installation provides Docker Engine, Docker CLI client, Docker
Compose, Docker Machine, and Kitematic.

94

95
Docker
Commands

96

97

98
View list of all Docker Images
To view list of all images present inside
your Docker Host
> docker image ls --all

99

100
View list of all Docker Containers
To view list of all containers present
inside your Docker Host
> docker container ls --all

101

102
Build a Docker Image
To build Docker Image from a dockerfile and
assign a tag to it
> docker build -t nginx_image .

103
Pull a Docker Image From Registry
To pull an existing image from Docker
Registry
> docker pull nginx

104
Launch a Docker Container
Launch a new container from existing
image.
> docker run -i -p 80:80 nginx /bin/bash

105
Various Other Docker Container
Commands
# To Delete a Container using Name
> docker rm docker-nginx
# To Stop a Container using Name
> docker stop docker-nginx
# To Start an existing stopped Container using Name
> docker start docker-nginx

106
Docker Container Lifecycle

107
Design Your Containerized Environment

Container
Management
Introduction
108

109

110
Kubernetes UI

111
Portainer UI

6. Infrastructure
As a Code (IaC)
112

113
IaC
Overview

114

115

7. DevOps
Using Cloud
Services
116

117

118
AWS
CodePipeline

119
AWS Infrastructure
Automation

120

121
Google
Container Engine

What NEXT
122

123
Here What You Can Jump in NEXT
 DevOps Best Practices
 Integration Pattern & Use Cases
 DevOps with IaC
 Automation in All Stages of DevOps

Thank You
Info Email: info@daffodilsw.com
Presenter Email: gaurav.sharma@daffodilsw.com
Website: www.daffodilsw.com
124

Transform Agile Development With Practical DevOps

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to Transform Agile Development With Practical DevOps

Similar to Transform Agile Development With Practical DevOps (20)

Recently uploaded

Recently uploaded (20)

Transform Agile Development With Practical DevOps