My presentation on DevOps at TestausOSY session, presenting DevOps maturity model and Continuous integration analysis tool.

1. 6/20/2017 Knowit 1

2. Gartner Hype Cycle for Cloud Computing 2012
6/20/2017 Knowit 2

3. Gartners Hype Cycle for Application Development, 2016
6/20/2017 Knowit 3

4. 6/20/2017 Knowit 4

5. DevOps and other terminology
6/20/2017 Knowit 5
Continuous Integration
Continuous Delivery
Continuous Deployment
DevOps
Prepare Implement Build QA Deliver Deploy Operate
Continuous feedback
Agile development

6. Typical steps to DevOps
6/20/2017 Knowit 6
Responsiveness
Agile methods in use
in development
scrum, backlog etc
Automation
deployed in larger
scale. Daily build
with automated
regression tests
Ability to deliver daily
SW quality metrics
support daily
activities
Changes deployed
automatically to
production
environment as soon
as available from
production pipe
Continuous
Integration
Continuous
delivery
Continuous
deployment
DevOps
Operational metrics
transparent, feature
throughput in days,
Processes fast and
lean, continuous
learning and
improvement
Automation in
product level,
automated
regression tests
Test Automation
Quality
Agile development

7. 6/20/2017 Knowit 7

8. DevOps, faster cycles and visibility
Traditionally you need months for the cycle
Prepare Implement Build QA Deliver Deploy Operate

9. months
DevOps, faster cycles and visibility
Prepare Implement Build QA Deliver Deploy Operate
Measure
AnalyseOptimise
Prepare Implement Build QA Deliver Deploy Operate

10. months
weeks
hours
DevOps, faster cycles and visibility
Speed demands quality
Prepare Implement Build QA Deliver Deploy Operate
Prepare Implement Build QA Deliver Deploy Operate

11. Typical steps to DevOps
6/20/2017 Knowit 11
Responsiveness
Agile methods in use
in development
scrum, backlog etc
Automation
deployed in larger
scale. Daily build
with automated
regression tests
Ability to deliver daily
SW quality metrics
support daily
activities
Changes deployed
automatically to
production
environment as soon
as available from
production pipe
Continuous
Integration
Continuous
delivery
Continuous
deployment
DevOps
Operational metrics
transparent, feature
throughput in days,
Processes fast and
lean, continuous
learning and
improvement
Automation in
product level,
automated
regression tests
Test Automation
Quality
Agile development

12. DevOps development example blocks
Requirement mgmt
Planning
Customer data mgmt
Release &
configuration mgmt
architechture
Implementation
Agile methods Scrum,
backlog
Unit tests, static
analysis
Accessrights +
versioncontrol
Testing
Test case mgmt
Delivery Deployment
Customer data
mgmt
Production
Fault mgmt
Usage metrics

13. DevOps development example blocks
Requirement mgmt
Planning
Customer data mgmt
Release &
configuration mgmt
architechture
Implementation
Agile methods Scrum,
backlog
Technical debt
visualisation e.G.
SonarQube
Continuous integration
/ jenkins etc
Unit tests, static
analysis
Accessrights +
versioncontrol
Testing
Test case mgmt
Regression test
automation
Delivery Deployment
Customer data
mgmt
Production
Fault mgmt
Usage metrics

14. Backend service
virtualisation
virtualisointi
DevOps development example blocks
Requirement mgmt
Planning
Customer data mgmt
Release &
configuration mgmt
Implementation
Agile methods Scrum,
backlog
Technical debt
visualisation e.G.
SonarQube
Continuous integration
/ jenkins etc
Unit tests, static
analysis
Accessrights +
versioncontrol
Virtualisation/ cloud
environments
Testing
Test case mgmt
Regression test
automation
Test data management
Installation automation
Delivery
Automated transfer
Käyttö
Käyttömittaristo
Vikaseuranta
Deployment
Customer data
mgmt
Automated
update/upgrade
Production
Fault mgmt
Usage metrics
architechture

15. Käyttö
Käyttömittaristo
VikaseurantaRequirement mgmt
Planning
Customer data mgmt
architechture
Release &
configuration mgmt
Implementation
Agile methods Scrum,
backlog
Technical debt
visualisation e.G.
SonarQube
Continuous integration
/ jenkins etc
Unit tests, static
analysis
Accessrights +
versioncontrol
Virtualisation/ cloud
environments
Testing
Test case mgmt
Regression test
automation
Test data management
Installation automation
Virtualisation/ cloud
environments incl
backend services
Full test automation
Optimised transfer
Delivery
Automated transfer
Deployment
Customer data
mgmt
Cloud environment
(Docker, Ansible
etc)
Automated
update/upgrade
Production
Fault mgmt
Usage metrics
DevOps development example blocks

16. Optimised transfer
DevOps development example blocks
Requirement mgmt
DeploymentImplementation
Agile methods Scrum,
backlog
Planning
Technical debt
visualisation e.G.
SonarQube
Continuous integration
/ jenkins etc
Unit tests, static
analysis
Testing
Test case mgmtAccessrights +
versioncontrol
Regression test
automation
Test data management
Installation automation
Delivery
Automated transfer
Microservices
Customer data mgmt
architechture
Customer data
mgmt
Optimised update/
Microservices
Production
Fault mgmt
Usage metrics
Cloud environment
(Docker, Ansible
etc)
Virtualisation/ cloud
environments
Virtualisation/ cloud
environments incl
backend services
Optimised
cloud
environments
Automated
update/upgrade
Transparency /optimisation / KPI’s
Organisation / featureteams
Full test automation
Release &
configuration mgmt

17. What DevOps means for the pipe?
6/20/2017 Knowit 17

18. Visualisation of SW development pipe
SW Development
- SW development
-Unit test
-Module test
-Static analysis & security analysis
-etc
SW Building
-Adding SW
modules/versions to a
build
-Static analysis
-etc
Install /Commissioning
-Reservation
-SW download
-SWBT /sanity test
-etc
QA / testing
-Regression set
-Entity testing
-Functional
-performance
-security
-Network Verification
Time Time Time Time
Deliver
- internal / external
- Delivery logic (pull vs push)
- optimized methods
Time
Deploy
-Installation / phased
-Update/upgrade
-Fallback
Time
Project SW Development SW Building Install /
Commissioning
QA / Testing Deliver Deploy
Org1 Team1
Org1 Team2
Org2 Team1
Org2 Team3
Full automation Partly automated Under planning Not reasonable to
automate
Manual
6/20/2017 Knowit

19. Early fault finding - Financial benefit of a Shift Left - example
6/20/2017 Knowit 19
Based on real example product calculation
Cost of one single fault discovered by:
Development € 310
Functional tests € 1,500
Acceptance tests € 10,000
Customer € 15,000
A simple improvement to find two faults which should
have been found in unit test but were found on system
integration and acceptance test can save 10k€.
Requirements /
specification
Reviews
/static analysis unit test
system
integration
feature test
/regression
acceptance
tests customer
Requirements /
specification 2 1 0 5 6 4 0
Reviews /static
analysis 0 5 15 2 1 1 1
unit test 0 0 20 2 3 1 2
system integration 0 0 0 3 2 1 0
feature test
/regression 0 0 0 2 15 3 1
acceptance tests 10 1
total 2 6 35 14 27 20 5
cost € 620 1860 10850 21000 40500 200000 75000 349830
Requirements /
specification
Reviews
/static analysis unit test
system
integration
feature test
/regression
acceptance
tests customer
Requirements /
specification 2 1 0 5 6 4 0
Reviews /static
analysis 0 5 17 1 1 0 1
unit test 0 0 20 2 3 1 2
system integration 0 0 0 3 2 1 0
feature test
/regression 0 0 0 2 15 3 1
acceptance tests 10 1
total 2 6 37 13 27 19 5
cost € 620 1860 11470 19500 40500 190000 75000 338950
defect found in
Should
have
been
found
defect found in
Should
have
been
found
Faults not found in proper phase but leaked to next

20. Implement Build QA / Test Deliver Deploy Operate
Example view of devops (”The pipe”) overview
20.6.2017 [CONFIDENTIAL] KNOWIT PROPOSAL
• CI pipe dashboard: Visualizing the progress of each SW build, lava lamp view
of each phase.
• Jenkins jobs provide information of each phase.
• Jenkins is the main umbrella and scheduler, additional tools applied within each
phase, as necessary.
Implement Build QA / Test Deliver Deploy Operate
Build #n+1
Build #n
PAGE 20

21. Some things are feasible still to execute manually
• Architecture
• Actual coding, including refactoring&optimisation
• Test planning
• Risk evalution
• Test optimisation (until Artificial Intelligence replaces this, the next hype ...)
• Usability tests
• Exploratory tests (though using scripts and automation as helper tool)
6/20/2017 Knowit 21
But wait, can you automate everything?

22. DevOps principles according to DASA,
DevopsAgileSkillsAssociation
6/20/2017 Knowit 22
Customer-
centric
action
Create
with end in
mind
Automate
everything you
can
End-to-End
responsibility
Cross-
functional
autonomous
teams
Continuous
improvement

23. [CONFIDENTIAL] THIS IS KNOWIT
PAGE
23
Devops maturity 1 2 3 4
Organisation and culture Planning, Development
and quality assurance in
own silos
Agile methods in use in
planning&development, QA in
separate organisation
Feature team is responsible all the way to
production
Autonomous teams in both
feature and platform level
Continuous Delivery readiness (Dev &
Build & Test & Deliver)
No CI or only some
phases covered
CI in use, typical feedback loop over
a week.
CI pipeline fully automatic, includes all the
tests, cycle time max couple of days,
green build available for delivery
CI pipeline cycle less than 12
hours, test coverage and other
quality criteria meets the delivery
quality criteria
Fast deployment to production Deployment cycle
typically months
Requirement analysis and impact to
SW components takes weeks.
Development does the build ready
for acceptance tests.
Requirement analysis and impact to whole
product can be analysed in few hours.
Acceptance tests are automated,
Releasing/deployment to production done
manually.
Also deployment automatic,
when critical failure in update,
automatic fallback happans and
feedback to development
delivered.
Visibility and reporting No visibility, missing or
manual reporting
Reporting of each phase exists, but
visibility only to that certain phase.
Test coverage and static code analysis
results visible for whole development
Product operative metrics visible
for developers, in case of
problems automatic alert to
development
Architechture Architecture unclear, lot of
surprises when something
changed
Product is one own entity, installed
as a whole.
Product consists of subentities which
interfaces are known and defined.
Microservices in use and
independently updateable
Product releasing and sales (pull vs
push)
No controlled feature or
configuration
management
Release content defined in advance
80%. Customer pays when release
acceptance got and in production in
customer system.
Release content half fixed half adapting to
order base. Product/update can be
ordered via net.
Features done on demand, First
order and payment, then
development and delivery.
BDD/ATDD from customer
needs. Ordering and payment via
internet.

24. Thank You
Contact info:
Jouni Jätyri
+358 40 511 6013
jouni.jatyri@knowit.fi
6/20/2017 Knowit 24