1. CONTINUOUS INTEGRATION

2. TRAINING GOALS
Show that there is no way
present-day project could be
successful without usage of
continuous integration practice
Establish connection between
CMMI product integration
process area and continuous
integration practice
2

3. TRAINING PLAN
1. What is continuous integration?
2. Why do we need continuous integration?
3. Prerequisites for continuous integration process
4. General CI workflow
5. How does continuous integration affect our
development process?
6. Tools and their features
7. When CI is not effective?
8. We have "true CI". What next?
9. CI and CMMI product integration process area
3

4. INTRODUCTION TO CONTINUOUS
INTEGRATION
4 Basic ideas

5. WHAT IS CONTINUOUS
INTEGRATION?
5

6. WHAT IS CONTINUOUS
INTEGRATION?
• Integration is when you make everything work together
• Continuous is when you make everything work together very often 6
• You would go mad if you had to do it manually
• That‟s why it is not about manual actions, it is about automation

7. WHAT IS CONTINUOUS
INTEGRATION?
Software
engineering
practice
Another step
to the Approach
development helping to
process reduce risks
automation
7

8. WHY DO WE NEED CONTINUOUS
INTEGRATION?
 Martin Fowler:
… team integrate their work frequently,
… leading to multiple integrations per day.
… this approach leads to significantly reduced integration
problems and allows a team to
develop cohesive software more rapidly.
 CI is the one of the XP practices
 Allows to have fresh latest build
8

9. PREREQUISITES FOR CONTINUOUS
INTEGRATION
Not everything you can continuously integrate
9

10. PREREQUISITES FOR CONTINUOUS
INTEGRATION
Single source
code repo
(VCS system)
Build
CI tool
automation
Separate Self-testing
server app (unit- 10
(usually) tests)

11. GENERAL CI WORKFLOW
Build
Committing (compilation, unit-
latest testing, db
changes integration, etc)
Update by Application is
scheduler (on ready
CI server)
11

12. GENERAL CI WORKFLOW
changes
detected
build &
update deploy
commit
~
repository deployment server
continuous integration server
12

13. GENERAL CI WORKFLOW
Check
modification
Build
application
13

14. TOOLS
14 Instruments for continuous integration and their
features

15. TOOLS CLASSIFICATION
CI tools
Build management tools
Unit testing tools
Inspection tools
• Code coverage analysis
• Static analysis (syntax check, code
dependencies)
• Copy/paste detectors
15
Documentation generation tools

16. CI TOOLS
 CruiseControl
 CruiseControl.NET
 CruiseControl.rb
 JetBrains TeamCity
 Apache Hudson
 Apache Continuum
 Atlassian Bamboo
 FinalBuilder
 phpUnderControl
16
 XInc

17. CI TOOLS
 CruiseControl
 CruiseControl.NET
 CruiseControl.rb
 JetBrains TeamCity
 Apache Hudson
Most common, flexible, configurable, easy to
 Apache Continuum
use, de facto choice for Java projects
 Atlassian Bamboo
 FinalBuilder
 phpUnderControl
17
 XInc

18. CI TOOLS
 CruiseControl
 CruiseControl.NET
 CruiseControl.rb
 JetBrains TeamCity
 Apache Hudson
Most common, flexible, configurable, easy to
 Apache Continuum
use, de facto choice for .NET projects
 Atlassian Bamboo
 FinalBuilder
 phpUnderControl
18
 XInc

19. CI TOOLS
 CruiseControl
 CruiseControl.NET
 CruiseControl.rb
 JetBrains TeamCity
 Apache Hudson
Most successful adaptation of CI principle for the
 Apache Continuum
interpreted language (Ruby)
 Atlassian Bamboo
 FinalBuilder
 phpUnderControl
19
 XInc

20. CI TOOLS
 CruiseControl
 CruiseControl.NET
 CruiseControl.rb
 JetBrains TeamCity
 Apache Hudson
One of the best CI tools:
 Apache Continuum
• pre-tested commit
• build agents  Atlassian Bamboo
• multi-platform builds
 FinalBuilder
• build dependencies
 phpUnderControl
• comprehensive statistics and reporting 20
 XInc

21. CI TOOLS
 CruiseControl
 CruiseControl.NET
 CruiseControl.rb
Tagging after successful build, distributed builds
 JetBrains TeamCity
 Apache Hudson
 Apache Continuum
 Atlassian Bamboo
 FinalBuilder
 phpUnderControl
21
 XInc

22. CI TOOLS
 CruiseControl
 CruiseControl.NET
 CruiseControl.rb
Distributed builds
 JetBrains TeamCity
 Apache Hudson
 Apache Continuum
 Atlassian Bamboo
 FinalBuilder
 phpUnderControl
22
 XInc

23. CI TOOLS
 CruiseControl
 CruiseControl.NET
 CruiseControl.rb
Focused on processJetBrains TeamCity just a feature
 of building, CI is
 Apache Hudson
 Apache Continuum
 Atlassian Bamboo
 FinalBuilder
 phpUnderControl
23
 XInc

24. CI TOOLS
 CruiseControl
 CruiseControl.NET
 CruiseControl.rb
CI tools written in PHP for PHP projects
 JetBrains TeamCity
 Apache Hudson
 Apache Continuum
 Atlassian Bamboo
 FinalBuilder
 phpUnderControl
24
 XInc

25. CI TOOLS
 CruiseControl
 CruiseControl.NET
 CruiseControl.rb
Yet another CI tool written in TeamCity PHP projects
 JetBrains PHP for
 Apache Hudson
 Apache Continuum
 Atlassian Bamboo
 FinalBuilder
 phpUnderControl
25
 XInc

26. CI TOOLS
 Too many tools
 All are great
 All have the same principle
 But each has specific features
Diagrams, metrics, reporting Integration with SCM tools Notification
Interface, usability Distributed builds Different ALM approaches
26

27. CI TOOLS. ALM APPROACH
Requirements
Initiation development & Design
management
Testing Integration Implementation
Maintenance &
Deployment Utilization 27
support

28. CI TOOLS. ALM APPROACH
Requirements
Initiation development & Design
management
Basic continuous integration
Testing Integration Implementation
Maintenance &
Deployment Utilization 28
support

29. CI TOOLS. ALM APPROACH
Requirements
Initiation development & Design
management
Basic continuous integration
Testing Integration Implementation
Extended build
management
Maintenance &
Deployment Utilization 29
support

30. BUILD TOOLS
 Ant
 NAnt
 MSBuild
 Maven
 Make
 Phing
 Rake
 …
30

31. BUILD TOOLS
 Ant
 NAnt
 MSBuild
 Maven
Most common, flexible, configurable, easy to
 Make
use, de facto choice for Java projects
 Phing
 Rake
 …
31

32. BUILD TOOLS
 Ant
 NAnt
 MSBuild
 Maven
Most common, flexible, configurable, easy to
 Make
use, de facto choice for .NET projects
 Phing
 Rake
 …
32

33. BUILD TOOLS
 Ant
 NAnt
 MSBuild
 Maven
 Make
de facto case for Visual Studio and TFS users
 Phing
 Rake
 …
33

34. BUILD TOOLS
 Ant
Build management + dependencies management
 NAnt
for Java projects. Introduced POM concept.
 MSBuild
 Maven
 Make
 Phing
 Rake
 …
34

35. BUILD TOOLS
 Ant
Standard de facto for UNIX applications
 NAnt
 MSBuild
 Maven
 Make
 Phing
 Rake
 …
35

36. BUILD TOOLS
 Ant
Implementation of build management tool in PHP
 NAnt
for PHP projects
 MSBuild
 Maven
 Make
 Phing
 Rake
 …
36

37. BUILD TOOLS
 Ant
Implementation of build management tool in Ruby
 NAnt
for Ruby projects
 MSBuild
 Maven
 Make
 Phing
 Rake
 …
37

38. UNIT TESTING TOOLS
 Junit
 NUnit
 CppUnit
 PHPUnit
 SimpleTest
 JSUnit
 J3Unit 38

39. INSPECTIONS TOOLS.
TEST COVERAGE
 Cobertura
 Atlassian Clover
 jTest
 JCoverage
 CodeCover
 EMMA
 Parasoft Insure++
 Ncover
 Xdebug
39
 Coverage.py

40. INSPECTIONS TOOLS.
TEST COVERAGE
 Cobertura
 Atlassian Clover
 jTest
 JCoverage
Java
 CodeCover
 EMMA
 Parasoft Insure++ C++
 Ncover C#
 Xdebug PHP
40
 Coverage.py Python

41. INSPECTIONS TOOLS.
STATIC ANALYSIS
 PMD
 FindBugs
 JLint
 FxCop
 Checkstyle
 ReSharper
 PHP_CodeSniffer
 Yasca
41
 NDepend

42. INSPECTIONS TOOLS.
COPY/PASTE DETECTORS
CPD (Copy paste detector)
Checkstyle
Simian
Jplag
Atomiq
Clone digger
PBA 42

43. DOCUMENTATION GENERATION
TOOLS
Doxygen
JavaDoc
NDoc
CppDoc
phpDocumentor
pyDoc
RDoc 43

44. OTHER TOOLS
Source code Dead code
metrics (loc) detectors
Profiling …
44

45. CI IN EPAM MICROSOFT
TECHNOLOGIES DIVISION
Project Project Product Primary CCNET Code Analysis Unit Test Copy & Automat Automate
Language tools testing coverage Paste e build deploymen
tools detectors delivery t
ACT-INT C# x - x (Unit) - - - -
ACT-LVI C# x x (FxCop, PBA) x - - - -
CON-GEN C#, ASP.NET - - - - - - -
EPM-UTIL C#, ASP.NET x x (FxCop,PBA) x (NUnit) x (NCover) x (Simian) + +
ESS-MSFT - - - - - - -
FSTIDX C#, ASP.NET x x (PBA) x - - - -
ICPDP C# x - - - - - -
LOGIDEX Java, J#, C#, ASP.NET - - - - - - -
LSEEC C# x x (FxCop, PBA) x (NUnit) - x (Simian) - +
MIS-OPC C# x x x - - - -
MIS-SPR C# x x x (Nunit) x - - -
MultAPI VBScript - - - - - - -
MultCons C# x x (FxCop, PBA) - - x (Simian) - -
MultData C#, C++, HTML, VB x - x - - - -
MultEDG C++, C# - x (FxCop) x - - x -
Mult-IPAS C#, C++ x x (FxCop) - - - - -
MultRIN C# - - x - - - x
Mult-Torn C++ - - x - - - -

46. CI IN EPAM MICROSOFT
TECHNOLOGIES DIVISION
Project Project Product Primary CCNET Code Analysis Unit Test Copy & Automat Automate
Language tools testing coverage Paste e build deploymen
tools detectors delivery t
RTRS-AAAM C# - - - - - - -
RTRS-MTST C#, ASP.NET - x (FxCop) - - - - -
RTRS-RKSD C# x x - - x (Simian) - -
RTRS-RKWM C# - x - - - - -
TAK-PBLD C# - x (FxCop) x (Nunit) - - - -
TRR-ODC (O2I) C#, ASP.NET x x (FxCop,PBA) x (Nunit) - x (Simian) - -
TRR-ODC (SapSn) C# x x (FxCop,PBA) - - x (Simian) - -
46

47. MOST TYPICAL QUESTIONS
 All the above-described info sounds cool. I
feel that it might be useful in my project. But I
am not sure I can find enough resources to start
and support this activity.
 EPM-BET project is started long time ago for this
purpose. It provides corresponding resources and
support
 There are too many different tools. They are
all different. How could I use them both
independently and all at once?
 I recommend to use AgileSCM approach. It uses
47
unified versions numbering which could be used by
any mentioned tool.

48. ADVANCED CONTINUOUS
INTEGRATION
48 More complex issues related to the continuous
integration

49. HOW DOES CI AFFECT
DEVELOPMENT PROCESS?
 Commit policy (mainlines are on the watch)
 Don't break the build rule (make local build)
 Everyone commits to the mainline everyday
 Keeping the build fast
 Run fast tests first
 We can automate deployment
 Build is primary, deployment is secondary
 Continuous design
49

50. “DON’T BREAK THE BUILD”
RULE
 Test code properly before
checking in
 Avoid depending on a local
resource that is not under
version control or does not
exist on the target computer.
 Write tests to cover common
problems
 Make sure local
inspections were run
successfully
 Broken build in most cases
means that it is impossible or
not recommended to use
corresponding codebase until
it has been fixed 50

51. “DON’T BREAK THE BUILD”
OBSESSION
 It is often impossible to
reproduce integration
problem without checking in.
 You might spend A LOT of
time looking for a failure reason
in someone‟s else module
 During that time you will be the
one WHO FAILED THE BUILD
 If you want to avoid such
situations, every checking in
becomes a nightmare
 Attitude to the “don‟t break the
build” rule may be a detector of
how mature the team is.
51

52. “DON’T BREAK THE BUILD”
OBSESSION. SOLUTION
 Nightly builds instead of
builds „on commit‟
 Team needs more strict and
less strict variations of the
“don‟t break the build” rule
during different SDLC
phases
 There are architectural
issues to pay attention to
 Sometimes several small
projects should be started
instead of one large
52
 Build policy is to be
developed

53. WHEN CI IS NOT EFFECTIVE?
 DVCS
 Experimental development
 Small projects
 Interpreted languages without unit-tests
 When database is being changed too often
 Types of projects
 Documentation (BA, etc)
 DB
 Support & maintenance 53

54. WE HAVE "TRUE CI".
WHAT'S NEXT?
 Building tags
 Separate builds having different maturity 
levels 
 Codebase structure filtering
 Triggering build only for changes in
specified list of folders
 Integration with issue tracking systems
 Different approach for different SDLC
54
phases

55. WE HAVE "TRUE CI".
WHAT’S NEXT?
1.x.x 2.x.x
/trunk
PA 1.x.0 1.x.3 2.x.0
A 1.x.1 1.x.4 2.x.1
builds
B 1.x.2 1.x.5 2.x.2
/1.x.x
AR 1.0.0
BR 1.0.1
RC 1.0.2 1.0.3 releases
ST 1.0.4
56
/1.0.x

56. WE HAVE "TRUE CI".
WHAT’S NEXT?
1.x.x 2.x.x
/trunk
1.x.0.0 1.x.0.1 1.x.1.0 1.x.1.1 1.x.2.0 2.x.x.0 2.x.x.1 2.x.x.2 2.x.x.3 2.x.x.4 … 2.x.0.0 2.x.0.1 2.x.1.0 2.x.1.1
integration (dev)
2.x.0 2.x.1 2.x.2 builds
CI server
1.x.0 1.x.1 1.x.2 1.x.3 1.x.4 1.x.5 builds
1.x.2.1 … 1.x.3.0 1.x.3.1 1.x.4.0 1.x.4.1 1.x.4.2 1.x.4.3 integration (dev)
/1.x.x
CI server
releases
1.0.0 1.0.1 1.0.2 1.0.3 1.0.4
integration (rel) 1.0.x.0 1.0.0.0 1.0.0.1 1.0.1.0 1.0.1.1 1.0.2.0 1.0.2.1 1.0.3.0 1.0.3.1
57
/1.0.x

57. WHAT NEXT? VERSIONS
NUMBERING PATTERNS
N.x.x.L pilot integration
integration
N.x.K.L development integration
N.?.?.L
N.M.x.L pre-release integration
N.M.K.L release integration
2.0.x.0 2.0.x.1
2.0.0
2.x.x.0 2.x.x.1 2.x.x.2 2.x.x.3 2.x.x.4
2.x.0 …
… 1.x.1.0 1.x.1.1
1.x.2 1.1.x.0
1.1.0
1.x.2.0 1.x.2.1
1.x.3 1.x.3.0 1.x.3.1
1.x.4
dev rel
58
N.x N.M 1.0.x.0
1.0.0 1.0.0.0 1.0.0.1
1.0.1

58. 59

59. CONCLUSION
 Thereis a lot of possibilities to make your
continuous integration better:
 Several integration streamlines (not only
trunk)
 Development and release integration
streamlines
 Introduce version number consisting of four
symbols
 Proper version number inheritance
 Integration of builds and releases
 …

60. CONCLUSION
 CI is another SCM tool (practice)
 Middle size project should definitely
have it.
 It is used mostly in agile world
 Agile SCM is the result of attempt to
understand what version should have
artifacts which are being built by CI
server

61. AFTERWORD
62

62. RECOMMENDED READING
1. Continuous Integration: Improving Software Quality and
Reducing Risk by Paul M. Duvall, Steve Matyas, Andrew
Glover
63

63. RECOMMENDED READING
2. Continuous Delivery: Reliable Software Releases through
Build, Test, and Deployment Automation by By Jez
Humble, David Farley
64

64. RECOMMENDED READING
3. Release It!: Design and Deploy Production-Ready
Software by Michael T. Nygard
65

65. RECOMMENDED READING
4. Pragmatic Project Automation: How to Build, Deploy, and
Monitor Java Applications by Mark Clark
66

66. USEFUL LINKS
 http://martinfowler.com/articles/continuousIntegration.ht
ml - the main article about CI from Martin Fowler
 http://www.infoq.com/news/2009/03/Continuous-
Deployment - Beyond Continuous Integration:
Continuous Deployment
 http://radar.oreilly.com/2009/03/continuous-deployment-
5-eas.html - Continuous Deployment in 5 easy steps
 http://www.proudlyserving.com/archives/2007/10/fear_of
_a_broke.html - „fear of the broken build‟ effect
 http://lib.custis.ru/Continuous_Integration - wiki about
Continuous Integration in Russian
 http://habrahabr.ru/blogs/php/91777/ - Providing quality
for web applications (rus)
 http://www.youbrokethebuild.com/ - great posters
created with the purpose of motivation people avoid
breaking the build 67