Slides from my talk on Codemotion Spain 28th November 2015

2. Scaling up CI to a CD Pipeline
BUILD TEST DEPLOY

3. ➔ Social Casino Startup
➔ Web and Mobile
➔ 250K DAU
➔ ~20 Technical Staff
➔ No QA
@Plumbee_Games
http://www.plumbee.com
Engineering Manager at Plumbee Ltd
@mserranom
miguel@plumbee.com

4. Scaling up CI to a CD Pipeline
BUILD TEST DEPLOY

5. Mission + People
BUILD TEST DEPLOY
Any organisation’s pipeline is shaped by:
● The requirements to achieve its mission
● The views of the team on Software Development

6. Decisions
● Tools
● Test Strategy
● Frequency on deployment
● On-call strategy
BUILD TEST DEPLOY

7. it’s all about building an
Engineering Culture
BUILD TEST DEPLOY

8. Trunk vs Branching

9. Trunk vs Branching
● Fast ● Safe

10. Trunk vs Branching
● Fast
● Minimises merge pain
● Safe
● Asynchronous

11. Trunk vs Branching
● Fast
● Minimises merge pain
● Old Fashioned
● Safe
● Asynchronous
● Git is cool!

12. Trunk
“The most direct way to improve as a
software developer is to be absolutely fearless
when it comes to changing your code”
Jeff Atwood
Stack Exchange/Discourse founder

13. Trunk
“Don’t be scared of changing your code.
Pave your way to not to break things”
Miguel Serrano
The guy talking right now

14. Versioning
JAR
1.0-SNAPSHOT
WAR
1.4-SNAPSHOT
JAR
1.4-SNAPSHOT

15. JAR
1.0-SNAPSHOT
WAR
1.4-SNAPSHOT
JAR
1.4-SNAPSHOT
Versioning
Nondeterministic!
The exact sequence of builds is not reproducible

16. JAR
1.0-SNAPSHOT
WAR
1.4-SNAPSHOT
JAR
1.4-SNAPSHOT
Versioning
Releases require:
● Update snapshots to static version number
● Ensure there are no snapshot dependencies
● Code freeze while releasing
● Manual updates
JAR
1.0
WAR
1.4
JAR
1.4

17. JAR
1.0-SNAPSHOT
WAR
1.4-SNAPSHOT
JAR
1.4-SNAPSHOT
Versioning
Releases require:
● Update snapshots to a static number
● Ensure there are no snapshot dependencies
● Code freeze while releasing
● Manual updates
}
mvn release:prepare
mvn release:perform
JAR
1.0
WAR
1.4
JAR
1.4

18. Versioning
Rules
● Every artifact is assigned a new version at build time
● Every artifact gets its dependencies versions updated to
the latest version in every build
● Every artifact is ready to be deployed
JAR
1.0
WAR
1.4
JAR
1.4

19. JAR
1.0-SNAPSHOT
WAR
1.4-SNAPSHOT
JAR
1.4-SNAPSHOT
Versioning

20. JAR
JAR
JAR
JAR
JAR
WAR
Test
DEPLOY
JAR
JAR WAR DEPLOY

21. JAR
JAR
JAR
JAR
JAR
WAR
Test
DEPLOY
JAR
JAR WAR DEPLOY
Actual Pipeline ~= 154 builds

22. Pipeline Graph
C
E
D
A
StagingTestB
A developer makes a change in module A

23. Pipeline Graph
B
C
E
D
A
StagingTest

24. Pipeline Graph
B
C
E
D
A
StagingTest

25. Pipeline Graph
B
C
E
D
A
StagingTest
#1
#2

26. Pipeline Graph
B
C
E
D
A
StagingTest
x2
x3 x3
● 7 Modules, 12 Builds
● 1 changeset triggers 3 deploys
#Builds(n) = #UniquePaths(A->n)

27. Build Explosion!
#Builds grow exponentially when the graph grows
...but computing power is cheap (linux)

28. Build Explosion!
#Builds grow exponentially when the graph grows
...but computing power is cheap (linux)
Option: ignore intermediate results
… while wasting resources
Test
#1
#2
#3
the only one with
changeset applied to all
nodes

29. #Builds
B
C
D
C
A
WAR Test
● Join: Build doesn’t start until all the previous one have
succeeded
● #Build = #Modules
● Test runs once per changeset

30. Smart pipelines in Jenkins
● Build Flow Plugin
○ https://github.com/jenkinsci/build-flow-plugin
● Workflow Plugin
○ https://github.com/jenkinsci/workflow-plugin

31. Build Flow Plugin
join = parallel ({
build(“B”)
build(“C”),
})
build(“D”, param1 : join )
B
C
D

32. BUILD TEST STAGING PRODUCTION
Engineering
Business
+ Engineering
Continuous Delivery
BUILD TEST DEPLOY

33. STAGING PRODUCTION
Continuous Delivery
Schema Update
IT Provisioning
Config updates
Roll out
Non-Technical

34. BUILD TEST STAGING PRODUCTION
Continuous Delivery
BUILD TEST PRODUCTION
Continuous Deployment

35. BUILD TEST
PRODUCTION
DEPLOY
RELEASE
Release and Deploy are decoupled
● Changes in Production are incremental -> Less risky
● Bug Fixes are the sole responsibility of the Engineers
Engineering Business
Continuous Deployment

36. BUILD TEST
PRODUCTION
DEPLOY
RELEASE
Feature Toggles allow decoupling Release from Deployment
Also helpful for:
● Canary Releases (release features to subset of users)
● A/B Tests
Engineering Business
Continuous Deployment

37. BUILD TEST PRODUCTION
Continuous Deployment
Limitations:
➔ Legal & Regulatory
➔ Distribution (App Stores!)
➔ Not suitable for critical software

38. Q&A