1. JFokus 2012 Stockholm, Feb 14th 2012 #jfokus
Jenkins Evolutions
Toomas Römer
ZeroTurnaround
Tartu, Estonia, 500km
@toomasr
http://dow.ngra.de/

2. 2

3. JFokus 2012 Stockholm, Feb 14th 2012 #jfokus
Why Evolutions?
Testing

4. JFokus 2012 Stockholm, Feb 14th 2012 #jfokus
Why Evolutions?
Continuous Integration

5. JFokus 2012 Stockholm, Feb 14th 2012 #jfokus
Why Evolutions?
Continuous Deployment

6. JFokus 2012 Stockholm, Feb 14th 2012 #jfokus
Why Evolutions?
Workflow management
– Trigger jobs
– Join jobs
Building software
Testing builds
Propagating DVCS repositories
Updating website
– Changelog
– Documentation
– Nightly builds
Release management

7. JFokus 2012 Stockholm, Feb 14th 2012 #jfokus
Why Evolutions?
Authentication
Authorization
Logging
Notifications
Remote machines

8. JFokus 2012 Stockholm, Feb 14th 2012 #jfokus
Why Evolutions?

9. JFokus 2012 Stockholm, Feb 14th 2012 #jfokus
Why Evolutions?
Cron on Steroids

10. JFokus 2012 Stockholm, Feb 14th 2012 #jfokus
Why Evolutions?
Unopinionated build tool
(via @lusis)

11. JFokus 2012 Stockholm, Feb 14th 2012 #jfokus
Agenda
Background
Stories
– Functional Tests
– Servlet Tests
– Random Thoughts

12. JFokus 2012 Stockholm, Feb 14th 2012 #jfokus
Background - 2008
First installation
Clicked “Run” when wanted to test stuff

13. JFokus 2012 Stockholm, Feb 14th 2012 #jfokus
Background - 2012
200 300 jobs
5 8 dedicated machines
Amazon ~7000 instance-hour per month
3 in house plugins developed, 2
abandoned
Custom dashboard (35 x 215 grid)

14. JFokus 2012 Stockholm, Feb 14th 2012 #jfokus
14

15. JFokus 2012 Stockholm, Feb 14th 2012 #jfokus
15

16. JFokus 2012 Stockholm, Feb 14th 2012 #jfokus
https://github.com/jenkinsci/jenkins-
scripts
15

17. JFokus 2012 Stockholm, Feb 14th 2012 #jfokus
Custom Dashboard

18. JFokus 2012 Stockholm, Feb 14th 2012 #jfokus
Story of Functional Tests

19. JFokus 2012 Stockholm, Feb 14th 2012 #jfokus
JRebel Functional JDK Tests

20. JFokus 2012 Stockholm, Feb 14th 2012 #jfokus
JRebel Functional JDK Tests

21. JFokus 2012 Stockholm, Feb 14th 2012 #jfokus
JRebel Functional JDK Tests
Per JDK
– Per functional test
• Compile couple of classes
• Run some methods and assert
• Recompile couple of classes
• Run some methods and assert
• …

22. JFokus 2012 Stockholm, Feb 14th 2012 #jfokus
JRebel Functional JDK Tests
Which Jenkins feature is the best fit here?

23. JFokus 2012 Stockholm, Feb 14th 2012 #jfokus
JRebel Functional JDK Tests

24. JFokus 2012 Stockholm, Feb 14th 2012 #jfokus
JRebel Functional JDK Tests
Groovy Script

25. JFokus 2012 Stockholm, Feb 14th 2012 #jfokus
JRebel Functional JDK Tests

26. JFokus 2012 Stockholm, Feb 14th 2012 #jfokus
JRebel Functional JDK Tests
How to get test results? Jenkins feature…

27. JFokus 2012 Stockholm, Feb 14th 2012 #jfokus
JRebel Functional JDK Tests
Output JUnit XML!

28. JFokus 2012 Stockholm, Feb 14th 2012 #jfokus
JRebel Functional JDK Tests

29. JFokus 2012 Stockholm, Feb 14th 2012 #jfokus
JRebel Functional JDK Tests
Creates about 10 000 new files
Compile about 1700 Java class files
Invoke the JVM 550 times

30. JFokus 2012 Stockholm, Feb 14th 2012 #jfokus
JRebel Functional JDK Tests
Sloooow

31. JFokus 2012 Stockholm, Feb 14th 2012 #jfokus
JRebel Functional JDK Tests
IO
CPU
Memory

32. JFokus 2012 Stockholm, Feb 14th 2012 #jfokus
JRebel Functional JDK Tests

33. JFokus 2012 Stockholm, Feb 14th 2012 #jfokus
JRebel Functional JDK Tests
Bare metal
Cloud
Bare metal

34. JFokus 2012 Stockholm, Feb 14th 2012 #jfokus
JRebel Functional JDK Tests
Cloud
– 1 small instance per JDK
– Manual update of AMIs
– Slow
– Scales!

35. JFokus 2012 Stockholm, Feb 14th 2012 #jfokus
JRebel Functional JDK Tests
Which Jenkins plugin to use?

36. JFokus 2012 Stockholm, Feb 14th 2012 #jfokus
JRebel Functional JDK Tests
Amazon EC2 Plugin

37. JFokus 2012 Stockholm, Feb 14th 2012 #jfokus
JRebel Functional JDK Tests
Which Jenkins plugin to use for results?

38. JFokus 2012 Stockholm, Feb 14th 2012 #jfokus
JRebel Functional JDK Tests
Bare metal
– 8 core & 16GB RAM gaming machines
– Recycled developer machines
– Marketing demo machines (iMacs)

39. JFokus 2012 Stockholm, Feb 14th 2012 #jfokus
JRebel Functional JDK Tests
SSD
– 30% gain in time, that’s it

40. JFokus 2012 Stockholm, Feb 14th 2012 #jfokus
Story of Servlet Tests

41. JFokus 2012 Stockholm, Feb 14th 2012 #jfokus
Servlet tests
Start a container version
– Websphere, Weblogic, Glassfish etc.
Deploy applications
Test results, reload changes, test results…

42. JFokus 2012 Stockholm, Feb 14th 2012 #jfokus
Servlet tests
How do you start a
server?

43. JFokus 2012 Stockholm, Feb 14th 2012 #jfokus
Servlet tests
Duh! With a startup
script!

44. JFokus 2012 Stockholm, Feb 14th 2012 #jfokus
Servlet Tests – 1 app server, 2 jobs
Which Jenkins feature to use?

45. JFokus 2012 Stockholm, Feb 14th 2012 #jfokus
Servlet Tests – 1 app server, 2 jobs

46. JFokus 2012 Stockholm, Feb 14th 2012 #jfokus
Servlet Tests – 1 app server, 2 jobs

47. JFokus 2012 Stockholm, Feb 14th 2012 #jfokus
Servlet Tests

48. JFokus 2012 Stockholm, Feb 14th 2012 #jfokus
Servlet Tests
How do you deploy an
application?

49. JFokus 2012 Stockholm, Feb 14th 2012 #jfokus
Servlet Tests
Start Tomcat
Copy WAR files to webapps/
GET /some-url
Assert
Update application
GET /some-url
Assert

50. JFokus 2012 Stockholm, Feb 14th 2012 #jfokus
Sloooow

51. JFokus 2012 Stockholm, Feb 14th 2012 #jfokus
Servlet Tests
Memory
IO
CPU

52. JFokus 2012 Stockholm, Feb 14th 2012 #jfokus
Testing Multiple Containers
Cloud
– 1 Small instance per Container
– Need to update AMIs
– Slow
– Isolation

53. JFokus 2012 Stockholm, Feb 14th 2012 #jfokus
Single Machine
Port Hell
Memory Limits
Speed

54. JFokus 2012 Stockholm, Feb 14th 2012 #jfokus
Story of DVCS

55. JFokus 2012 Stockholm, Feb 14th 2012 #jfokus
Story of DVCS

56. JFokus 2012 Stockholm, Feb 14th 2012 #jfokus
Story of DVCS
Problem
– Many JDKs
– Many App Servers
– Many Servlet Containers

57. JFokus 2012 Stockholm, Feb 14th 2012 #jfokus
Story of DVCS
Problem
– Many JDKs
– Many App Servers
– Many Servlet Containers
Slow feedback cycle

58. JFokus 2012 Stockholm, Feb 14th 2012 #jfokus
Story of DVCS
Problem
– Many JDKs
– Many App Servers
– Many Servlet Containers
Slow feedback cycle

59. JFokus 2012 Stockholm, Feb 14th 2012 #jfokus
Story of DVCS
Problem
– Many JDKs
– Many App Servers
– Many Servlet Containers
Slow feedback cycle
Difficult to install environments

60. JFokus 2012 Stockholm, Feb 14th 2012 #jfokus
Story of DVCS
Tests green?
Push to branch/repository
DEV STABLE
n tests on N servers m tests on M
(30 min) servers
(3 hours)

61. JFokus 2012 Stockholm, Feb 14th 2012 #jfokus
Story of DVCS

62. JFokus 2012 Stockholm, Feb 14th 2012 #jfokus
General stuff

63. JFokus 2012 Stockholm, Feb 14th 2012 #jfokus
Fight for executors

64. JFokus 2012 Stockholm, Feb 14th 2012 #jfokus
Optimum
utilization

65. JFokus 2012 Stockholm, Feb 14th 2012 #jfokus
Quick feedback

66. JFokus 2012 Stockholm, Feb 14th 2012 #jfokus
There is an app
plugin for that

67. JFokus 2012 Stockholm, Feb 14th 2012 #jfokus
Q&A