San Francisco Java User Group presents Kohsuke Kawaguchi from Sun who introduces us to Hudson, an open-source continuous integration (CI) system, which improves the productivity of a development team by automating various things. Hosted By: SUPINFO International University Sponsored By: TEKsystems, Sun, O'Reilly, JetBrains, and Marakana Organized By: Marakana Video By: Max Walker Video available at http://marakana.com/forums/java/general/79.html

1. HudsonSF JUG Kohsuke Kawaguchi Sun Microsystems, Inc.

2. Rise of Continuous Integration Offload from people, push to computers $ computers people time 2

3. Spend more CPU power to help you First on your laptops and workstations IDEs are at the forefront And then to the servers a.k.a. “Continuous Integration” More frequent build/test executions Static code analysis tools And more to come 3

4. Hudson Open-source CI server Emphasis on ease of installation and use “java -jar hudson.war” execution Or your OS-specific package Configure everything from browsers Extensibility 230+ community-developed public plugins By 230+ contributors Estimated 13,000 installations 4 https://hudson-ci.org/

5. Adoption in all kinds of businesses 5

6. Support Subscriptions Community releases every week Over 340 releases to date Rapid new features & bug fixes delivery “Sun Continuous Integration Server” For enterprise that needs stability Bug fix only patch releases for 18 months 6

7. Basic Feature Set Hudson notices that there is a change Check out the source code CVS, Subversion, ClearCase, Mercurial, Accurev, Perforce, Git, StarTeam, … Do builds Ant, Maven, shell script, NAnt, … Record and publish results Close the feedback loop RSS feeds E-mail/IM notification Tray application to notify developers Tool integration Findbugs, cobertura, … 7

8. Demo

9. Why do I care? Life before Hudson Dev forgets to commit a file RE nightly build breaks. It blocks SQE People waste time Life after Hudson Hudson will tell you in 5 mins if a build broke No one else notices that you forgot to commit a file 9

10. Why do I care? Life before Hudson Dev makes a change Monday AM SQE nightly finds a bug Monday night Dev fixes it Tuesday Lot of wasted time Life after Hudson Dev makes a change Monday AM Hudson finds a regression 30 mins later Dev can fix it before lunch 10

11. Why do I care? Life before Hudson SQE test runs every night, results sent out in e-mail After the enthusiasm of the 1st week is gone, nobody looks at them anymore Regressions go unnoticed until it’s too late Life after Hudson Tests run Hudson after every commit E-mail sent out only when tests start failing So it manages to keep people’s attention 11

12. Matrix projects Run the same thing on different environments such as different JDKs, different databases, and different OSes Aggregate the results 12

13. FindBugs integration 13

14. Tracking changes 14

15. … and more Browse workspaces Build time trend report 15

16. Inter-Project Support You can host many projects on 1 Hudson And when you do, 1+1 becomes more than 2 You can… Execute tests separately and correlate them with builds Mark builds as “promoted” based on test outcome Track which build of libraries are used where And more 16

17. Test/Build Separation Tests often dominate the build time Multiple tests are run in parallel Improves turn-around time time JAX-WS #10 JAX-WS #11 JAX-WS #12 … Unit test with GF Unit test with Tomcat SQE test Test w/o container 3OS x 2JDK = 6 tests 17

18. Test/Build Separation Technique Copy bits from the build to test via wget by permalinks Ant has the equivalent <get> task “Fingerprinting” to keep track of where binaries are used $ wget –O foo.zip http://server/hudson/job/foo/lastSuccessfulBuild/ artifact/foo.zip 18

19. Build Promotion CI produces a lot of “successful” builds Often overwhelming to downstream consumers Run tests as fast as you can If a build passes tests, promote it 19

20. Got the idea? Automation Reduce turn-around time Make things transparent Remove people from the loop Save people’s time Push jobs to servers, keep workstations idle for you 20

21. Doing Distributed Builds 21

22. Why? There’s only so much 1 computer can do But lots of PCs = tedious What does Hudson do to help you? 22

23. Installing new slaves For first 20 or so slaves, we did it manually Insert CD, click, type, click, type, click, … But that doesn’t scale Then we automated Available as “Hudson PXE Plugin” 23

25. ISO images of OSSlaves Power on, hit F12 PC boots from network (PXE) 24

27. ISO images of OSYour corporate IT guy & his DHCP server Slaves Power on, hit F12 PC boots from network (PXE) Choose OS from menu Installs non-interactively 25

28. Distributed builds with Hudson Master Serves HTTP requests Stores all important info Slaves 170KB single JAR Assumed to be unreliable Scale to at least 100 Link Single bi-di byte stream No other requirements 26

29. How master and slaves start talking For Unix slaves, via SSH Only need SSHD and JRE on slaves We just need a host name 27

30. How master and slaves start talking For Windows, DCOM We just need admin user name and password No manual intervention Works even from Unix masters 28

31. How master and slaves start talking Via Java Web Start When master cannot see slaves A separate socket connection is made 29

32. Automating JNLP launch Once started, can be installed as Windows service 30

33. Automating JNLP launch Emulate the JNLP client headless $ java -jar slave.jar -jnlpUrl URL 31

34. Automated Tool Installation - JDK JDK from http://java.sun.com/ Hudson automatically chooses the right bundle Always up to date with new releases 32

35. Automated Tool Installation - Apache Ant and Maven from Apache 33

36. Heterogeneous Cluster Challenge Your builds/tests need to run in specific environment Dependency on individual nodes hurts utilization jobs slaves WombatWindows test Windows #1 GlassFishWindows test Windows #2 Hudson Windows test Solaris#1 Hudson Solaris test 34

37. Labels to rescue Label is a group of slaves Tie jobs to labels jobs slaves WombatWindows test Windows #1 Windows GlassFishWindows test Windows #2 Hudson Windows test Solaris#1 Solaris Hudson Solaris test Windows #3 35

38. Concurrent Builds 36 Hudson #10 Hudson #11 Hudson #12 Hudson #13 Hudson #14 Hudson #15 time

39. Making builds sticky We want jobs to be mostly on the same slave Faster check out Consistent results Minimizes disk consumption But does it softly Hudson uses consistent hash* to achieve this More schedule controls become possible: Use faster machines more frequently Slowly ramp up newly installed slaves * http://en.wikipedia.org/wiki/Consistent_hashing 37

40. Forecasting failures Hudson monitors key health metrics of slaves Low disk space, insufficient swap Clock out of sync Extensible Slaves go offline automatically 38

41. Clean up mess after builds Kill run-away processes Daemons, background processes left by your build Works on Windows, Linux, Mac, and Solaris 39

42. Load Statistics Monitoring 40

43. When it’s time to add more slaves 41

44. Hudson EC2 plugin Automatically provisions slaves on EC2 on demand Based on the current load Picks the right AMI depending on demand Starts slave agent Shuts down unused instances 42

45. Putting it all together capacity # of executors usage queue length time 43

46. Hudson “Appliance” on EC2 Run the master in the cloud too, if you like Hudson on stock OpenSolaris AMI Data stored persistently in Elastic Block Storage Dynamically expandable thanks to ZFS Online, too Packaged as a wizard 44

47. And a lot more I couldn’t show IDE plugins iPhone/Android apps REST API and CLI Deployment Automation Hadoop, Selenium … 45

48. Conclusions CI is here to stay Using lots of PCs is a key Hudson is very easy to get started Once started, there are a lot you can do 46

49. Q&A Kohsuke Kawaguchi Sun Microsystems, Inc.

50. Compartmentalization full Metro build #10 full Metro build #11 JAXB #10 JAXB #11 JAXB #12 Faster feedback Good engineering practice anyway Isolation of problems JAX-WS #20 JAX-WS #21 JAX-WS #22 time 48

51. Automated System Installations Supports OpenSolaris, Ubuntu, CentOS, Fedora Trivial with most Linux Cooperate with Windows, too Quite useful outside Hudson, too No more broken CD drives No more CD-Rs 49