1. CONTINUOUS INTEGRATION

2. INTRODUCTIONS
 Brendan Jennings
 Alexandre Morot
 Intuit
 Any opinions are our own

3. OVERVIEW OF PRESENTATION
 What Is Continuous Integration
 Previous System Pain Points
 Build Server
 Unit Tests
 Automated Deployment
 Verdict
 Questions

4. WHAT IS CONTINUOUS INTEGRATION

5. PREVIOUS SYSTEM PAIN POINTS

6. SOURCE CONTROL/BROKEN BUILDS

7. SOURCE CONTROL/BROKEN BUILDS
 Issues Encountered (It works on my
machine)
 Incorrect DLL references
 DLL is in a different location to hard coded path
 DLL does not exist on machine
 DLL reference is to output directory

8. SOURCE CONTROL/BROKEN BUILDS
 Issues Encountered (It works on my
machine)
 Incorrect
DLL references
 Incomplete check-ins
 Fileswithout project
 Project without new files

9. SOURCE CONTROL/BROKEN BUILDS
 Issues Encountered (It works on my
machine)
 IncorrectDLL references
 Incomplete check-ins
 Overwritten changes
 Check in timing (end of day check-ins)
 Staying on old version to avoid build errors
 Fairly Frequent Downtime for Entire Team
Resolving Build Errors

10. MANUAL TESTING OF CHANGES

11. MANUAL TESTING OF CHANGES
 Time Intensive
 No Full Regression Test
 Testing Time Reduced When Running Late

12. MANUAL PUSH TO PRODUCTION

13. MANUAL PUSH TO PRODUCTION
 Requires Observer to Reduce Risk of Errors

14. STEPS INVOLVED

15. STEPS INVOLVED
 Build

16. STEPS INVOLVED
 Zip

17. STEPS INVOLVED
 Push to Datacenter

18. STEPS INVOLVED
 Pull to Pre-Production

19. STEPS INVOLVED
 Extract and Replace Build

20. STEPS INVOLVED
 QA

21. STEPS INVOLVED
 QA
 Restart if Failed

22. STEPS INVOLVED
 For Each Server

23. STEPS INVOLVED
 For Each Server
 Pull from Rotation

24. STEPS INVOLVED
 For Each Server
 Extract and Replace Build

25. STEPS INVOLVED
 For Each Server
 Warm Up Server

26. STEPS INVOLVED
 For Each Server
 Put Back In Rotation

27. STEPS INVOLVED
 Pray

28. STEPS INVOLVED
 Release Complete
 Typically 2 Hours Later

29. SERVERS NOT ALL PUSHED TOGETHER
 Leaves Site in Inconsistent State

30. SERVERS NOT ALL PUSHED TOGETHER
 Leaves Site in Inconsistent State
 If
inconsistency is severe, site needs to be taken
down during push

31. BUILD SERVER

32. BUILD SERVER
 Continuous Build of Every Check-In

33. BUILD SERVER
 Continuous Build of Every Check-In
 Used Hudson, later switched to Jenkins

34. BUILD SERVER
 Continuous Build of Every Check-In
 Immediate Notification of Build Errors
 Names and shames guilty party
 Notifications come through to phone
 Encourages pride in build quality
 Only check-in working code
 Immediate action must be taken for build errors
 Notification sent for every check-in that is still broken
 Final notification sent when problem resolved

35. UNIT TESTS

36. UNIT TESTS
 Gradually Adding Unit Tests for Existing
Code
 Focus on coverage for most critical areas first

37. UNIT TESTS
 Gradually Adding Unit Tests for Existing
Code
 Default to Unit Tests for New/Updated Code
 Add Test Execution to Build Server
 Failed unit tests also send notifications to team
 History maintained; can track fixes to tests

38. UNIT TESTS
 Gradually Adding Unit Tests for Existing
Code
 Default to Unit Tests for New/Updated Code
 Add Test Execution to Build Server
 All Tests Must Pass
 If the test has found a bug, the code must be
fixed
 If the test has failed incorrectly, the test must be
fixed

39. DEPLOYMENT

40. DEPLOYMENT
 Latest Build
 Added server to run latest development version
 All successful builds pushed automatically

41. DEPLOYMENT
 Latest Build
 Data Center
 Use source control labels
 Precise control of what is included in each build
 History of all prior builds; can recreate prior versions
of site

42. DEPLOYMENT

43. SCRIPT TO AUTOMATE PUSHES

44. SCRIPT TO AUTOMATE PUSHES
 Push to Different Environments

45. SCRIPT PROCESSING
 Take One Server Out Of Rotation

46. SCRIPT PROCESSING
 Updates Server with Differential Push

47. SCRIPT PROCESSING
 Checks For Deployment Errors

48. SCRIPT PROCESSING
 Manual Final Verification Step

49. SCRIPT PROCESSING
 Takes Half of the Servers Out of Rotation

50. SCRIPT PROCESSING
 Pushes New Code to these Servers in
Parallel with Differential Pushes

51. SCRIPT PROCESSING
 Warms Up Servers

52. SCRIPT PROCESSING
 Checks for Deployment Errors

53. SCRIPT PROCESSING
 Swaps All Servers In/Out of Rotation
 Site is now effectively upgraded

54. SCRIPT PROCESSING
 Updates Remaining Servers with Differential
Pushes

55. SCRIPT PROCESSING
 Warms Up Servers

56. SCRIPT PROCESSING
 Check for Deployment Errors

57. SCRIPT PROCESSING
 Puts Servers Back into Rotation

58. SCRIPT PROCESSING
 Final Verification that All Servers are In
Rotation

59. TECHNOLOGIES USED

60. MICROSOFT WEB DEPLOY
Web Deploy simplifies the migration,
management and deployment of IIS Web
servers, Web applications and Web
sites.

61. CYGWIN BASH AND SSH SERVER
Linux-like environment for Windows making it
possible to port software running on POSIX
systems (such as Linux, BSD, and Unix
systems) to Windows.

62. JENKINS
Jenkins is an open-source
continuous integration server with
300+ plugins to support all kinds of
software

63. PLUGINS FOR TFS AND MSBUILD
 TFS plugin customized to handle labeling strategy
Customized
TFS Plugin
MSBUILD Plugin
Deployment Script
Called from Jenkins

64. VERDICT

65. QUALITY

66. QUALITY
 Build Issues Typically Identified Rapidly
 No longer have the ‘Monday morning blues’ where
nobody can build

67. QUALITY
 Build Issues Typically Identified Rapidly
 Automated Regression Tests provide Higher
Levels of Confidence
 Issues are uncovered that we believe we would have
missed in the past
 Analytics bugs were extremely unlikely to be seen
until someone complained

68. DEPLOYMENT

69. DEPLOYMENT
 Can Be Deployed by One Person
 No Downtime Required in Most Updates
 Time to Deploy Reduced to 6 Minutes
 Team more productive
 Enables rapid deployment of small changes
 Multiple deployments per day
 2 hour window can be hard to find between
meetings

70. DEPLOYMENT
 Can Be Deployed by One Person
 No Downtime Required in Most Updates
 Time to Deploy Reduced to 6 Minutes
 Can Rollback Quickly if Necessary
 Deployment can be Unattended
 If
something fails, deployment aborts
automatically
 Failure notification sent including failure reason

71. DEPLOYMENT
 Can Be Deployed by One Person
 No Downtime Required in Most Updates
 Time to Deploy Reduced to 6 Minutes
 Can Rollback Quickly if Necessary
 Deployment can be Unattended
 Team Do Not Need Production Server Login
 Removes temptation to perform fix directly on
server

72. QUESTIONS