Puppet Camp Düsseldorf 2014: Continuously Deliver Your Puppet Code with Jenkins, r10k and Git (Intermediate)

1. Continuously deliver your puppet code with jenkins, r10k and git Toni Schmidbauer October 14, 2014

2. whoami I SysAdmin@s-itsolutions.at I toni@stderr.at I stderr@jabber.org I http://stderr.at I http://github.com/tosmi

3. Agenda I A short story about con

4. guration management I What is continuous delivery I Tools used to achieve continuous delivery I DEMO I Things to improve

5. A short story about con

6. guration management (CM) I We manage a very diverse environment of UNIX/Linux Systems (Solaris 10/11 SPARC/i386, AIX, RHEL/CentOS 5/6/7)

8. guration management (CM) I We manage a very diverse environment of UNIX/Linux Systems (Solaris 10/11 SPARC/i386, AIX, RHEL/CentOS 5/6/7) I We've got around 1000 nodes in total

10. guration management (CM) I We manage a very diverse environment of UNIX/Linux Systems (Solaris 10/11 SPARC/i386, AIX, RHEL/CentOS 5/6/7) I We've got around 1000 nodes in total I Before CM we had strict standards on how to manage these systems

12. guration management (CM) I We manage a very diverse environment of UNIX/Linux Systems (Solaris 10/11 SPARC/i386, AIX, RHEL/CentOS 5/6/7) I We've got around 1000 nodes in total I Before CM we had strict standards on how to manage these systems I The problem: count(team members) == count(standards)

14. guration management (CM) I We manage a very diverse environment of UNIX/Linux Systems (Solaris 10/11 SPARC/i386, AIX, RHEL/CentOS 5/6/7) I We've got around 1000 nodes in total I Before CM we had strict standards on how to manage these systems I The problem: count(team members) == count(standards) I So con

15. guration management is the solution to all our problems

16. The solution to all our problems

17. The solution to all our problems I Broke our systems

18. WHY????

19. Problems with our old CM system I Deployments sucked

20. Problems with our old CM system I Deployments sucked I Deployment via manual tagging and checkout, so mistakes happened I Deployment in stages, but we always had to cross our

21. ngers

23. ngers I Testing sucked

25. ngers I Testing sucked I No Unittests I No acceptance tests

27. ngers I Testing sucked I No Unittests I No acceptance tests I No immediate feedback if things where OK or not

29. ngers I Testing sucked I No Unittests I No acceptance tests I No immediate feedback if things where OK or not I Systems installed without CM are hard to bring under CM control

31. ngers I Testing sucked I No Unittests I No acceptance tests I No immediate feedback if things where OK or not I Systems installed without CM are hard to bring under CM control I Every system was a special case

32. So what's our solution?

33. Continuous delivery I is a pattern for getting software from development to release 1 1Continuous Delivery: Jez Humble, David Farley

34. Continuous delivery I is a pattern for getting software from development to release I this pattern is called the deployment pipeline 1 1Continuous Delivery: Jez Humble, David Farley

35. The deployment pipeline Commit Stage Automated unit testing Automated acceptance testing Tag release Deployment

36. Why are we using continuous delivery I When you automate your deployment,

37. Why are we using continuous delivery I When you automate your deployment, I less mistakes will happen and the same mistake will not happen twice

38. Why are we using continuous delivery I When you automate your deployment, I less mistakes will happen and the same mistake will not happen twice I less mistakes means less stress when deploying

39. Why are we using continuous delivery I When you automate your deployment, I less mistakes will happen and the same mistake will not happen twice I less mistakes means less stress when deploying I less stress means you are going to deploy more often

40. Why are we using continuous delivery I When you automate your deployment, I less mistakes will happen and the same mistake will not happen twice I less mistakes means less stress when deploying I less stress means you are going to deploy more often I more deployments means a more exible environment

42. Tools to build a deployment pipeline

43. Jenkins I Jenkins is an Open Source continuous integration server I It's purpose is to execute and monitor jobs I Jobs are shell scripts or any other thing that's executable and returns 0 on success I Many plugins available to extend Jenkins (e.g. git, build-pipeline, monitor) I You can link jobs together, thats our pipeline

44. Jenkins II

45. Monitoring with Jenkins

46. GIT I git push triggers the deployment pipeline

47. GIT I git push triggers the deployment pipeline I one central repository for internal modules I gitolite for access control I 3 main branches I development I testing I production I feature branches for new site local modules I hiera data is in the same repository

48. GIT repository layout I modules/: I where r10k stores external (forge, github) modules I site/: I site local modules, we do not want to share I hiera/: I our hiera yaml

49. les I Puppetfile: I con

51. le for r10k that speci

52. es which external modules we need I Vagrantfile: I To boot a development puppet environment on your local workstation

53. GIT work ow Puppet developer Development Puppet Master Testing Puppet Master Production Puppet Master Feature Branch Development Branch Testing Branch Production Branch 1 2 3 4 Dev Nodes (7−8) Testing Nodes (~40) Production Nodes 1 2 3 4 1 Features Branches get automatically created on Puppet Master (Dynamic Environments) 2 Development Branch gets deployed on commit via Jenkins 3 Testing Branch gets deployed via GIT tag 4 GIT Jenkins pushing to testing triggers a deployment Production Branch gets deployed via GIT tag pushing to production triggers a deployment It’s all the same for Hiera yaml files!

54. r10k I a tool to deploy puppet environments and modules I every git branch gets deployed to a corresponding puppet environment I it also downloads and installs modules from puppetforge or github I in the current version (1.3.2) dependencies have to be managed manually

55. Example Puppet

56. le 1 f o r g e ' f o r g e . p u p p e t l a b s . com' 3 mod ' p u p p e t l a b s / ntp ' , ' 3 . 1 . 2 ' mod ' p u p p e t l a b s / p o s t g r e s q l ' , ' 3 . 4 . 2 ' 5 mod ' p u p p e t l a b s / s t d l i b ' , ' 4 . 3 . 2 ' mod ' p u p p e t l a b s / f i r e w a l l ' , ' 1 . 1 . 3 ' 7 mod ' p u p p e t l a b s / apache ' , ' 1 . 1 . 1 ' mod ' p u p p e t l a b s /lvm ' , ' 0 . 3 . 2 ' 9 mod ' n o s o l u t i o n s /tsm ' , ' 0 . 2 . 2 ' mod ' s a z / sudo ' , ' 3 . 0 . 6 ' 11 mod ' s p i e t t e / s e l i n u x ' , ' 0 . 5 . 4 ' 13 mod ' concat ' , : g i t => ' h t t p s : / / g i t h u b . com/ p u p p e t l a b s / puppe t l absconcat ' , 15 : commit = ' feba3096c99502219043b8161bde299ba65e7b8a ' You are able to pin to a git tag / branch / commit hash

57. a word on testing I you must have unit tests for your puppet code: rspec-puppet I for acceptance tests there's puppetlabs/beaker I you need to test everything to get most out of the build pipeline I we test I internal puppet modules I hiera data I puppet con

58. guration I all internal modules are required to have rspec tests

59. rspec-puppet example samplemodule/manifests/init.pp 1 c l a s s samplemodule ( $mes sage = ' d e f a u l tme s s a g e ' ) f n o t i f y f ' samplemes sage ' : 3 mes sage = Thi s i s the sample module , my mes sage i s : $mes sage , g 5 g samplemodule/spec/classes/samplemodules spec.rb 1 r e q u i r e ' s p e c h e l p e r ' 3 d e s c r i b e ' samplemodule ' , : t ype = : c l a s s do c o n t e x t ' wi th d e f a u l t parame t e r s ' do 5 i t f s h o u l d c o n t a i n n o t i f y ( ' samplemes sage ' ) g end 7 end

60. beaker example 1 r e q u i r e ' s p e c h e l p e r a c c e p t a n c e ' 3 d e s c r i b e ' p r o f i l e s : : o s s b a s e c l a s s ' do i t ' s h o u l d work wi th no e r r o r s ' do 5 a p p l y ma n i f e s t ( ' i n c l u d e p r o f i l e s : : os sba s e ' , : c a t c h f a i l u r e s = t r u e ) end 7 end

61. DEMO

62. Do try this at home I You need: I Vagrant from http://vagrantup.com I Virtualbox I Git client I You have to run: I git clone https://github.com/tosmi/puppetcamp2014.git I cd puppetcamp2014 I vagrant up I vagrant ssh

63. Things we have to improve I We need more test Systems (CentOS/RHEL/Solaris/AIX?) I We need more acceptance tests I Once again use stages in production I Our documentation

64. Things puppetlabs should improve I Puppetlabs should package beaker as a rpm/deb whatever, gems suck in production

65. Summary I Continuous Delivery is implemented via a deployment pipeline

66. Summary I Continuous Delivery is implemented via a deployment pipeline I Within the pipeline everything is automated

67. Summary I Continuous Delivery is implemented via a deployment pipeline I Within the pipeline everything is automated I When everything is automated you need to test everything

68. Summary I Continuous Delivery is implemented via a deployment pipeline I Within the pipeline everything is automated I When everything is automated you need to test everything I When everything is automated and well tested deploying becomes easy

69. Summary I Continuous Delivery is implemented via a deployment pipeline I Within the pipeline everything is automated I When everything is automated you need to test everything I When everything is automated and well tested deploying becomes easy I Tools we are using to implement a deployment pipeline I Jenkins: to execute jobs I GIT: version control I Gitolite: access control and authorization for GIT I r10k: install puppet modules / create puppet environments from branches I rspec-puppet: unittests for puppet I puppetlabs/beaker: acceptance tests for puppet

70. Thanks for your attention!

Puppet Camp Düsseldorf 2014: Continuously Deliver Your Puppet Code with Jenkins, r10k and Git (Intermediate)

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Viewers also liked

Viewers also liked (9)

Similar to Puppet Camp Düsseldorf 2014: Continuously Deliver Your Puppet Code with Jenkins, r10k and Git (Intermediate)

Similar to Puppet Camp Düsseldorf 2014: Continuously Deliver Your Puppet Code with Jenkins, r10k and Git (Intermediate) (20)

More from Puppet

More from Puppet (20)

Recently uploaded

Recently uploaded (20)

Puppet Camp Düsseldorf 2014: Continuously Deliver Your Puppet Code with Jenkins, r10k and Git (Intermediate)