A simple search for "puppet-apache" on GitHub returns 70 separate repositories. An awful lot of people are busy reinventing the same configuration wheel. Configuration management tools promise write once, run anywhere code; but writing code that can be used by anyone looks like a lot of work. This presentation aims to show anyone familiar with Puppet how to write reusable modules and importantly how to make them compatible with already shared modules released on the Forge or elsewhere. We'll look at when and why testing a declarative language is actually useful, examples of good and bad modules and how to re-factor puppet code for re-usability. We'll also talk about potential improvements to Puppet that would make reuse easier. Gareth Rushgrove Technical Architect, Government Digital Service Gareth Rushgrove is now a technical architect at the Government Digital Service, part of the UK Government. He is mainly interested in configuration management, infrastructure and platform as a service, deployment and monitoring tooling and the whole devops community. He thinks when used well together these allow you to move really fast, even in tightly controlled environments like Government. When not working, Gareth can be found blogging over on morethanseven.net or uploading code to GitHub. He also curates the Devops Weekly newsletter and occasionally organises community events.

1. Puppet Module
Reusability
Learning from shipping to the Forge
Gareth Rushgrove

2. Who
(Who is this person?)

3. @garethr

4. UK Government
Digital Service

8. Last code
I wrote

11. 7. Gareth Rushgrove

12. Last puppet
module I wrote

13. The Problem
(Sharing modules is hard)

14. Not invented here
syndrome
1

15. A search for
puppet-apache

16. 145
Gareth Rushgrove

17. 145Gareth Rushgrove

18. Puppetlabs
apache

19. 67
contributors
Gareth Rushgrove

20. 281
forks
Gareth Rushgrove

21. 281Gareth Rushgrove

22. Vendor everything
pattern
2

23. Not publishing to the
Forge
3

24. Puppet
Forge

25. Undocumented
modules
4

26. Official docs
guidelines

27. (Too) opinionated
configuration
5

28. Sometimes I just
want to write nginx
configuration
Gareth Rushgrove

29. Duplicate resources
6

30. Package { 'build-essential’:
ensure => installed,
}
Gareth Rushgrove

31. Stdlib
(Start here)

32. Standard
Library from
Puppetlabs

33. Fail fast
Gareth Rushgrove

34. Validation
Gareth Rushgrove

35. Useful error
messages
Gareth Rushgrove

36. fail("${::operatingsystem}
not supported")
Gareth Rushgrove

37. validate_string
Gareth Rushgrove

38. validate_string($version)
Gareth Rushgrove

39. <Puppet::Error: true is
not a string.
Gareth Rushgrove

40. validate_slength
Gareth Rushgrove

41. validate_re
Gareth Rushgrove

42. validate_hash
Gareth Rushgrove

43. Gareth Rushgrove
validate_cmd

44. validate_bool
Gareth Rushgrove

45. Avoid duplicate
resources
Gareth Rushgrove

46. Package { 'build-essential’:
ensure => installed,
}
Gareth Rushgrove

47. include 'gcc'
Gareth Rushgrove

48. package{[
'python-pip',
'python-ldap',
]:
ensure => installed
}
Gareth Rushgrove

49. ensure_packages
Gareth Rushgrove

50. ensure_packages([
'python-pip',
'python-ldap',
])
Gareth Rushgrove

51. ensure_resource
Gareth Rushgrove

52. Nicer interfaces
Gareth Rushgrove

53. str2bool
Gareth Rushgrove

54. any2array
Gareth Rushgrove

55. And much more
Gareth Rushgrove

56. Dependency
management is
everywhere
(Else)

57. NPM, Bundler, Pip,
Mix, Leiningen
Gareth Rushgrove

58. Librarian
Puppet

59. r10k

60. echo 'modules' >> .gitignore
Gareth Rushgrove

61. dependency 'puppetlabs/stdlib'
dependency 'garethr/erlang'
dependency 'maestrodev/wget'
Gareth Rushgrove

62. forge "http://forge.puppetlabs.com"
mod 'puppetlabs/ruby'
mod 'puppetlabs/ntp'
mod 'puppetlabs/git'
mod 'puppetlabs/vcsrepo'
mod 'puppetlabs/apt'
mod 'puppetlabs/gcc'
Gareth Rushgrove

63. librarian-puppet install
Gareth Rushgrove

64. A Nice
Module Pattern
(Structuring modules for reuse)

65. R.I.Pienaar

66. install, config,
service, params
Gareth Rushgrove

67. manifests
!"" config.pp
!"" init.pp
!"" install.pp
!"" params.pp
#"" service.pp
Gareth Rushgrove

68. class sample (
) inherits sample::params {
class { 'sample::install': } ->
class { 'sample::config': } ~>
class { 'sample::service': } ->
Class['sample']
}
Gareth Rushgrove

69. anchor { 'sample::begin': } ->
class { 'sample::install': } ->
class { 'sample::config': }
class { 'sample::service': } ->
anchor { 'sample::end': }
Gareth Rushgrove

70. Anchor['sample::begin'] ~>
Class['sample::service']
Class['sample::install'] ~>
Class['sample::service']
Class['sample::config'] ~>
Class['sample::service']
Gareth Rushgrove

71. Puppet module tool
Gareth Rushgrove

72. puppet module generate name
Gareth Rushgrove

73. Default module
skeleton
Gareth Rushgrove

74. In puppet
source code

75. !"" manifests
!"" spec
!"" tests
!"" Modulefile
!"" .gitignore
#"" README
Gareth Rushgrove

76. Creating your own
module skeleton
Gareth Rushgrove

77. ~/.puppet/var/puppet-module/skeleton
Gareth Rushgrove

78. puppet module
skeleton

79. !"" manifests
!"" spec
!"" tests
!"" templates
!"" files
!"" lib
!"" Gemfile
Gareth Rushgrove

80. !"" Rakefile
!"" .nodeset.yml
!"" .fixtures.yml
!"" .travis.yml
!"" Modulefile
!"" .gitignore
#"" README.md
Gareth Rushgrove

81. Creates install,
config, service,
params classes
Gareth Rushgrove

82. Dependency
management with
Bundler
Gareth Rushgrove

83. Better unit testing
setup using rspec-
puppet-helper
Gareth Rushgrove

84. Adds syntax and lint
checking
Gareth Rushgrove

85. Adds Travis CI
configuration
Gareth Rushgrove

86. Adds integration tests
with rspec-system
Gareth Rushgrove

87. Adds module
management with
maestrodev
blacksmith
Gareth Rushgrove

88. Focus on the
interface
Gareth Rushgrove

89. Minimize number of
entry points
Gareth Rushgrove

90. Allow overriding
provided templates
Gareth Rushgrove

91. Multi-OS Support
(Where to abstract the differences)

92. Vary default
parameters
Gareth Rushgrove

93. Keep logic in one
place
Gareth Rushgrove

94. case $::osfamily {
'Debian': {
}
'RedHat', 'Amazon': {
}
default: {
fail("${::operatingsystem} not su
}
Gareth Rushgrove

95. ARM-9 Data
in Modules

96. garethr-erlang

97. 0.0.x supported
Ubuntu only
Gareth Rushgrove

98. 0.1.x supports
Ubuntu, Debian,
Redhat, Suse
Gareth Rushgrove

99. Gareth Rushgrove

100. Pull requests to
the rescue

102. Insist on tests with
open source
contributions
Gareth Rushgrove

103. Module Testing
(Code should have tests)

104. Why test a
declarative
language?
Gareth Rushgrove

105. Modules increasingly
contain logic
Gareth Rushgrove

106. Modules increasingly
take arguments
Gareth Rushgrove

107. Modules increasingly
have interfaces with
other modules
Gareth Rushgrove

108. Modules increasingly
used in many Ruby
and Puppet version
combinations
Gareth Rushgrove

109. Good news. The tools
got better
Gareth Rushgrove

110. puppet-lint
Gareth Rushgrove

111. Puppet
style guide

112. Available
as a gem

113. puppet-lint --with-filename /etc/puppet/modules
foo/manifests/bar.pp: trailing whitespace found
on line 1
apache/manifests/server.pp: variable not enclosed
in {} on line 56
Gareth Rushgrove

114. puppet-syntax
Gareth Rushgrove

115. Validate Puppet
and ERB syntax

116. require 'puppet-syntax/tasks/puppet-syntax'
Gareth Rushgrove

117. bundle exec rake syntax
---> syntax:manifests
---> syntax:templates
Gareth Rushgrove

118. rspec-puppet
Gareth Rushgrove

119. Unit testing
for Puppet

120. context "epel enabled" do
let(:params) {{ :epel_enable => true }}
it { should contain_class('epel') }
end
Gareth Rushgrove

121. context "epel disabled" do
let(:params) {{ :epel_enable => false }}
it { should_not contain_class('epel') }
end
Gareth Rushgrove

122. Fixtures,
matchers

123. Test the interface not
the implementation
Gareth Rushgrove

124. All of the above
Gareth Rushgrove

125. task :test => [
:syntax,
:lint,
:spec,
]
Gareth Rushgrove

126. bundle exec rake test
Gareth Rushgrove

127. Gareth Rushgrove

128. Nice continuous
integration

130. Test pull request
branches too

131. ---
language: ruby
before_install: rm Gemfile.lock || true
rvm:
- 1.8.7
- 1.9.3
script: bundle exec rake test
env:
matrix:
- PUPPET_VERSION="~> 2.7.0"
- PUPPET_VERSION="~> 3.1.0"
- PUPPET_VERSION="~> 3.2.0"
Gareth Rushgrove

132. A matrix of
possibilities
Gareth Rushgrove

133. rspec-system
Gareth Rushgrove

134. Integration
test against
real systems

135. Puppet
helpers for
rspec-system

136. Vagrant
driver

137. VSphere
provider

138. Test that Puppet runs
without errors
Gareth Rushgrove

139. it 'should run without errors' do
pp = "class { 'sample': }"
puppet_apply(pp) do |r|
r.exit_code.should == 2
end
end
Gareth Rushgrove

140. Test runs are
idempotent
Gareth Rushgrove

141. it 'should run without errors' do
pp = "class { 'sample': }"
puppet_apply(pp) do |r|
r.exit_code.should == 2
r.refresh
r.exit_code.should be_zero
end
end
Gareth Rushgrove

142. Test that the module
installs relevant
binaries
Gareth Rushgrove

143. it 'should install the erl binary' do
shell 'which erl' do |r|
r.stdout.should =~ //usr/bin/e
r.stderr.should be_empty
r.exit_code.should be_zero
end
end
Gareth Rushgrove

144. Test against different
operating systems
Gareth Rushgrove

145. default_set: 'centos-64-x64'
sets:
'centos-64-x64':
nodes:
"main.foo.vm":
prefab: 'centos-64-x64'
'fedora-18-x64':
nodes:
"main.foo.vm":
prefab: 'fedora-18-x64'
'debian-607-x64':
nodes:
"main.foo.vm":
Gareth Rushgrove

146. Room for
improvements
(Making reuse easier in Puppet)

147. Run your own Forge
1

148. A more secure Forge
2

149. Bless a dependency
management tool
3

150. Optional
dependencies
4

151. dependency 'puppetlabs/stdlib'
optional_dependency 'puppetlabs/apache'
Gareth Rushgrove

152. Private classes
5

153. private class elixir::install {
Gareth Rushgrove

154. Parameter naming
conventions
6

155. example42
Proposal

156. Interfaces in Puppet
7

157. interface packageandservice {
$version
$enable
$package_name
$template
}
Gareth Rushgrove

158. class diamond (
$version,
$enable,
$package_name,
$template,
) {
implements packageandservice
Gareth Rushgrove

159. Questions?
(And thanks for listening)