Puppet Camp Dallas 2014: "Replacing Simple Puppet Modules with Providers" by Greg Swift, Rackspace

1. Replacing simple modules with
Custom Types and Providers
Or Stop managing templates, and start managing your configs

2. 2
Greg Swift
Linux Admin/Engineer ~ 12 yrs
Red Hat Certified Engineer ~ 6 yrs
Augeas user ~6 yrs
Puppet user ~ 3 yrs
greg.swift@{rackspace.com,nytefyre.net}
google.com/+GregSwift
linkedin.com/gregoryswift
github.com/{gregswift,rackergs}
xaeth on Fedora, FreeNode, Twitter, and Ingress

3. 3
Bit of time travel...
•Past
–An unpleasant reminder of configs past
•Present
–Tools available today that help
•Future
–What's next?

4. 4
Stroll down memory lane

5. 5
systl.conf
# Controls the default maximum size of a message queue
kernel.msgmnb = 65536

6. 6
Lets change that value
sed ­i 's/^(kernel.msgmnb = )([0­9]*)$/## Changing
for db configuration. Was:n## 12n199999/'
sysctl.conf

7. 7
Looks good so far...
# Controls the default maximum size of a message queue
## Changing for db configuration. Was:
## kernel.msgmnb = 65536
kernel.msgmnb = 99999

8. 8
But the next run?
# Controls the default maximum size of a message queue
## Changing for db configuration. Was:
## ## Changing for db configuration. Was:
## kernel.msgmnb = 65536
kernel.msgmnb = 99999
## Changing for db configuration. Was:
## kernel.msgmnb = 99999
kernel.msgmnb = 99999

9. 9
That was then...

10. 10
Templates... yay?
•Great for 1 type of system... maybe even a couple
•Supporting multiple OS releases or distributions?

11. 11
Wouldn't it be nice?
•Safe
•Repeatable
•Extensible
•Multi-language

12. 12
But that is a herculean task...

13. 13
Meet team Hercules
David Lutterkort
(Now @ PuppetLabs)
Raphaël Pinson
Dominic Cleal
Francis Giraldeau

14. 14
and Augeas

15. 15
What is it?
•An API provided by a C library
•A domain-specific language to describe configuration file
formats, presented as lenses
•Canonical tree representations of configuration files
•A command line tool to manipulate configuration from
the shell and shell scripts
•Language bindings to do the same from your favorite
scripting language

16. 16
Lense all the things!

17. 17
Just to name a few....
access activemq_conf activemq_xml aliases anacron approx aptcacherngsecurity aptconf
aptpreferences aptsources apt_update_manager authorized_keys automaster
automounter avahi backuppchosts bbhosts bootconf build cachefilesd carbon cgconfig
cgrules channels cobblermodules cobblersettings collectd cron crypttab cups cyrus_imapd
darkice debctrl desktop device_map dhclient dhcpd dnsmasq dovecot dpkg dput erlang
ethers exports fai_diskconfig fonts fstab fuse gdm group grub gtkbookmarks host_conf
hostname hosts_access hosts htpasswd httpd inetd inifile inittab inputrc interfaces iproute2
iptables jaas jettyrealm jmxaccess jmxpassword json kdump keepalived krb5 ldif ldso
lightdm limits login_defs logrotate logwatch lokkit lvm mcollective mdadm_conf
memcached mke2fs modprobe modules modules_conf mongodbserver monit multipath
mysql nagioscfg nagiosobjects netmasks networkmanager networks nginx nrpe nsswitch
ntp ntpd odbc openshift_config openshift_http openshift_quickstarts openvpn pam
pamconf passwd pbuilder pg_hba php phpvars postfix_access postfix_main postfix_master
postfix_transport postfix_virtual postgresql properties protocols puppet puppet_auth
puppetfileserver pythonpaste qpid quote rabbitmq redis reprepro_uploaders resolv rsyncd
rsyslog rx samba schroot securetty sep services shells shellvars shellvars_list simplelines
simplevars sip_conf slapd smbusers solaris_system soma spacevars splunk squid ssh
sshd sssd stunnel subversion sudoers sysconfig sysctl syslog systemd thttpd up2date util
vfstab vmware_config vsftpd webmin wine xendconfsxp xinetd xml xorg xymon yum

18. 18
Don't see your favorite config?
•Build
•IniFile
•Rx
•Sep
•Shellvars
•Shellvars_list
•Simplelines
•Simplevars
•Util

19. 19
Our earlier example.. on Augeas
augeas { 'set kernel.msgmnb per db vendor':
context => '/files/etc/sysctl.conf',
onlyif => 'kernel.msgmnb != 99999',
changes => 'set kernel.msgmnb 99999',
}

20. 20
Making it re-usable
define sysctl ($value) {
augeas { “set ${title} in sysctl.conf”:
context => '/files/etc/sysctl.conf',
onlyif => “${title} != ${value}”,
changes => “set ${title} ${value}”,
}
}
sysctl { 'kernel.msgmnb':
value => '99999',
}

21. 21
A more complex example...
define ssh_allowgroup ($ensure) {
if $ensure == present {
$match = '=='
$change = “set AllowGroups/01 ${title}”
} else {
$match = '!='
$change = 'rm AllowGroups/[.=${title}]”
}
augeas { “sshd_config/AllowGroups ${title}”:
context => '/files/etc/sshd_config',
onlyif => “match AllowGroups/[.=${title}] size $match 0”,
changes => $change,
}
}
$sshd_default_groups = ['engineers', 'admins']
$sshd_allowed_groups = $::env ? {
/prod/ => $sshd_default_groups,
default => concat($sshd_default_groups, ['devs']),
}
ssh_allowgroup { $sshd_allowed_groups:
ensure => present,
}

22. 22
Well I tried it once, but...
•Lenses are hard to write
•Xpathing is hard
•Its just hard!

23. 23
Make it easier!

24. 24
Introducing AugeasProviders
•Collection of custom types and providers
•Written in native Ruby rather than Puppet's DSL
•Utilizes bindings directly for flexibility
•Heavily tested

25. 25
And that example on AugeasProviders
sysctl { 'kernel.msgmnb':
value => '99999',
comment => 'recommended by db vendor'
}

26. 26
And the more complex example
$sshd_default_groups = ['engineers', 'admins']
$sshd_allowed_groups = $::env ? {
/prod/ => $sshd_default_groups,
default => concat($sshd_default_groups, ['devs']),
}
sshd_config { 'AllowGroups':
value => $sshd_allowed_groups,
notify => Service['sshd'],
}

27. 27
What's it got?
•host
•mailalias
•sshd_config
•shellvars /etc/{defaults,sysconfig}/*
•puppet's auth.conf (puppet_auth)
•syslog.conf entries (rsyslog and sysklog!)
•Grub and Grub2 kernel_parameter
•pam
•And more!

28. 28
Give it to me!

29. 29
Load it up
puppet module install domcleal/augeasproviders
or
git clone https://github.com/hercules­
team/augeasproviders

30. 30
What about the future??

31. 31
AugeasProviders next

32. 32
What's changing?
•Minimized duplication of most common patterns
•Solid generic library for reuse-ability
•Enables Augeas based providers in your modules

33. 33
Contribute

34. 34
What can you do?
•Use it
•Report bugs
•Create new providers!
–resolv.conf
–systemd unit files
–etc

35. 35
Educate me!

36. 36
Augeas training
•Provided by camptocamp
•http://camptocamp.com
– Solutions->Infrastructure->Training
•Fundamentals
–Using augtool, XPath Augeas language, Augeas type
in Puppet
•Advanced
– Develop using augeas libraries and advanced tree
manipulation
•Extending Augeas
–Writing lenses and providers

37. 37
Info and Help
•augeas.net
•augeasproviders.com
•#augeas on FreeNode
•augeas@lists.redhat.com

38. 38