Here are the slides from Josh Snyder's presentation called Deconfiguration Management: Making Puppet Clean Up Its Own Mess. Watch the videos at https://www.youtube.com/playlist?list=PLV86BgbREluVjwwt-9UL8u2Uy8xnzpIqa
3. I have been known to deploy 20 times daily
And Puppet runs every 30 minutes
4. That thing you deployed in a hurry isn't always in the place it should be
long term.
Or...we deploy a thing, and decide next week (or tomorrow) that it isn't
right for us
Or we just straight-up make mistakes
Or our coworkers make mistakes
git revertcan't save me
5. My colleague sends me a PR to deploy memcached config files under a new
path. Should I comment on their failure to clean up the old ones?
YES!
8. IGNORE THE PROBLEM
Ignore it! It probably won't hurt anything
Except it will confuse people.
It WILL confuse you, at two AM
JVM versions will exist on some older dev machines, but not newer ones
9. WRITE CODE TO CLEAN UP AFTER OTHER CODE
file { '/etc/memcached.conf':
ensure => absent,
}
11. NO FUN AT ALL
more code to write
more code to review
my cleanup code might be broken
someone has to clean up the cleanup code
12. WRITE MORE CODE (IN A DIFFERENT LANGUAGE)
Use Ansible clean up a er Puppet?
13. REBUILD MACHINES
Good! Cattle, not pets
I'm deploying new changes every 30 minutes
We only clean things up every day?
30 days? 60 days? Five years?
∃ an impedance and tooling mismatch
19. WHAT WILL THIS DO?
Use puppet agent --noop
Or add noop => true
Look at the system:
$ ls /etc/cassandra
...
$ dpkg -S /etc/cassandra
...
20. EXAMPLE 1: PARTIAL MANAGEMENT
Situation: We want to purge /etc/cassandra, but we need to generate
the list of seeds outside of Puppet
file { '/etc/cassandra':
ensure => directory,
recurse => true,
purge => true,
force => true,
...
}
file { '/etc/cassandra/seeds':
ensure => file,
replace => false,
...
}
21. EXAMPLE 2: CRONJOBS
Lots of cronjobs
Lots of cronjobs!
Using Yelp's
One file per job in /etc/cron.d
puppet-cron
23. Option ImplicationsOption Implications
Recompile cron to read from a
supplemental directory
Anyone else using
would have to use our patched cron
Create Fileresources for each
file we expect a from a deb.
Whenever someone installs a
package with a new cronjob in it,
they'd get a nasty surprise
Find some way to identify those
cronjobs that were originally
created by Puppet
Good
puppet-cron
25. EXAMPLE 3: /ETC/HOSTS
Puppet agent has a RAL
(resource abstraction layer)
RAL is responsible for
representing resources on the
system as Puppet Resource objects
$ puppet resource host
host { 'ip6-allnodes':
ensure => 'present',
ip => 'ff02::1',
target => '/etc/hosts',
}
host { 'ip6-allrouters':
ensure => 'present',
ip => 'ff02::2',
target => '/etc/hosts',
}
host { 'localhost':
ensure => 'present',
26. Puppet diffs resources in the catalog against the RAL it constructs
Could we ask it remove resources present in the RAL but not in the catalog?
YES!
28. HOW IT WORKS
All on the agent, a er catalog compilation
Iterate over resources, calling the generateor eval_generate
method on each.
Each resource has the opportunity to add more resources to the Puppet
run.
29. Walkthrough: fetching files from a fileserver
file { '/etc/cassandra':
ensure => directory,
source => 'puppet:///modules/cassandra/config_dir',
recurse => true,
purge => true,
force => true,
}
1. Get catalog with this resource declared
2. Puppet agent calls eval_generateon this resource
3. eval_generateexamines the disk, compares it with the Puppet
fileserver
4. Generates more resources to represent the files beneath this directory
30. HOW THE RESOURCESTYPE WORKS
resources { 'host':
purge => true,
}
1. Puppet calls generate
2. Generate finds all resources of type Hostin the catalog
3. Asks providers of Hostfor their instances
4. Compare the two
5. Emit new resources:
host { 'ip6-allnodes':
ensure => absent,
}
36. THIS MAPS WELL TO PUPPET
puppet state ⇒ autoremover state
puppet state ⇒ autoremover state
in catalog manually installed
not in catalog automatically installed
37. AN IMPLEMENTATION COMES TOGETHER
1. Synchronize the autoremover database with the Puppet catalog
2. Run apt-get autoremove
3. Problem?
38. AN IMPLEMENTATION COMES TOGETHER (PARTE DUEX)
1. Synchronize the autoremover database with the Puppet catalog
2. Run apt-get -s autoremove
3. Read the output and create Puppet package resources
4. Much rejoicing!
github.com/hashbrowncipher/puppet-package_purging
39. GENERAL PURPOSE SOLUTIONS?
Could there be a jack-of-all-trades solution to purging?
What if we could do:
purge { 'user':
unless => [ 'uid', '<=', '500' ],
}
It exists: github.com/crayfishx/puppet-purge
41. NOT MY FAVORITE DEFAULT
Q: What does this do?
package { 'mysql-server-5.7': }
package { 'bash': }
A: Creates version dri
42. UPGRADE (SOME OF THE) THINGS
package { 'mysql-server-5.7':
ensure => $my_favorite_mysql_version
}
package { 'bash': }
aptly_purgecan set all versioned packages as held by dpkg
aptly_purge { 'packages':
hold => true,
}
Upshot: apt-get dist-upgradeand unattended-upgrades
will only touch packages without a specific Puppet version specified.
43. END MATTER
Please do tell me your stories of Puppet resource purging.
Walk up and say hi right now.
works toojosh@code406.com
in Puppet Community Slack@josnyder