Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...
Cobbler, Func and Puppet: Tools for Large Scale Environments
1. Cobbler, Func and Puppet:
Tools for Large Scale Environments
Francesco Crippa
fcrippa @ fedoraproject,org
This presentation is licensed under a Creative Commons
Attribution-NonCommercial-ShareAlike (BY-NC-SA) 3.0 license.
4. ...to this!
CC by Johnnie Walker: http://flickr.com/photos/johnniewalker/359440369
5. Large scale environments
● Huge number of machines
● Huge number of environments
– Production
– Pre-production
– Test
– Develop
● Huge number of people
● Need to scale!!!
6. All is complex
● An example: make a web server on line
● In theory it should be easy:
– Buy hardware
– Install operating system (next, next, next...)
– Install a http server (yum install httpd)
– Configure http (vi /etc/httpd/conf/httpd.conf)
– That's all
● ...but...
●
7. All is complex
● In a LSE you need:
– 2 web server to maintain service in high availability
– 2 geographical sites to support disaster recovery
– A testing environment to test configurations
– A development environment to setup machines
– Backup
– Monitoring
– Clustering
● Shared storage (SAN) required for each cluster
9. Where is the complexity?
● Web server configuration is simple
● But you need to configure:
– 7 different web server (with exactly the same
configuration!!!)
– Keep all configurations in sync
– 3 clusters (with different configurations)
– Shared storages (maybe with shared filesystems)
● The complexity is not on the “business”,
but on the “infrastructure”
10. Typical work flow
Installing Unconventional
Configurations
O.S. Tasks
● Installing OS to a new hardware
● Configure machines as you need
● Make (sometimes) unconventional tasks
14. OS Installation
● Usually KickStart driven
● Two main targets for a new installation:
– Real hardware
– Virtual Machine
● Usually expensive because require some
manual human tasks
– Put booting cd-rom into reader, configure network
and location of kickstart...
15. Cobbler
● Cobbler is a Linux provisioning server that
allows for rapid setup of network installation
environments.
● It keeps in sync:
– DHCP
– TFTP
– RPM Repositories
– KickStarts
19. KickStart templates
● Kickstart is a template
● You can pass values for all your variables
through “cobbler system add”
...
keyboard $KEY
...
[root@a~]# cobbler system add ksmeta=”KEY=us”
22. Configuration Management
– Centralize all configurations into a single place
– Defining “templates” to generalize similar
configurations (make one, apply hundreds)
– Manage exceptions
– Make possible for a lot of people to work all
together
30. Puppet architecture
– Client (the puppet) polls the server (the puppet-
master) to know if there are new configuration files
to download (every 30 minutes by default)
– Client is recognized through its hostname
– Server try to find a right configuration for that
hostname, else it uses a “default” config.
– ...caching, md5, checksum and ssl are part of
infrastructure
●
32. Goals
● Manage your hosts as python objects
– Groups, ACLs, SSL certificates, ...
>>> from func.overlord import client
>>> client1 = client.Client(“*.lan”)
>>> client1.service.restart(“httpd”)
>>> client1.command.run(“df h”)
[root@a~]# func “*.com” call service restart “httpd”
[root@a~]# func “*.lan” call command run “df h”
33. Modules and new features
● Func based on modules architecture
● A module support new stuff
● 20 modules (libVirt, jboss, info, process,
command, iptables, nagios, etc)
● Writing a new module is simple.
● When you write a module, it works on both CLI
and PyScripting, no modification on func are
needed.