Weitere ähnliche Inhalte Ähnlich wie External Data in Puppet 4 (20) Kürzlich hochgeladen (20) External Data in Puppet 42. R.I.Pienaar | rip@devco.net | http://devco.net | @ripienaar
Who am I?
• Puppet User since 0.22
• Really into external data
• Author facts.d, extlookup, Hiera, MCollective
• Consultant - DevOps, Automation,
Architect, Development
• Open Source @ github.com/ripienaar
• Volcane on IRC
4. R.I.Pienaar | rip@devco.net | http://devco.net | @ripienaar
Hierarchical Data
class monitor (
Array[Monitor::Plugin] $plugins = []
) {
…
}
5. R.I.Pienaar | rip@devco.net | http://devco.net | @ripienaar
Hierarchical Data
class{“monitor”:
plugins => [
“cpu",
“write_graphite”,
“df”, …
]
}
6. R.I.Pienaar | rip@devco.net | http://devco.net | @ripienaar
Automatic Param Lookup
include monitor
# hieradb/nodes/host1.yaml
monitor::plugins:
- my_app
class{“monitor”:
plugins => [
“my_app”, “cpu”, “df”, …
]
}
+
=
7. R.I.Pienaar | rip@devco.net | http://devco.net | @ripienaar
Hierarchical Data
host1 host2 host3 host4 host5 host6
prod stage dev
dc1 dc2
Site wide common data
8. R.I.Pienaar | rip@devco.net | http://devco.net | @ripienaar
Hierarchical Data
host1 host2 host3 host4 host5 host6
prod stage dev
dc1 dc2
Site wide common data
9. R.I.Pienaar | rip@devco.net | http://devco.net | @ripienaar
Hierarchical Data
host1 host2 host3 host4 host5 host6
prod stage dev
dc1 dc2
Site wide common data
10. R.I.Pienaar | rip@devco.net | http://devco.net | @ripienaar
Hierarchical Data
host1 host2 host3 host4 host5 host6
prod stage dev
dc1 dc2
Site wide common data
$trusted[“certname”]
$environment
$facts[“dc”]
11. R.I.Pienaar | rip@devco.net | http://devco.net | @ripienaar
Automatic Param Lookup
include monitor
# hieradb/nodes/host1.yaml
monitor::plugins:
- my_app
class{“monitor”:
plugins => [
“my_app”, “cpu”, “df”, …
]
}
+
=
12. R.I.Pienaar | rip@devco.net | http://devco.net | @ripienaar
Lookup
• lookup() function replaces hiera*()
• Automatic Param Lookup more useful
• environment and module aware
• Great CLI
• YAML/JSON out of the box, more to come
• Data compatible with Hiera, cohabits with
Hiera
• Pluggable
13. R.I.Pienaar | rip@devco.net | http://devco.net | @ripienaar
Lookup - CLI
$ puppet lookup monitor::plugins —merge unique
——
- my_app
- write_graphite
- cpu
14. R.I.Pienaar | rip@devco.net | http://devco.net | @ripienaar
$ puppet lookup monitor::plugins —merge unique —explain
Merge strategy unique
…
Data Provider "Nodes"
Path “/etc/…/production/data/nodes/dev3.devco.net.yaml”
Original path: "nodes/%{trusted.certname}"
Found key: "monitor::plugins" value: [
"my_app"
]
…
Data Provider "common"
Path “/etc/…/data/common.yaml”
Original path: "common"
Found key: "monitor::plugins" value: [
"cpu",
"write_graphite"
]
…
Merged result: [
"my_app",
"cpu",
"write_graphite"
]
15. R.I.Pienaar | rip@devco.net | http://devco.net | @ripienaar
$ puppet lookup monitor::plugins —merge unique —explain
Merge strategy unique
…
Data Provider "Nodes"
Path “/etc/…/production/data/nodes/dev3.devco.net.yaml”
Original path: "nodes/%{trusted.certname}"
Found key: "monitor::plugins" value: [
"my_app"
]
…
Data Provider "common"
Path “/etc/…/data/common.yaml”
Original path: "common"
Found key: "monitor::plugins" value: [
"cpu",
"write_graphite"
]
…
Merged result: [
"my_app",
"cpu",
"write_graphite"
]
Every tier
Final result
16. R.I.Pienaar | rip@devco.net | http://devco.net | @ripienaar
Lookup - Config
# environments/production/hiera.yaml
---
version: 4
datadir: data
hierarchy:
- name: "Nodes"
backend: yaml
path: "nodes/%{trusted.certname}"
- name: "common"
backend: yaml
18. R.I.Pienaar | rip@devco.net | http://devco.net | @ripienaar
Lookup - function
$plugins = lookup(“monitor::plugins”, {
“value_type” => Array[Monitor::Plugin],
“default_value” => [“cpu”, …]
})
19. R.I.Pienaar | rip@devco.net | http://devco.net | @ripienaar
Lookup - function
function monitor::params() {
$result = {
“monitor::plugins” => [
“cpu”, …
]
}
}
$plugins = lookup(“monitor::plugins”, {
“default_value_hash” => monitor::params()
})
20. R.I.Pienaar | rip@devco.net | http://devco.net | @ripienaar
Lookup - function
$servers = lookup(“monitor::plugins”, {
“merge” => “first”
})
Default behaviour
21. R.I.Pienaar | rip@devco.net | http://devco.net | @ripienaar
Lookup - function
$servers = lookup(“monitor::plugins”, {
“merge” => “unique”
})
Array Merge (hiera_array())
22. R.I.Pienaar | rip@devco.net | http://devco.net | @ripienaar
Lookup - function
$servers = lookup(“monitor::plugins”, {
“merge” => “hash”
})
Shallow Hash Merge (hiera_hash())
23. R.I.Pienaar | rip@devco.net | http://devco.net | @ripienaar
Lookup - function
$servers = lookup(“monitor::plugins”, {
“merge” => “deep”,
})
Deep Hash Merge
24. R.I.Pienaar | rip@devco.net | http://devco.net | @ripienaar
Lookup - function
# common.yaml
users::local:
rip:
email: “rip@devco.net"
comment: “R.I.Pienaar”
sudoer: true
shell: “/bin/zsh”
25. R.I.Pienaar | rip@devco.net | http://devco.net | @ripienaar
Lookup - function
# nodes/dev3.devco.net
users::local:
rip:
email: “arri.pienaar@client.com”
26. R.I.Pienaar | rip@devco.net | http://devco.net | @ripienaar
Lookup - function
$ puppet lookup users::local —merge deep
---
rip:
email: arri.pienaar@client.com
comment: R.I.Pienaar
sudoer: true
shell: "/bin/zsh"
27. R.I.Pienaar | rip@devco.net | http://devco.net | @ripienaar
Automatic Param Lookup
include monitor
# hieradb/nodes/host1.yaml
monitor::plugins:
- my_app
class{“monitor”:
plugins => [
“my_app”, “cpu”, “df”, …
]
}
+
=
28. R.I.Pienaar | rip@devco.net | http://devco.net | @ripienaar
Lookup - function
$lookup_options = {
“merge” => {
“strategy” => “unique”
}
}
$plugins = lookup(“monitor::plugins”, $lookup_options)
29. R.I.Pienaar | rip@devco.net | http://devco.net | @ripienaar
Lookup - function
$plugins = lookup(“monitor::plugins”)
# hieradata/common.yaml
lookup_options:
monitor::plugins:
merge:
strategy: unique
30. R.I.Pienaar | rip@devco.net | http://devco.net | @ripienaar
include monitor
# …/production/hieradata/common.yaml
lookup_options:
monitor::plugins:
strategy: unique
monitor::plugins:
- cpu
- …
class monitor (
Array[Monitor::Plugin] $plugins = []
) {}
Lookup - function
31. R.I.Pienaar | rip@devco.net | http://devco.net | @ripienaar
Lookup - Data In Modules
# modules/monitor/hiera.yaml
---
version: 4
datadir: "data"
hierarchy:
- name: "common"
backend: "yaml"
# modules/monitor/data/common.yaml
lookup_options:
monitor::plugins:
strategy: unique
32. R.I.Pienaar | rip@devco.net | http://devco.net | @ripienaar
include monitor
# hieradata/common.yaml
monitor::plugins:
- cpu
- write_graphite
Lookup - Data In Modules
# hieradata/nodes/node1.yaml
monitor::plugins:
- my_app
35. R.I.Pienaar | rip@devco.net | http://devco.net | @ripienaar
hiera_include()
# common.yaml
lookup_options:
classes:
merge:
strategy: “deep”
knockout_prefix: “-“
classes:
- nagios
# nodes/node1.yaml
classes:
- -nagios
- sensu
36. R.I.Pienaar | rip@devco.net | http://devco.net | @ripienaar
lookup(“classes”).include
hiera_include()
# common.yaml
lookup_options:
classes:
merge:
strategy: “deep”
knockout_prefix: “-“
37. R.I.Pienaar | rip@devco.net | http://devco.net | @ripienaar
params.pp
# modules/ntp/hiera.yaml
---
version: 4
datadir: data
hierarchy:
- name: "Operating System Family"
backend: yaml
path: "%{facts.os.family}"
- name: "common"
backend: yaml
38. R.I.Pienaar | rip@devco.net | http://devco.net | @ripienaar
params.pp
# modules/ntp/data/AIX.yaml
---
ntp::driftfile: '/etc/ntp.drift'
ntp::package_name:
- 'bos.net.tcp.client'
ntp::restrict:
- 'default nomodify notrap nopeer noquery'
- '127.0.0.1'
ntp::service_name: 'xntpd'
39. R.I.Pienaar | rip@devco.net | http://devco.net | @ripienaar
params.pp
# modules/ntp/data/Debian.yaml
ntp::restrict:
- '-4 default kod nomodify notrap nopeer noquery'
- '-6 default kod nomodify notrap nopeer noquery'
- '127.0.0.1'
- '::1'
ntp::service_name: 'ntp'
ntp::servers:
- '0.debian.pool.ntp.org'
- '1.debian.pool.ntp.org'
40. R.I.Pienaar | rip@devco.net | http://devco.net | @ripienaar
Future
• Hiera deprecated
• No more site wide hiera, only environment
and modules
• Pluggable via functions in modules - native or
ruby, much much simpler than old backends
• lookup.yaml or lookup.conf
41. R.I.Pienaar | rip@devco.net | http://devco.net | @ripienaar
Questions?
twitter: @ripienaar
email: rip@devco.net
blog: www.devco.net
github: ripienaar
freenode: Volcane
slack.puppet.com: ripienaar
https://www.devco.net/