Puppet's declarative language is a powerful tool for many configuration management problems. Sometimes it can be a burden to describe complex deploys or ad hoc commands in terms of the desired state of a node. Puppet tasks provide a solution to these problems that integrates these tasks with the rest of your puppet code. This talk will cover how to write simple tasks, How to install and run in PE and FOSS, common patterns for writing more complex reusable tasks, using task metadata to document your task and take full advantage of the task UI, writing tasks for PE RBAC, details of the Puppet Task API, how tasks are executed in PE and with Bolt, and how to write a simple task plan.

1. Puppet Tasks:
Taming SSH in a for loop
Alex Dreyer
Principal Software Engineer
Puppet

2. Agenda
1. About Tasks
2. Writing Tasks
3. Writing Plans

3. Why Tasks
! Live discovery or debugging across nodes
! Actions that don’t relate to desired state.
! Intermediate state during complex deploys
! Immediacy
Puppet’s declarative model isn’t enough

4. What are tasks
A task is a single action you can take on a node
! Tasks are scripts or executables written in any language
! Tasks are packaged in puppet modules
! Tasks are copied to and executed on the target

5. bolt vs puppet-task
! Bolt is OS vs puppet-task is PE
! Bolt is a ruby command line tool it loads tasks from the local disk
! puppet-task is a lightweight client for the orchestrator API
! Both should be functionally equivalent for running tasks where possible
Task Runners

6. Bolt vs Orchestrator
Bolt
! Bolt is commandline application
! Bolt does not require puppet
! Bolt can run plans
! Bolt code is installed locally
! Bolt uses ssh and winrm to connect to
nodes
Orchestrator
! Orchestrator is a service with multiple
clients(puppet-task, PE GUI)
! Orchestrator run s on a PE master and
agents on all nodes
! Orchestrator does not support plans yet
! Orchestrator uses code installed on the
master
! Orchestrator sends messages to the pxp-
agent running on nodes
Differences between the two task runners

8. Simple tasks
! Task parameters are passed as environment variables prefixed with PT_
! Version and deploy this with normal code management tools
! Grant PE rbac permissions and run the task in the console.
! Interact with your own simple scripts just like complex tasks from the Puppet Forge
Tasks are easy!
#!/usr/bin/env bash
touch $PT_file

9. Running tasks in the CLI
! "task" is the puppet or bolt subcommand
! "run" is the action
! "--nodes master.vm" is the target
! "touch" is the task to run
! "file=/tmp/foo" is the parameter to the task
! "--modules=./"
puppet task run --nodes master.vm touch file=/tmp/foo
bolt task run --nodes master.vm touch file=/tmp/foo --modules=./

10. Task Names
! A full task name has two parts: <module_name>::<task_name>
! Task names do not include the file extension
! Task names must match the puppet class name regex: /A[a-z][a-z0-9_]*Z/
! The init task's name is just <module_name> like the init class
! Task files must be at the top level of the tasks directory
! There must be only one task with any name

11. JSON Task API
! Accept typed or complex input without special parsing
! Return structured data that can be processed later
! Generate better errors and messages
! Even for simple tasks the Task Runner coerces their output in JSON

12. Using Structured Input
! A single JSON object will be passed to the task on stdin
! Parameter names must match the puppet parameter regex
! Don't treat absence and null values differently
! Use --params to pass typed JSON on the CLI

13. Structured input
1 #!/opt/puppetlabs/puppet/bin/ruby
2 #touch/tasks/array.rb
3
4 require 'json'
5 require 'fileutils'
6
7 params = JSON.parse(STDIN.read)
8
9 params['files'].each do |filename|
10 FileUtils.touch(filename)
11 puts "Updated file #{filename}"
12 end

14. Generating Structured output
! Print only a single JSON object to stdout
! Use only keys that match the parameter regex
! It's best to have defined keys at the top level
! Use standard '_' prefixed keys when appropriate.
! Use the '_output' key for a human readable summary of the result

15. 1 #!/opt/puppetlabs/puppet/bin/ruby
2 #touch/tasks/output.rb
3 require 'json'
4 require 'fileutils'
5
6 params = JSON.parse(STDIN.read)
7 file_result = params['files'].reduce({}) do |files, filename|
8 files[filename] = {}
9 files[filename]['new'] = !File.exists?(filename)
10 FileUtils.touch(filename)
11 files[filename]['success'] = true
12 files
13 end
14
15 result = { '_output' => "Successfully touched all files.",
16 'files' => file_result }
17 STDOUT.puts(result.to_json)

16. Handling errors
! Put errors in an '_error' object
! Exit non-zero for errors
! Use 'msg' in the object for the error message
! Use 'kind' in the object like an error class.
! Use 'details' for any structured information.
! Try to catch all errors otherwise you have no control over the error object
{ "msg": "Failed to update file: '/tmp/foo'",
"kind": "touch/file-error",
"details": { "file": "/tmp/foo" } }

17. 2 # touch/tasks/error.rb
..
9 exitcode = 0
10 result = {}
11 result['files'] = params['files'].reduce({}) do |files, filename|
12 begin
..
16 rescue StandardError => e
17 exitcode = 1
18 files[filename]['success'] = false
19 files[filename]['error'] = e.message
20 end
21 files
22 end
23 if exitcode == 0
24 result['_output'] = "Successfully touched all files."
25 STDOUT.puts(result.to_json)
26 else
27 errored_files = result.map { |filename, r| filename unless
r[:success] }.compact
28 STDOUT.puts({ _error: { kind: 'touch/file-error',
29 msg: "Failed to update files: #{errored_files.join(',')}",
30 details: { files: result['files'] } } }.to_json)
31 end
32 exit exitcode

18. Task Metadata
! Tasks with metadata are self documenting
! Tasks with metadata can have auto-generated Interfaces
! The Task runner will validate parameters against metadata
! Metadata can change how the task is executed
! Metadata can be used to enable features like noop
{ "description": "touch files on the target system",
"input_method": "stdin",
"supports_noop": true,
"parameters": {
"files": {
"description": "An array of files to touch",
"type": "Array[String]" } } }

19. --noop
! Tasks that support noop can be safely run in noop mode as a simulation
! When the task is called with '--noop' "_noop": true is sent with the params
! The author must make sure that no state on the system is changed when that flag is present
! Tasks that don't set "supports_noop" in metadata will not be run when a noop run is requested
2 # touch/tasks/noop.rb
16 if params['_noop']
17 raise StandardError "#{filename} isn't writable" unless File.writable?(filename)
18 dirname = File.basename(filename)
19 raise StandardError "Directory #{dirname} does not exist." unless File.directory?(dirname)
20 else
21 FileUtils.touch(filename)
22 files[filename]['success'] = true
23 end

20. Tasks on windows
Task execution program is hardcoded
Extension Program
.rb Puppet agent ruby
.pp puppet agent apply
.ps1 powershell

21. 1 #.DESCRIPTION
2 #Updates the write time on a list of files similar to the 'touch' command.
3 #
4 #.PARAMETER files
5 #The files to touch
6 [CmdletBinding()]
7 Param(
8 [Parameter(Mandatory=$true,Position=1)]
9 [String[]] $files
10 )
11
12 $ErrorActionPreference = "Stop"
13 $fileresults = @{ }
14
15 ForEach ($file in $files) {
16 if(Test-Path $file) {
17 (Get-Item $file).LastWriteTime = Get-Date
18 $fileresults.add($file, @{ "success" = [bool]$true; "new" = [bool]$true })
19 } else {
20 echo $null > $file
21 $fileresults.add($file, @{ "success" = [bool]$true; "new" = [bool]$false })
22 }}
23 ConvertTo-Json -InputObject @{ "files" = $fileresults }

22. Puppet manifest tasks
! Puppet manifest can be applied as tasks
! Powerful tool for cross platform tasks
! Code needs to installed in the configured environment on the target
! Beware of conflicts and concat
1 #!/opt/puppetlabs/bin/puppet apply
2 $pkg = case $::osfamily {
3 'redhat': 'httpd',
4 'debian': 'apache2'}
5 package {$pkg: ensure => present }

23. Writing tasks for PE RBAC
! Grant permissions for specific tasks
! Use the type system and pattern type restrictively to prevent injections
! Create versions of tasks with fewer parameters
! Create versions of tasks that refuse to run on some nodes

24. Testing Tasks
! Use the unit testing library for the language you're working in.
! Use beaker to automate system level testing with a Task Runner

25. Tasks Plans
! The Plan language allows you to string tasks together
! Task plans are written in puppet language
! Task plans are executed top to bottom unlike puppet manifest
! Task plans call tasks with the run_task function
! Tasks plans are experimental and only available in bolt

26. run_task
! Create a plan object at the top scope of your plan
! Use the run_task function to call tasks
plan touch::promote([String] $tier = 'dev') {
$node = "db.${tier}.vm"
run_task('touch::noop', [$node], 'files' => ['/etc/postgres/trigger'])
}

27. 1 plan canary::random(
2 String $task,
3 String $nodes,
4 Hash[String, Data] $params = {},
5 Integer $canary_size = 1
6 ) {
7 [$canaries, $rest] = canary::random_split($nodes.split(','),
$canary_size)
8
9 $canr = run_task($task, $canaries, $params)
10 if($canr.ok) {
11 util::print("Successfully deployed to canaries: ${canr.names}")
12 $restr = run_task($task, $rest, $params)
13 } else {
14 util::print("Deploy to canaries failed: ${canr.error_nodes.names}")
15 $restr = canary::skip($rest)
16 }
17
18 canary::merge($canr, $restr)
19 }

28. Extending Task Plans
! The Plan language is very small limited but there are a few extension points
! Tasks: Use tasks to capture anything you don't want to write in ruby or puppet. Run tasks
locally to send notifications or collect data.
! Functions: Use functions especially functions on ExecutionResult and Error objects
! Plans: Call plans from other other plans with run_plan. Use this to capture generic actions like
canary deployments.

29. Plans and PE
! In addition to running plans over ssh and winrm bolt supports using orchestrator as a
transport for running tasks.
! Install bolt
! Make sure puppet-task works, bolt will use the same config ~/.puppetlabs/client-tools/
orch.conf
! Make sure any tasks you need are available both in the production environment of your
master and in the --modules dir bolt is using
! If you want to run scripts, run commands or upload files make sure bolt is installed as a
module in the production environment of your master too. It has a single bolt task used for
these
! use the 'pcp' protocol to use orchestrator. ie --nodes pcp://agent1.example.com

30. Resources
! Code: https://github.com/adreyer/puppetconf-modules
! Task Tutorial: https://github.com/puppetlabs/tasks-hands-on-lab
! More content: https://github.com/puppetlabs/task-modules
! Forge: https://forge.puppet.com/modules?with_tasks=yes
! Docs: https://puppet.com/docs/bolt/0.5/writing_tasks_and_plans.html
! Slack: #puppet-tasks
! November Virtual PUG