1.
10 things i learned building
Nomad packs
Bram Vogelaar
@attachmentgenie

2.
Confidential and Proprietary
~ ❯ whoami => Bram Vogelaar
• Used to be a Molecular Biologist
• Then became a Dev, now an Ops
• Currently Cloud Engineer @ The Factory
• Amsterdam HUG organizer

3.
Confidential and Proprietary
Nomad
• Open-source tool for dynamic workload scheduling
• Batch, containerized, and non-containerized applications.
• Has native Consul and Vault integrations.
• Has token based access setup.
• Jobs written in (H)ashiCorp (C)onfiguration (L)anguage
https://www.nomadproject.io/

4.
Confidential and Proprietary
Nomad Job Structure
job "lorem-ipsum" {
group ”frontend" {
network {
port "http" { to = ”3000” }
}
service {
name = ”lorem"
port. = ”http"
}
task "server" {
driver = "docker"
config {
image = ”cicero/lorem-ipsum:v1.0.0"
ports = ["http"]
}
}
}

5.
Confidential and Proprietary
Surprisingly Dynamic
job "lorem-ipsum" {
group ”frontend" {
network {
port "http" { to = ”3000” }
}
service {
name = ”lorem"
port. = ”http"
}
task "server" {
driver = "docker"
config {
image = ”cicero/lorem-ipsum:v1.0.0"
ports = ["http"]
}
}
}

6.
Confidential and Proprietary
Incredibly Dynamic
● Data Centers
● Region
● Namespace
● Constraints
● Count
● Restart Configuration
● Network
● Volumes
● Service Checks
● Consul Connect
● Resource Limits
● Artifacts
● Templates
● Autoscaler Configuration

7.
Confidential and Proprietary
UX Pyramid

8.
Confidential and Proprietary
Nomad Pack
• Templating and Packaging tool
• Easily deploy popular applications to Nomad
• Re-use common patterns across internal applications
• Find and share job definitions with the Nomad community
• Jobs written in (H)ashiCorp (C)onfiguration (L)anguage
• Templates are written using Go Template Syntax.
• Nightlies only right now!
https://github.com/hashicorp/nomad-pack

9.
Confidential and Proprietary
Pack Registries
$ nomad-pack registry list
$ nomad-pack registry add o11y https://github.com/attachmentgenie/nomad-pack-o11y-registry
$ nomad-pack run grafana --var job_name=dashboard --registry=o11y
$ nomad-pack run packs/grafana -f vars/grafana.hcl –f vars/lab.hcl
https://github.com/hashicorp/nomad-pack-o11y-registry

10.
Confidential and Proprietary
Default Registry
$ nomad-pack registry list
PACK NAME | REF | METADATA VERSION | REGISTRY | REGISTRY URL
-----------------------------+--------+------------------+-----------------+-----------------------------
alertmanager | latest | 0.0.1 | default | github.com/hashicorp
aws_efs_csi | latest | 0.0.1 | default | github.com/hashicorp
mkdir –p $HOME/.nomad/packs/default on offline systems!

11.
Confidential and Proprietary
Pack Structure
lorem-ipsum ❯ tree |--
CHANGELOG.md
|-- README.md
|-- metadata.hcl
|-- outputs.tpl
|-- templates
| |-- _helpers.tpl
| `-- lorem-ipsum.nomad.tpl
`-- variables.hcl
1 directory, 7 files

12.
Confidential and Proprietary
metadata.hcl
app {
url = "https://grafana.com/"
author = "Grafana Labs"
}
pack {
name = "grafana"
description = "Grafana is a multi-platform open source analytics and interactive visualization tool."
url = "https://github.com/attachmentgenie/nomad-pack-o11y-registry/grafana"
version = "0.1.0"
}

13.
Confidential and Proprietary
variables.hcl
variable "datacenters" {
description = "A list of datacenters in the region which are eligible for task placement"
type = list(string)
default = [“dc1”]
}
Variable “resources” {
description = “The resource to assign to the Grafana service task”
type = object({
cpu = number
memory = number
})
default = {
cpu = 200,
memory = 256
}
}

14.
Confidential and Proprietary
Pack Templates
$ cat packs/grafana/templates/grafana.nomad.tpl
….
datacenters = [[ .my.datacenters | toStringList ]]
…
resources {
cpu = [[ .my.grafana_resources.cpu ]]
memory = [[ .my.grafana_resources.memory ]]
}
…
https://github.com/hashicorp/nomad-pack-community-registry

15.
Confidential and Proprietary
CI-CD
$ nomad-pack plan packs/loki --var version=vX.Y.Z -f vars/loki.hcl
+/- Job: "loki"
+ VaultToken: "s.IJcEJqpsCkGU0mfY3GmnCLSd"
+/- Task Group: "loki" (1 create, 2 in-place update)
+/- Count: "2" => "3" (forces create)
Task: "connect-proxy-loki" Task: "server"
» Scheduler dry-run:
- All tasks successfully allocated.
Plan succeeded
$ nomad-pack nomad-pack run packs/loki --var version=vX.Y.Z -f vars/loki.hcl

16.
Confidential and Proprietary
CI-CD Paranoid Version
$ nomad-pack render packs/loki --var version=vX.Y.Z -f vars/loki.hcl -o $WORKSPACE/render
$ nomad run $WORKSPACE/render/loki/loki.nomad
https://github.com/marketplace/actions/setup-hashicorp-nomad-pack

17.
Confidential and Proprietary
Nomad UI

18.
Confidential and Proprietary
Helper template
$ cat packs/grafana/templates/grafana.nomad.tpl
job [[ template "job_name" . ]] {
[[ template "region" . ]]
[[ template "namespace" . ]]
….
$ cat packs/grafana/templates/_helpers.tpl
…
[[- define "job_name" -]]
[[- if eq .grafana.job_name "" -]]
[[- .nomad_pack.pack.name | quote -]]
[[- else -]]
[[- .grafana.job_name | quote -]]
[[- end -]]
[[- end -]]
…

19.
Confidential and Proprietary
Abstracting away boring repetitive bits
$ cat packs/grafana/templates/_helpers.tpl
…
[[ define "resources" -]]
[[- $resources := . ]]
resources {
cpu = [[ $resources.cpu ]]
memory = [[ $resources.memory ]]
}
[[- end ]]
…
$ cat packs/grafana/templates/grafana.nomad.tpl
…
[[ template " resources " . ]]
…

20.
Confidential and Proprietary
Abstracting away boring repetitive bits
$ cat packs/grafana/templates/_resources.tpl
…
[[ define "resources" -]]
[[- $resources := . ]]
resources {
cpu = [[ $resources.cpu ]]
memory = [[ $resources.memory ]]
}
[[- end ]]
…
$ cat packs/grafana/templates/grafana.nomad.tpl
…
[[ template " resources " . ]]
…

21.
Confidential and Proprietary
Abstracting away boring repetitive bits
$ cat packs/grafana/metadata.hcl
…
dependency ”hashitalks_helpers" {
name = "hashitalks_helpers"
source = "https://github.com/attachmentgenie/hashitalks-registry/helpers"
}
$ cat packs/grafana/templates/grafana.nomad.tpl
…
[[ template "hashitalks_helpers .resources" . ]]
…

22.
Confidential and Proprietary
Wishlist: pre-commit-nomad
Currently no clear alternatives/equivalents for:
Terraform_docs
Terraform_fmt
Terraform_tflint
Terraform_validate
Terrascan

23.
Confidential and Proprietary
Wishlist: Locals
network {
mode = "bridge"
port "mysql" {
to = 3306 <- local.mysql_port
}
}
[[ if .my.register_consul_service ]]
service {
name = "[[ .my.consul_service_name ]]"
tags = [[ .my.consul_service_tags | toStringList ]]
port = "mysql"
connect {
sidecar_service {
tags = [""]
proxy {
local_service_port = 3306 <- local.mysql_port
…

24.
Confidential and Proprietary
Wishlist: Meta package support
$ cat deploy.sh
#!/bin/bash
set -e
nomad-pack run minio -f vars/minio.hcl -f vars/lab.hcl --registry=attachmentgenie
nomad-pack run packs/loki -f vars/loki.hcl -f vars/lab.hcl
nomad-pack run packs/mimir -f vars/mimir.hcl -f vars/lab.hcl
nomad-pack run packs/phlare -f vars/phlare.hcl -f vars/lab.hcl
nomad-pack run packs/tempo -f vars/tempo.hcl -f vars/lab.hcl
nomad-pack run packs/grafana -f vars/grafana.hcl -f vars/lab.hcl
nomad-pack run redis -f vars/redis.hcl -f vars/lab.hcl --registry=attachmentgenie
nomad-pack run packs/grafana_oncall -f vars/grafana_oncall.hcl -f vars/lab.hcl
nomad-pack run packs/prometheus -f vars/prometheus.hcl -f vars/lab.hcl
nomad-pack run packs/promlens -f vars/promlens.hcl -f vars/lab.hcl

25.
Confidential and Proprietary
Wishlist: Dependency health checks
$ cat deploy.sh
#!/bin/bash
set -e
export NOMAD_ADDR=http://192.168.1.30:4646/ui/jobs
wait-for-url() {
echo "Testing $1"
timeout -s TERM 45 bash -c
'while [[ "$(curl -s -o /dev/null -L -w ''%{http_code}'' ${0})" != "200" ]];
do echo "Waiting for ${0}" && sleep 2;
done' ${1}
echo "OK!"
}
nomad-pack run minio -f vars/minio.hcl -f vars/lab.hcl --registry=attachmentgenie
wait-for-url https://s3.teambla.dev/minio/health/live
nomad-pack run packs/loki -f vars/loki.hcl -f vars/lab.hcl

26.
Questions Before Takeoff?
bram@attachmentgenie.com
@attachmentgenie
https://www.slideshare.net/attachmentgenie