В Dev-Pro DevOps-специалисты работают с Terraform в рамках Azure. Команда работает с множеством окружений и ресурсов, среди которых есть AKS (Kubernetes). Сергей поделится опытом успешного написания модулей и провайдеров для Terraform.

1. Terraform
make some simple, readable, reusable
code and don't commit a suicide
a novel about modules, providers, security, and pain
April 6, 2019

2. Who am I?
Sergii Marchenko
Head of IT at Dev-Pro
More than 10 years in IT
Loves Terraform, and PowerShell :))
Knows a bit about DevOps
Thinks he can write some code in Go
Email: sergii.marchenko@dev-pro.net
Skype: dev-pro.sergii.marchenko

5. Good

6. TF is good
● Well documented (code is a configuration guideline)
● Clear change management (version control)
● Reusable (dev, stg, prod)
● Not only for a small team, works for 10+ DevOps
● The best way to implement Immutable infrastructure approach
● Fast (hey, Ansible)

7. Reusable
● Test
● Dev
● QA
● Automation
○ AQA Development
○ Integration tests
○ Performance tests
● Demo
● Staging
● Prod

8. Modules
1. DRY
2. Reusable
3. Versioning and smooth updates
4. Roll back is more or less simple
5. You see all changes

9. Bad

10. The state file
1. Security
2. More security!!!
3. Backups of the state file

11. What if I already have some envs?
1. Import does NOT generate TF code
2. If your setup is complicated (local-exec, API provider) you can NOT import
that

12. If in TF is a joke
CONDITION ? TRUEVAL : FALSEVAL
resource "aws_instance" "web" {
subnet = "${var.env == "production" ? var.prod_subnet : var.dev_subnet}"
}
What if I have Dev, QA, Stg, Prod?

13. Sometimes it’s hard to understand
resource "aws_eip" "example" {
count = "${var.create_eip}"
instance = "${aws_instance.example.id}"
}
resource "aws_route53_record" "example" {
count = "${1 - var.create_eip}"
zone_id = "A1B2CDEF3GH4IJ"
name = "foo.example.com"
type = "A"
ttl = 300
records = ["${aws_instance.example.public_ip}"]
}

14. Or this one
depends_on = ["azurerm_network_security_group.AKS-security-group"]
depends_on = ["azurerm_subnet.AKS-subnet"]

15. Backend
Interpolation is NOT supported.
terraform {
backend "s3" {
bucket = "${var.env_name}-state"
key = "state.tfstate"
}
}
Our current recommendation is to treat Terraform -- and thus the Terraform states
-- as something "outside" the environments they manage, rather than as part of
the environment.

16. Count in modules
module "my-awesome-app" {
source = "../my-module"
name = "Prod-VM"
count = 2
}
Count does NOT work in modules

17. Acceptance

18. Why?
1. In most cases it is easy to understand
2. Fast (Hi Ansible)
3. Declarative
4. Count
5. Modules, Modules, Modules

19. Our vision

20. No manual actions!
1. No manual actions
2. No, you can't create a tiny resource manually
3. Yes, it matters
4. No, there are no exceptions to the rule
5. Yes, local-exec is better than manual actions

21. Use Hashi Vault for secrets
1. Integration with AD (SSO)
2. Vault provider out of the box
3. RBAC is flexible
4. Supports interpolation in secret path

22. Use Hashi Vault instead of remote backend
1. Supports interpolation in secret path
2. Can save and get required data in secure way

23. Use Hashi Vault instead of remote backend

24. Use Hashi Vault instead of remote backend
resource "vault_generic_secret" "AKS_Ingress_IP" {
path = "${var.hashivault_root_path}/Global/AKS/${var.cluster_name}/Ingress"
data_json = <<EOT
{
"ingress_public_ip": "${data.kubernetes_service.k8s_cluster.load_balancer_ingress.0.ip}"
}
EOT
}
data "vault_generic_secret" "AKS_Ingress_IP" {
path = "${var.hashivault_root_path}/Global/AKS/${var.cluster_name}/Ingress"
}

25. Keys structure

26. Keys structure

27. Keys structure

28. How to store states
1. Storage account with firewall rules and VPN (+MFA)
2. We have to rotate access keys (one by one)
3. Different storage accounts for different ENVs
4. Go wrapper. We call it init.

29. Git structure, files structures
Demo

30. Pull requests
1. 1-2 people who can review and approve a PR
2. Pull request validation

31. Validate pull requests

32. Terraform tests
1. Use QA automation team
2. If you don’t have it, terratest works as well

33. Terraform is about immutable infrastructure
1. PaaS services
2. Deploy containers or images
3. If you have to run remote-exec, use Ansible :)

34. TIPS

35. TF tips
BAD
depends_on = ["azurerm_network_security_group.AKS-security-group"]
depends_on = ["azurerm_subnet.AKS-subnet"]
GOOD
depends_on = ["azurerm_network_security_group.AKS-security-group","azurerm_subnet.AKS-subnet"]
THE BEST
depends_on = [
"azurerm_network_security_group.AKS-security-group",
"azurerm_subnet.AKS-subnet"
]

36. If you don’t have a required provider, use restapi
provider "restapi" {
uri = "https://api.sendgrid.com"
username = "securrency_test"
password = "**************"
debug = true
id_attribute = "api_key_id"
create_returns_object = true
}
resource "restapi_object" "sgkey" {
path = "/v3/api_keys"
data = "{ "name": "Dev-Pro Test Terraform API key creation", "scopes": ["alerts.read"] }"
}
https://github.com/Mastercard/terraform-provider-restapi

37. Or just write your own
Yes, just write it
https://www.terraform.io/docs/extend/writing-custom-providers.html

38. How to write a provider
func resourceServer() *schema.Resource {
return &schema.Resource{
Create: resourceServerCreate,
Read: resourceServerRead,
Update: resourceServerUpdate,
Delete: resourceServerDelete,
Schema: map[string]*schema.Schema{
"address": &schema.Schema{
Type: schema.TypeString,
Required: true,
},
},
}
}

39. API requests

40. Q/A