Slides from my meetup talks at various AWS and DevOps meetups.
Follow me:
https://twitter.com/antonbabenko
https://github.com/antonbabenko
https://linkedin.com/in/antonbabenko
Scanning the Internet for External Cloud Exposures via SSL Certs
Terraform AWS modules and some best practices - September 2019
1. Terraform AWS modules and some best-
practices
Anton Babenko
@antonbabenko
September 2019
2. Anton Babenko
AWS Community Hero / Certified Terraform fanatic since 2015
Organiser of HashiCorp UG, AWS UG, DevOps Norway, DevOpsDays Oslo
I 💚 open-source:
terraform-community-modules + terraform-aws-modules
antonbabenko/pre-commit-terraform — clean code and documentation
antonbabenko/tfvars-annotations — update terraform.tfvars using annotations
antonbabenko/modules.tf-lambda — generate Terraform code from visual diagrams
antonbabenko/terragrunt-reference-architecture — Terragrunt reference architecture
www.terraform-best-practices.com
medium.com/@anton.babenko
@antonbabenko — Twitter, GitHub, Linkedin
3. What do I do?
All-things Terraform + AWS + DevOps
Consulting
Workshops
Trainings
Mentorship
My interview: https://medium.com/@anton.babenko/my-terraform-aws-journey-hashitimes-interview-73d1b542fcc0
My email: anton@antonbabenko.com
LinkedIn: https://www.linkedin.com/in/antonbabenko
@antonbabenko
4. Collection of open-source Terraform AWS modules supported by the community.
More than 5 mil. downloads since September 2017.
(VPC, Autoscaling, RDS, Security Groups, ELB, ALB, Redshift, SNS, SQS, IAM, EKS, ECS…)
github.com/terraform-aws-modules
registry.terraform.io/modules/terraform-aws-modules
@antonbabenko
6. cloudcraft.co features
• Manage components in browser (EC2 instances, autoscaling groups, RDS, etc)
• Connect components
• Import live AWS infrastructure
• Calculate the budget
• Share link to a blueprint
• Export as image
• Embed drawing to wiki, Confluence, etc
@antonbabenko
7. Infrastructure as code makes DevOps possible
Key benefits:
• Treat infrastructure like application code
• Always know what changed
• Validate infrastructure before deployment
https://dzone.com/articles/infrastructure-as-code-the-benefits @antonbabenko
8. – What is infrastructure?
“Infrastructure is all what is needed to support the computing
requirements of an application.”
@antonbabenko
9. Evolution — from manual to automated
Maturity and complexity grows:
Manual at first
Ad-hoc scripts
Virtualisation
IaaS, SaaS, PaaS, *aaS
Datacenter as computer
@antonbabenko
12. Why Terraform and not AWS CloudFormation,
Azure ARM, Google Cloud Deployment Manager?
Terraform supports 250+ providers, has easier syntax (HCL), has native support for
modules and remote states, has teamwork related features, is an open-source project
Provides a high-level abstraction of infrastructure, unifies the view of resources
Supports the modern datacenter (IaaS, PaaS, SaaS)
Separates planning from execution (dry-run)
Provides a workflow which is technology agnostic
Manages anything with an API
@antonbabenko
18. Terraform — universal tool for everything with an API
Google G Suite
Dropbox files and access
New Relic metrics
Datadog users and metrics
Jira issues
Minecraft, or even order Domino’s pizza
All Terraform providers — https://www.terraform.io/docs/providers/index.html
@antonbabenko
19. Let’s begin — everything fits in main.tf
@antonbabenko
32. – What is Terraform module?
“Modules in Terraform are self-contained packages
of Terraform configurations that are managed as a group.”
@antonbabenko
40. Would you use Terraform module to manage AWS EC2
security group?
In 0.11 — Yes, of course, sure and certainly!
In 0.12 — Yes!
@antonbabenko
41. Infrastructure modules
Also known as "curated modules" and "company-wide modules"
Consist of resource modules
Enforce tags and company standards
In 0.11 — use preprocessors, jsonnet, cookiecutter
In 0.12 — may implement complex logic (conditions, loops, nested
blocks)
@antonbabenko
64. How to structure Terraform
configurations? How to call them?
@antonbabenko
65. Call Terraform modules
Use Terraform modules, because amount of resources and code is
increasing
How to organize Terraform configurations and invoke them?
How to orchestrate modules?
@antonbabenko
66. All-in-one
Good:
Declare variables and outputs in
fewer places
Bad:
Large blast radius
Everything is blocked at once
Impossible to specify
dependencies between modules
(depends_on)
@antonbabenko
71. All-in-one
Undefined project scope
Fast prototyping and initial
development phase
Small number of resources &
developers
Tightly connected resources
1-in-1
@antonbabenko
Defined project scope
Different types of developers
involved *
Code reuse is encouraged
(across organization and
environments)
Use Terragrunt
73. Problems with Terraform workspaces
Terraform Workspaces aren’t infrastructure-as-code friendly. You
can’t answer straight from the code:
"How many workspaces do you have?"
"What infrastructure has been deployed in workspaceX?"
"What is the difference between workspaceX and workspaceY?"
Introducing complexity almost in all cases.
@antonbabenko
74. Solution — use re-usable modules
instead of workspaces
@antonbabenko
75. What kind of orchestration method do you use?
-target
Makefile
…
@antonbabenko
85. before_hook + shell script Go binary
https://github.com/antonbabenko/tfvars-annotations — Update values in
terraform.tfvars using annotations (WIP)
or take a look at modules.tf
@antonbabenko
92. Edge cases
Different AWS regions (version of S3 signature, EC2 ClassicLink,
IPv6)
Date of creation of AWS account
Limits on resources in AWS
Services and features availability
@antonbabenko
93. Avoid in Terraform
Not secret arguments should not be specified as command line
arguments => put them in tfvars
Reduce usage of "-target" and "-parallelism"
"Terraform workspaces" evil in=> separate by directories
Dependency hell in modules
@antonbabenko
99. Terraform developers
Write and support Terraform modules
Implement company’s standards (security, encryption, integrations)
Maintain reference architectures
@antonbabenko
100. Terraform users (everyone)
Use Terraform modules by specifying correct values
Domain experts
May not have "Terraform" in LinkedIn profile
@antonbabenko
101. Terraform 0.12 for developers
DevOps&Terraform developers
Allow to implement flexible/dynamic/reusable Terraform modules
@antonbabenko
102. Terraform 0.12 for users
Terraform users
Like HCL2 lightweight syntax more
@antonbabenko
104. How to handle secrets in Terraform?
• Can you accept secrets to be saved in state file in plaintext? Probably not.
• AWS IAM password & access secret keys — use PGP as keybase.io
• AWS RDS — set dummy password and change after DB is created
• AWS RDS — use iam_database_authentication_enabled = true
• EC2 instance user-data + AWS KMS
• EC2 instance user-data + AWS System Manager’s Parameter Store
• AWS Secrets Manager
• https://github.com/opencredo/terrahelp
• https://secrethub.io/ - A simple tool to keep secrets out of .tfvars and .tfstate files
• Other options:
• Secure remote state location (S3 bucket policy, KMS key)
• Terraform Saas (Cloud or Enterprise)
@antonbabenko
105. What are the tools/solutions out there?
• Terraform Registry (https://registry.terraform.io/) — collection of public Terraform
modules for common infrastructure configurations for any provider.
• Terraform linter to detect errors that can not be detected by `terraform plan` —
https://github.com/wata727/tflint
• Terraform version manager — https://github.com/kamatama41/tfenv
• Terraform modules manager — https://github.com/devopsmakers/xterrafile
• A web dashboard to inspect Terraform States — https://github.com/camptocamp/
terraboard
• Jsonnet — The data templating language — http://jsonnet.org
• pre-commit-terraform — pre-commit git hooks to take care of Terraform
configurations —https://github.com/antonbabenko/pre-commit-terraform/
@antonbabenko
106. Atlantis — Start working on Terraform as a team
A unified workflow for collaborating on Terraform through GitHub, GitLab and Bitbucket
https://www.runatlantis.io
@antonbabenko
111. Some of Terraform Best Practices
https://www.terraform.io/
http://learn.hashicorp.com
https://www.terraform-best-practices.com/
+ ask me for more
@antonbabenko
117. ✓ cloudcraft.co — design, plan and visualize
✓ terraform-aws-modules — building blocks of AWS infrastructure
✓ Terraform — infrastructure as code
118. Infrastructure as code generator — from visual diagrams to Terraform
https://github.com/antonbabenko/modules.tf-lambda
Demo video: https://www.youtube.com/watch?v=F1Ax1zfZbiY
119. 1. Go to cloudcraft.co
2. Sign up, sign in (free account)
3. Draw your AWS infrastructure
4. Click "Export"
5. Click "Terraform code export"
Try it yourself!
120. modules.tf — generated code
✓ Potentially ready-to-use Terraform configurations
✓ Suits best for bootstrapping
✓ Enforces Terraform best-practices
✓ Batteries included (terraform-aws-modules, terragrunt, tfvars-
annotations, pre-commit)
✓ 100% free and open-source (https://github.com/antonbabenko/
modules.tf-lambda)
✓ Released under MIT license