Building a CI/CD pipeline with Concourse and terraform.
Resources:
- concourse-pipelines - Collection of concourse pipelines including terraform-pr and terraform-commit pipeline https://github.com/cesar-rodriguez/concourse-pipelines
- terrascan - Static code analysis of terraform templates. https://github.com/cesar-rodriguez/terrascan
- hello-hug - Example terraform project that uses concourse-pipelines https://github.com/cesar-rodriguez/hello-hug
18. Resources
• concourse-pipelines - Collection of concourse pipelines
including terraform-pr and terraform-commit pipeline
https://github.com/cesar-rodriguez/concourse-pipelines
• terrascan - Static code analysis of terraform templates.
https://github.com/cesar-rodriguez/terrascan
• hello-hug - Example terraform project that uses
concourse-pipelines
https://github.com/cesar-rodriguez/hello-hug
Hinweis der Redaktion
How many of you are familiar with terraform?
- Declarative language. Describe what the target state and terraform figures API calls
- Version control
- Preview any changes
- Consistent infrastructure across all environments
Terraform workflow from your desktop
Writes templates into a canonical format.
Templates look clean and consistent.
- Download terraform providers binaries
- Downloads any modules
- Terraform remote states
Check terraform templates against your terraform state file and calculates any changes to resources in your templates.
Execute any changes from the terraform plan into AWS.
Collaboration challenges. Native features:
Remote states
State locking
Challenges with native workflow:
Reviewing Pull Requests
No automated testing in this workflow, manual inspection.
No guarantee that GitHub reflects what’s in production
Credentials to AWS environments in our desktop
No central place to verify testing was completed. No central audit trail
What is Concourse?
Declarative YAML templates to design pipelines
UI to view the pipeline’s workflow
Easy to extend its functionality to solve the challenges faced with terraform provisioning at scale.
Docker container define tasks within your pipeline. Ensuring repeatability and consistency
Integration with GitHub OAuth for authentication/authorization.
You can limit access to your pipeline and its secrets only to members of your team in GitHub.
Native integration with Hashicorp Vault for pipeline secrets.
Secrets are only retrieved at time of use by Concourse and are never persisted.
There are 4 different concepts within Concourse that define a pipeline.
Pipeline - declarative YAML template where you define the inputs and outputs of your CI/CD tasks.
Jobs - which are a collection of tasks that form our build plans.
For terraform provisioning, we’ll have 3 different jobs. 1 for building our infrastructure, 1 for testing the terraform templates, and 1 for provisioning the resources into AWS.