Are you terraforming or clickopsing your data infrastructure? From spinning up virtual machines to managing application infrastructure across cloud regions, infrastructure-as-code tools have been widely used and adopted. Are you treating your data infrastructure in a similar way or still using complex scripts to create and manage your data infrastructure? Terraform, an open-source infrastructure as code tool by HashiCorp, can be used to provision data infrastructure across multiple clouds. In this talk, I will discuss the need for an infrastructure-as-code approach for databases and streaming platforms. A large portion of the talk will consist of a demo that walks the audience through creating multiple data services using Terraform for a real use case. The demo will show the benefits of using such a tool over creating these services manually. By attending the talk, the audience will understand the benefits of codifying the data infrastructure and walk away with the resources to be able to try Terraform out for their own data infrastructure.

1. Do NOT click-ops your
data infrastructure
Dewan Ahmed

2. Hi, I’m Dewan
• Senior Developer Advocate, Aiven
• New Brunswick, Canada
• Focus on app/data infrastructure
• Pro bono career coach
in/diahmed | @DewanAhmed

3. A story of clicks…

4. Application Infrastructure
in/diahmed | @DewanAhmed
@aiven_io

5. Why do we build applications?
To move data
@aiven_io in/diahmed | @DewanAhmed

6. Data Infrastructure
in/diahmed | @DewanAhmed
@aiven_io

7. Infrastructure-as-Code Principles
•Reproducibility
•Repeatability
•Disposability
•Consistency
•Ability to incorporate design changes
in/diahmed | @DewanAhmed
More in the book:
Infrastructure as Code (Kief Morris)
@aiven_io

8. Infrastructure-as-Code Challenges
• Resistance to learning
• Con
fi
guration drift
• Duplicate work
• Security issues*
• Handling unicorns
in/diahmed | @DewanAhmed
*Source: https://bridgecrew.io/blog/state-of-open-source-terraform-security-report-2020
@aiven_io

9. Handling Unicorns - Databases
• Perform version upgrade
• Prevent version upgrade
• Change con
fi
gurations
in/diahmed | @DewanAhmed
How to IaC without downtime
@aiven_io

10. in/diahmed | @DewanAhmed
@aiven_io

11. Aiven Terraform Provider
in/diahmed | @DewanAhmed
Source: https://www.terraform.io/intro
@aiven_io

12. in/diahmed | @DewanAhmed
Reproducibility
@aiven_io

13. in/diahmed | @DewanAhmed
Repeatability
Consistency
@aiven_io

14. in/diahmed | @DewanAhmed
Apache Kafka® and Apache Kafka® MirrorMaker 2
@aiven_io

15. DEMO!
in/diahmed | @DewanAhmed

16. But… How do I start?
• Research - is this right for us?
• Awareness - are we ready for this?
• Principles - Do you know the fundamentals?
• Pilot - Let’s get something working
• All-in Strategy - Roll-out where it makes sense
in/diahmed | @DewanAhmed
@aiven_io

17. Recap
•Infrastructure-as-Code (IaC) principles
•Challenges and concerns - IaC with
data infrastructure
•How to get started in your IaC journey?
in/diahmed | @DewanAhmed
@aiven_io

18. • Aiven Terraform Provider: https://developer.aiven.io/docs/tools/
terraform.html
• Demo: https://developer.aiven.io/docs/tools/terraform/reference/
cookbook/kafka-mirrormaker-recipe.html
• IaC Blog: https://www.dewanahmed.com/iac-principles-patterns
References:
in/diahmed | @DewanAhmed
@aiven_io

19. Questions?
dewan@aiven.io
in/diahmed | @DewanAhmed
@aiven_io