Video and slides synchronized, mp3 and slide download available at URL http://bit.ly/2be0JZB. Kief Morris explains how a team can implement a change management pipeline to create a fast, reliable process for building and maintaining a testing and hosting infrastructure for their microservices-based system. He presents a hypothetical application team, and walks through the creation of a cloud-based infrastructure using automation tools such as Packer, Terraform, and Ansible. Filmed at qconnewyork.com. Kief Morris is Cloud Practice Lead at ThoughtWorks. He works with organisations to understand how to take advantage of cloud, infrastructure automation, DevOps, and Continuous Delivery to become more effective at delivering IT services.

1. INFRASTRUCTURE
AS CODE

2. InfoQ.com: News & Community Site
• 750,000 unique visitors/month
• Published in 4 languages (English, Chinese, Japanese and Brazilian
Portuguese)
• Post content from our QCon conferences
• News 15-20 / week
• Articles 3-4 / week
• Presentations (videos) 12-15 / week
• Interviews 2-3 / week
• Books 1 / month
Watch the video with slide
synchronization on InfoQ.com!
https://www.infoq.com/presentations/
infrastructure-as-code-2016

3. Presented at QCon New York
www.qconnewyork.com
Purpose of QCon
- to empower software development by facilitating the spread of
knowledge and innovation
Strategy
- practitioner-driven conference designed for YOU: influencers of
change and innovation in your teams
- speakers and topics driving the evolution and innovation
- connecting and catalyzing the influencers and innovators
Highlights
- attended by more than 12,000 delegates since 2007
- held in 9 cities worldwide

4. kief@thoughtworks.com
Cloud Practice Lead (UK)
DevOps, Continuous Delivery, Agile Ops
Twitter: @kief
Book: http://oreil.ly/1JKIBVe
Site: http://infrastructure-as-code.com
June 2016

5. AGENDA
CONTEXT
● Motivations
● Challenges
INFRASTRUCTURE AS CODE
● Key Practices
● Simple Pipeline
● Scaling Pipelines

6. SPEED
Get something to
market quickly
Iterate it
Continuously
improve it

7. TECHNOLOGY
Cloud,
automation, etc.
lowers barriers
for making
changes

8. DANGER
Security
Performance
Stability
Compliance
Maintainability

9. SECRET
High quality
services rely on
the ability to
make changes
quickly

10. GOAL
Be able to make
changes
Rapidly,
Frequently,
and Responsibly

11. CHALLENGES

12. SERVER SPRAWL
Creating new servers is the easy part

13. CONFIGURATION DRIFT
Servers start
out identical
But changes
accumulate
over time

14. AUTOMATION FEAR CYCLE

15. INFRASTRUCTURE
AS CODE
“Applying software engineering tools and practices to
infrastructure”

16. UNATTENDED AUTOMATION
Tools run on a
schedule to apply,
re-apply, and update
configuration
BENEFITS OF UNATTENDED:
● Discover problems quickly
● Force yourself to fix those
problems
● Force yourself to improve
your tools and processes
● Discourages “out of band”
changes

17. AUTOMATE SERVER UPDATES
Automation isn’t just for new servers!
Configuration
synchronization
Run Chef, Puppet, Ansible, etc. on a
schedule
Immutable servers Apply changes by rebuilding
servers
Containerized
servers
Apply changes by deploying new
container instances

18. RE-USE & PROMOTE DEFINITIONS
Re-use the same definition files across environments for
a given application or service
DEV STAGE PROD
Playbooks,
Cookbooks,
Manifests,
templates, etc.

19. TEST INFRASTRUCTURE CHANGES
Preventing DevOops
INFRA
TEST
DEV
TEST
PROD

20. PIPELINES
Using Continuous Delivery pipelines to manage
infrastructure

21. WHAT?
Terraform,
Puppet, etc.
Changes are
made and
committed to
VCS
Tools are run on
agents to apply
changes to
environments
Changes are only
promoted after
passing tests &
authorization

22. WHY?
Validates changes
to infrastructure
before applying
them to production
Confidence for
frequent, small
improvements to
infrastructure
Limit direct
changes to
infrastructure

23. TESTING
Correctness
Security policies
Performance
Stability

24. GOVERNANCE
The process for applying changes is auditable
Changes can be traced back to commits
Automation ensures processes are followed
Authorization can be required as needed

25. SIMPLE
An example with a fairly simple
environment

26. VPC
Subnet
10.0.0.0/16
Security Group
1.1.1.0/16 -> :443
DEFINING A SIMPLE ENVIRONMENT
ANSIBLE PLAYBOOK
Server configuration
TERRAFORM FILE
Environment structure
APPLICATION SOURCE
Deployable application

27. SIMPLE PIPELINE DESIGN
BUILD
STAGE
TEST
STAGE
QA STAGE
PROD
STAGE
Application Ansible Terraform
Deploy application, configuration, and infrastructure

28. SCALING
Handling more complex infrastructure

29. ALIGN INFRASTRUCTURE DESIGN TO
TEAMS
Ensure teams can
make the changes
they need easily
and safely

30. COMPLEX ENVIRONMENTS
Infrastructure involving multiple teams

31. FAN-IN PIPELINE
ServiceA
ServiceB
ServiceC
SYSTEM
TEST
QA PROD
BUILD
BUILD
BUILD
SERVICE
TEST
SERVICE
TEST
SERVICE
TEST

32. DECOUPLED PIPELINES
ServiceA
TESTBUILD
ServiceB
TESTBUILD
ServiceC
TESTBUILD
QA PROD
QA PROD
QA PROD

33. DEPENDENCIES
ServiceA
TESTBUILD
ServiceB
QA PROD
TESTBUILD QA PROD
Create test
instance of
provider
Implement
Consumer
Driven Contract
(CDC) Tests
Use mocks
and stubs

34. ISSUE: DUPLICATION
Multiple teams using similar systems, e.g. database clusters

35. RE-USE BY FORKING DEFINITIONS
Disadvantages:
- Divergence and
Inconsistency
Advantages:
- Avoid tight coupling
- Handles diverse requirements

36. RE-USE WITH DEFINITION LIBRARIES
Challenges:
- Avoid tight coupling, so teams
aren’t blocked when making
changes
- Ownership of code shared by
multiple teams
Guidance:
- Use separate pipelines for
each
- Use CDC & other dependency
testing strategies

37. LIBRARY PIPELINE
Server AMI
BUILD
Service
TEST
BUILD TEST
Test shared definitions before pulling them into dependent
pipelines

38. ISSUE: SHARED ELEMENTS
LOAD BALANCER
ServiceA
VIP
ServiceB
VIP
Shared
infrastructure
definitions
Service-specific
infrastructure
definitions

39. SHARING ELEMENTS
Avoid monoliths - optimize to simplify making changes

40. OUTCOMES ● Quickly provision and
evolve infrastructure
● Effortlessly roll out fixes
● Keep systems consistent
and up to date
● Spend time on high value
work

41. Book: http://oreil.ly/1JKIBVe
Site: http://infrastructure-as-code.com
Twitter: @kief
kief@thoughtworks.com
Cloud Practice Lead (UK)
DevOps, Continuous Delivery, Agile Ops

42. Watch the video with slide
synchronization on InfoQ.com!
https://www.infoq.com/presentations/
infrastructure-as-code-2016