Evolutionary infrastructure agile 2018 - kief morris
•
4 gefällt mir•641 views
This talk explores infrastructure design patterns to support continuous change to: - Reduce the “blast radius” for a given change - Make it easy to update, upgrade, and refactor systems without requiring massive, organization-wide programmes of change - Improve security, visibility, auditability, and observability of systems - Increase the number of people and teams able to work across systems while minimizing coordination overhead Topics covered include: - Designing and implementing an effective infrastructure test automation strategy - Creating change management pipelines that enforce rigorous change control processes while supporting rapid, frequent changes - Structuring infrastructure codebases to optimize for continuous change
Empfohlen
Empfohlen
Weitere ähnliche Inhalte
Was ist angesagt?
Was ist angesagt? (20)
Ähnlich wie Evolutionary infrastructure agile 2018 - kief morris
Ähnlich wie Evolutionary infrastructure agile 2018 - kief morris (20)
Kürzlich hochgeladen
Kürzlich hochgeladen (20)
Evolutionary infrastructure agile 2018 - kief morris
- 2. kief@thoughtworks.com Infrastructure Technologist@ Twitter: @kief Book: http://oreil.ly/1JKIBVe Site: http://infrastructure-as-code.com Some visuals found from the Vault of the Atomic Space Age http://bit.ly/2KBTbAd
- 3. Why? Ongoing pressure to deliver while requirements change
- 4. PRODTESTDEV Inconsistent path to production
- 6. PROD Continuous, reliable delivery of changes to all your stuff TESTDEV
- 7. "Since we can't avoid change, we need to exploit it" Building Evolutionary Architectures Neal Ford, Rebecca Parsons, Pat Kua
- 14. Slow Careful Careless Prioritize correctness over speed of change Fragile mess
- 15. Slow Careful Careless Prioritize correctness over speed of change Fragile mess Time pressures Over- engineering Technical debt Barriers to improvement
- 16. FastSlow Careful Careless Agile, Lean, DevOpsBuild the simplest thing Build quality in Optimize for change Fast, frequent feedback
- 18. Dynamic infrastructure platform Change delivery services Infrastructure management tools Operational management services Application Packaging and Deployment Development Tooling Application runtime platform
- 19. Dynamic infrastructure platform Change delivery services Infrastructure management tools Operational management services Application Packaging and Deployment Development Tooling Application runtime platform
- 20. Dynamic infrastructure platform Compute StorageNetwork A pool of resources
- 21. Provision resources • On demand • Programmatically
- 22. Physical Servers Containers Serverless (FaaS) Variations of compute resources Virtual Machines
- 23. Dynamic infrastructure platform Change delivery services Infrastructure management tools Operational management services Application Packaging and Deployment Development Tooling Application runtime platform
- 24. Stack management tool Stack instances Platform API Stack definition (Terraform, CloudFormation, ...)
- 25. Server configuration tool Configuration tool Configuration definitions Server instance (Ansible, Chef, Puppet, Saltstack, ...)
- 26. Server image creation Base server image Custom image Server configuration definitions Image definition (Packer, ...)
- 28. Infrastructure stack A collection of infrastructure resources provisioned and updated as a unit
- 29. A stack may include the infrastructure for a single application
- 30. Or, multiple applications might run in a single stack
- 31. Or the parts of one application could be spread across several stacks
- 32. Stack definition A stack definition is code used by a tool to provision stack instances Stack definition Stack instanceStack management tool
- 33. Stack instances A single stack definition may be used to manage one or more stack instances Stack definition Stack instances
- 35. Starting small – single stack instance One or two people working on it, ... the system is fairly simple, ... and there are few users, with low stakes
- 36. And then ... More people join the team, ... the system becomes more complex, ... and more people rely on the system to be working
- 38. PRODUCTION QA TEST Multiple environments defined in one stack instance our_env/ └── test.tf └── qa.tf └── production.tf
- 39. But the blast radius for a change is too wide PRODUCTION QA TEST
- 40. our_env/ └── test/ └── servers.tf our_env/ └── qa/ └── servers.tf our_env/ └── production/ └── servers.tf PRODUCTION QA TEST Separate stack definition project for each stack instance
- 41. QA PRODUCTION TEST Single definition project template, used to create a separate stack instance for each environment our_env/ └── servers.tf
- 43. Automatically test every change before applying it
- 44. Promote changes to environments using a pipeline BUILDLOCAL APPLY TO QA APPLY TO PROD Sandbox QA Production APPLY TO TEST Test
- 45. Stack tool is only run by pipeline agents BUILDLOCAL APPLY TO QA APPLY TO PROD APPLY TO TEST
- 46. Why not apply changes from our laptops? Our working environments may not be consistent Multiple people may clash We may shortcut our process We may neglect to run all appropriate tests
- 47. "Local" pre-commit sandboxes VCS Nita's Instance Alex's Instance Freddy's Instance
- 49. server "web" { name = "web-${env}" type = "${type}" } env = "prod" type = "t2.large" env = "qa" type = "t2.large" env = "test" type = "t2.tiny" web-test web-qa web-prod Parameters per stack instance
- 50. Parameters in source project folders web-test web-qa web-prod . ├── infra │ ├── web_server.tf │ ├── subnet.tf │ └── vpc.tf └── environments ├── test.tfvars ├── qa.tfvars └── prod.tfvars
- 51. env = "test" type = "t2.tiny" env = "qa" type = "t2.large" env = "prod" type = "t2.large" Parameters in pipeline job definitions BUILD APPLY TO QA APPLY TO PROD APPLY TO TEST web-test web-qa web-prod
- 52. Parameters from configuration registry web-test web-qa web-prod envs/test/type t2.tiny envs/qa/type t2.large envs/prod/type t2.large
- 53. Recommendation Minimize the configurable surface area to reduce variation between environments
- 55. An effective testing regime drives a clean, decoupled, evolvable system design
- 56. Fitness functions Define the important characteristics of your system, how to measure these, and how to assess progress
- 58. QA PRODTEST ALL TEST JDK COOKBOOK TEST TOMCAT COOKBOOK TEST APP SERVER ROLE BUILD APP SERVER AMI VALIDATE INFRA STACK BUILD OUR-APP TEST INFRA STACK TEST OUR- APP Example pipeline for our-app
- 59. QA PRODTEST ALL TEST JDK COOKBOOK TEST TOMCAT COOKBOOK TEST APP SERVER ROLE APP SERVER AMI VALIDATE INFRA STACK BUILD OUR-APP TEST INFRA STACK TEST OUR- APP APPLICATION STAGES: Compile, unit tests, container tests, packaging, etc. (run on build agents) JUnit, Rspec, etc.
- 60. QA PRODTEST ALL TEST JDK COOKBOOK TEST TOMCAT COOKBOOK TEST APP SERVER ROLE BUILD APP SERVER AMI BUILD INFRA STACK BUILD OUR-APP TEST INFRA STACK TEST OUR- APP SERVER PACKAGES: Validate, test individual package configurations in isolation SERVER ROLE: Test aggregated packages Typically run on build agents, perhaps containerized test-kitchen, serverspec, inspec, puppet-rspec, testinfra, etc.
- 61. QA PRODTEST ALL TEST JDK COOKBOOK TEST TOMCAT COOKBOOK TEST APP SERVER ROLE BUILD APP SERVER AMI VALIDATE INFRA STACK BUILD OUR-APP TEST INFRA STACK TEST OUR- APP SERVER IMAGE: Build and test serverspec, inspec, etc.
- 62. QA PRODTEST ALL TEST JDK COOKBOOK TEST TOMCAT COOKBOOK TEST APP SERVER ROLE BUILD APP SERVER AMI VALIDATE INFRA STACK BUILD OUR-APP TEST INFRA STACK TEST OUR- APP INFRASTRUCTURE STACK Validate (e.g. syntax, provision non-application specific infrastructure elements awspec, inspec, etc.
- 63. QA PRODTEST ALL TEST JDK COOKBOOK TEST TOMCAT COOKBOOK TEST APP SERVER ROLE BUILD APP SERVER AMI VALIDATE INFRA STACK BUILD OUR-APP TEST INFRA STACK TEST OUR- APP INTEGRATED STAGES: Test the various components in combination (deployment environments) selenium, etc.
- 64. Recommendations Those who write the code write the tests Avoid reflexive testing (AKA testing your tools) Test where there is risk, complexity Test contracts
- 66. Monolithic stack
- 67. Coordinating changes Team A could make a change that breaks things for team B
- 68. Divide infrastructure into multiple, independently changeable stacks
- 69. Each stack has its own pipeline to deliver changes
- 71. Draw boundaries to minimize changes that cross stacks
- 72. Infrastructure code boundaries != network boundaries
- 73. Infrastructure code boundaries != network boundaries
- 74. Infrastructure code boundaries != network boundaries
- 75. Group things together when they tend to be changed together
- 76. Align team boundaries and change boundaries
- 79. Base infrastructure module pipeline Application stack pipelines Components shared across stacks
- 80. Challenges with shared components Increases coupling
- 82. Shared stack instance Shared network resource stack Multiple application infrastructure stacks
- 83. Shared stack pipeline Application infrastructure pipelines
- 84. Shared infrastructure stacks The provider stack should not have knowledge of its consumers Becomes a bottleneck as the number of consumers grows Should be kept simple, minimal Split into smaller stacks to avoid monoliths
- 86. Each stack is self-contained
- 87. Each stack is independently releasable
- 89. Example of integrated stacks Shared infrastructure stack Application infrastructure stack vpc_id = "vpc-abcde" private_subnets = ["sn-12345", "sn-54321"]
- 90. Integration at apply-time vpc_id = "vpc-abcde" private_subnets = ["sn-12345", "sn-54321"]
- 91. Integration by resource discovery data "aws_vpc" "vpc" { filter { name = "tag:Environment" values = ["${var.env}"] } } data "aws_subnet" "private_subnet" { filter { name = "tag:Environment" values = ["${var.env}"] } }
- 92. Integration by configuration registry envs/qa/vpc_id vpc-abcde envs/qa/private_subnets "sn-12345", "sn-54321"
- 93. Integration by remote state
- 94. Recommendations Stack integration points are contracts Follow good practice for API design when making changes to contracts Use fitness functions (automated tests) to ensure the integrity of contracts
- 95. kief@thoughtworks.com Infrastructure Technologist@ Twitter: @kief Book: http://oreil.ly/1JKIBVe Site: http://infrastructure-as-code.com