Diese Präsentation wurde erfolgreich gemeldet.
Wir verwenden Ihre LinkedIn Profilangaben und Informationen zu Ihren Aktivitäten, um Anzeigen zu personalisieren und Ihnen relevantere Inhalte anzuzeigen. Sie können Ihre Anzeigeneinstellungen jederzeit ändern.
MATT LONG
TESTING PROGRAMMABLE
INFRASTRUCTURE
PROGRAMMABLE
INFRASTRUCTURE IS GREAT, BUT
WE'RE MISSING SOMETHING.
TESTING.
I'M A TESTER
HELLO, I'M MATT
I WORK HERE ↑
I AM NOT A
SYSADMIN
WHAT IS
PROGRAMMABLE
INFRASTRUCTURE?
TESTING PROGRAMMABLE INFRASTRUCTURE
THE APPLICATION OF METHODS AND TOOLING
FROM SOFTWARE DEVELOPMENT TO
MANAGEMENT OF IT I...
TESTING PROGRAMMABLE INFRASTRUCTURE
EXAMPLES OF PROGRAMMABLE INFRASTRUCTURE
▸ Automated provisioning & configuration
▸ Confi...
TESTING PROGRAMMABLE INFRASTRUCTURE
TOOLING EXAMPLES
PROGRAMMABLE
INFRASTRUCTURE
IS AWESOME!
Credit: Vault Boy, Bethesda Softworks
IT'S
FAST!
IT'S
AUTOMATIC!
IT'S ALL
CODE!
BUT IT GETS
COMPLEX
TESTING IS USED TO
MITIGATE COMPLEXITY
& RISK
BUT TESTING IS RARE
Credit: Gunshow, KC Green
TESTING PROGRAMMABLE INFRASTRUCTURE
WHAT I'M GOING TO TALK ABOUT
▸PART 1: Testing a cloud broker
▸PART 2: Building a Kuber...
TESTING A
CLOUD BROKER
AN INFRASTRUCTURE HEAVY PRODUCT
THE PROBLEM
TESTING PROGRAMMABLE INFRASTRUCTURE
WE WANT TO MOVE TO THE CLOUD...
BUT WE'RE WARY OF LOCK IN
Large organisation
TESTING PROGRAMMABLE INFRASTRUCTURE
USERS
USE MULTIPLE CLOUD PROVIDERS
TESTING PROGRAMMABLE INFRASTRUCTURE
PROBLEMS
▸ Different interfaces, feature sets & lingo
▸ Can't switch easily
▸ Spending...
TESTING PROGRAMMABLE INFRASTRUCTURE
USERS
CLOUD BROKER
TESTING PROGRAMMABLE INFRASTRUCTURE
BENEFITS
▸ Quick, easy provisioning
▸ one team previously took 3 months
▸ Common inter...
THIS IS A REALLY
COMPLICATED
APPLICATION
TESTING PROGRAMMABLE INFRASTRUCTURE
TESTING PROGRAMMABLE INFRASTRUCTURE
WORKFLOW
▸ Log into Web UI
▸ Fill in information about environment
▸ Broker creates an...
TESTING PROGRAMMABLE INFRASTRUCTURE
WEB TESTING
▸ Log into Web UI
▸ Fill in information about environment
TESTING PROGRAMMABLE INFRASTRUCTURE
???
▸ Broker creates and bootstraps resources
▸ SSH into resources
HOW DO YOU TEST
INFRASTRUCTURE?
TESTING PROGRAMMABLE INFRASTRUCTURE
WHAT TO TEST?
Do our deployment 

scripts work?
Does the VPN server work?
Can instance...
THIS SEEMS
FAMILIAR..
TESTING PROGRAMMABLE INFRASTRUCTURE
Does the VPN box work?

Can I SSH into a server?
Do our deployment scripts work?
Are s...
TOOLING
TESTING PROGRAMMABLE INFRASTRUCTURE
TOOLS AVAILABLE
▸ Bats
▸ ShUnit2
▸ Goss
▸ ServerSpec / Inspec / TestInfra
▸ Test Kitch...
UNIT TESTING
TESTING PROGRAMMABLE INFRASTRUCTURE
BATS
▸ "Bash Automated Testing
System"
▸ Unit testing for bash
▸ Like JUnit
TESTING PROGRAMMABLE INFRASTRUCTURE
SH UNIT 2
▸ Shell unit testing framework
▸ Runs on all Bourne shells
▸ sh, BASH, DASH,...
INTEGRATION TESTING
OR: SERVER VALIDATION
TESTING PROGRAMMABLE INFRASTRUCTURE
GOSS
▸ Go based
▸ Specs in YAML
▸ Minimal, fast, and simple
▸ Some neat features
▸ .. ...
TESTING PROGRAMMABLE INFRASTRUCTURE
SERVERSPEC
▸ Server based assertions
▸ Ruby/RSpec based
▸ Probably the most famous
▸ C...
TESTING PROGRAMMABLE INFRASTRUCTURE
INSPEC
▸ Written & maintained by Chef
▸ Very similar to ServerSpec
▸ Different feature...
TESTING PROGRAMMABLE INFRASTRUCTURE
TESTINFRA
▸ ServerSpec, but in Python
TEST
HARNESS
TESTING PROGRAMMABLE INFRASTRUCTURE
TEST KITCHEN
▸ Orchestrates setup, test, teardown
▸ Runs BATS, shUnit2, RSpec,
Servers...
OUR
SOLUTION
TESTING PROGRAMMABLE INFRASTRUCTURE
USERS
CLOUD BROKER
TESTING PROGRAMMABLE INFRASTRUCTURE
USERS
WEB TEST FRAMEWORK
TESTING PROGRAMMABLE INFRASTRUCTURE
USERS
INFRASTRUCTURE TEST FRAMEWORK
TESTING PROGRAMMABLE INFRASTRUCTURE
USERS
WEB TESTS
https://github.com/opencredo/test-automation-quickstart
TESTING PROGRAMMABLE INFRASTRUCTURE
INFRASTRUCTURE TESTS
Serverspec
TESTING PROGRAMMABLE INFRASTRUCTURE
INFRASTRUCTURE TESTING STACK
/ Serverspec
???
TESTING PROGRAMMABLE INFRASTRUCTURE
WHY RUBY?
▸ Fantastic testing community
▸ More suitable for SSHing into boxes
▸ "Win R...
TESTING PROGRAMMABLE INFRASTRUCTURE
SERVERSPEC SMOKE TESTS
▸ Run before everything else
▸ Really quick
▸ Catches obvious e...
TESTING PROGRAMMABLE INFRASTRUCTURE
SERVERSPEC EXAMPLE
describe package('jenkins') do
it { should be_installed }
end
descr...
TESTING PROGRAMMABLE INFRASTRUCTURE
Background:

Given environment has been created

And the following user details:

| us...
TESTING PROGRAMMABLE INFRASTRUCTURE
Background:

Given environment has been created

And the following user details:

| us...
TESTING PROGRAMMABLE INFRASTRUCTURE
Background:

Given environment has been created

And the following user details:

| us...
TESTING PROGRAMMABLE INFRASTRUCTURE
Background:

Given environment has been created

And the following user details:

| us...
TESTING PROGRAMMABLE INFRASTRUCTURE
UNDER THE CUCUMBER, PLAIN RUBY
Then(/^I should be able to echo "([^"]*)"$/) do |text|
...
THOUGHTS
TESTING PROGRAMMABLE INFRASTRUCTURE
THE GOOD
▸ Specialised tests for each layer
▸ Really quick, expressive
ServerSpec test...
TESTING PROGRAMMABLE INFRASTRUCTURE
THE BAD
▸ Over reliance on acceptance
tests
▸ Awkward switching between
two suites
▸ O...
TESTING PROGRAMMABLE INFRASTRUCTURE
THE UGLY
▸ Starting infrastructure is SLOW.
▸ It's expensive...
IT WAS WORTH IT
DESPITE ALL THAT
BUILDING A
KUBERNETES CLUSTER
APPLYING TDD TO INFRASTRUCTURE
INTERNAL DEVOPS
TRAINING COURSE
I LEARNED A LOT!
Credit: The Simpsons, Fox
TESTING PROGRAMMABLE INFRASTRUCTURE
BUILD THIS:
WITH THESE:
TESTING PROGRAMMABLE INFRASTRUCTURE
NOT A STRAIGHTFORWARD TASK
TESTING PROGRAMMABLE INFRASTRUCTURE
BUT HOW TO TEST IT?
▸ This is a dev activity
▸ Want fast feedback
▸ Complexity is miti...
TESTING PROGRAMMABLE INFRASTRUCTURE
▸ Provisions cloud infrastructure
▸ Declarative files
▸ Some support for variables
TERR...
TESTING PROGRAMMABLE INFRASTRUCTURE
TERRAFORM COMMANDS
▸ terraform plan
▸ Tells you what will change
▸ terraform apply
▸ A...
TESTING PROGRAMMABLE INFRASTRUCTURE
TERRAFORM FILE EXAMPLE
resource "aws_instance" "etcd-node" {
count = 3
ami = "ami-7abd...
TESTING PROGRAMMABLE INFRASTRUCTURE
LINT WITH 'TERRAFORM VALIDATE' COMMAND
Omitting a variable:
TESTING PROGRAMMABLE INFRASTRUCTURE
BUT IT DOESN'T CATCH ALL PROBLEMS
Duplicate subnet CIDRS:
TESTING PROGRAMMABLE INFRASTRUCTURE
LINTING ISN'T ENOUGH
▸ Devs don’t just rely on compilers
▸ We need something more
powe...
UNIT TESTING
TESTING PROGRAMMABLE INFRASTRUCTURE
TERRAFORM_VALIDATE
▸ Python based unit testing
▸ NOT to be confused with 'validate' co...
TESTING PROGRAMMABLE INFRASTRUCTURE
TERRAFORM_VALIDATE FORK
OC has forked the terraform validate repo
https://github.com/o...
INTEGRATION
TESTING
TESTING PROGRAMMABLE INFRASTRUCTURE
GOSS
▸ Easy to get up and running
▸ Doesn’t support remote
# example usage: ./goss-tes...
TESTING PROGRAMMABLE INFRASTRUCTURE
▸ Terraform compatibility
▸ Already a talk about this
▸ “Untangling Infrastructure Cod...
TESTING PROGRAMMABLE INFRASTRUCTURE
TestCreate Config Destroy
]TEST KITCHEN MANAGES YOUR TEST LIFECYCLE
TESTING PROGRAMMABLE INFRASTRUCTURE
TestCreate Config Destroy
TEST KITCHEN DOESN'T SUPPORT MULTIPLE PROVISIONERS
TESTING PROGRAMMABLE INFRASTRUCTURE
TEST KITCHEN DOESN'T SUPPORT MULTIPLE PROVISIONERS
https://github.com/test-kitchen/tes...
TESTING PROGRAMMABLE INFRASTRUCTURE
TERRAFORM 'NULL RESOURCE'
resource "null_resource" "ansible" {
triggers {
instance_ids...
TESTING PROGRAMMABLE INFRASTRUCTURE
TestCreate
Config
Destroy
THOUGHTS
TESTING PROGRAMMABLE INFRASTRUCTURE
THE GOOD
▸ Tooling exists!
▸ You can totally get a test
suite working
Credit: Overwatc...
TESTING PROGRAMMABLE INFRASTRUCTURE
THE BAD
▸ Unit testing extremely immature
▸ Tools immature in general
Credit: Overwatc...
TESTING PROGRAMMABLE INFRASTRUCTURE
THE HACKY
▸ Be prepared to hack
▸ It might not even be possible
Credit: Overwatch, Bli...
THIS IS BRAND
NEW GROUND
REMEMBER:
TESTING TOOLS
DEPEND ON YOUR
STACK
CONCLUSIONS
TESTING IS
IMPORTANT
BUT OFTEN IGNORED
TESTERS AND OPS
SHOULD WORK TOGETHER
WE NEED TO GET OUT OF
OUR COMFORT ZONES
TOOLS EXIST
BUT BE PREPARED
TO HACK
FINALLY...
TESTING PROGRAMMABLE INFRASTRUCTURE
THE APPLICATION OF METHODS AND TOOLING
FROM SOFTWARE DEVELOPMENT TO
MANAGEMENT OF IT I...
TESTING IS A SOFTWARE
DEVELOPMENT METHOD
WE SHOULD APPLY IT TO
INFRASTRUCTURE
THANKS
QUESTIONS?
@burythehammer
matt.long@opencredo.com
Nächste SlideShare
Wird geladen in …5
×

London Hashicorp Meetup #8 - Testing Programmable Infrastructure By Matt Long

3.350 Aufrufe

Veröffentlicht am

With Hashicorp tools like Terraform, Packer and Vagrant, programmable infrastructure is reaching widespread adoption. However, although automated testing of software is becoming ever more common, the same cannot be said with testing programmable infrastructure. With microservices making our deployments more and more complex, we can no longer afford to ignore this type of testing. This talk will cover some experiences we have had testing the programmable infrastructure of complex applications, especially Terraform, and the lessons we have learned.

Veröffentlicht in: Software
  • Settling for less on valentine's? Then you need VigRX pLUS! ▲▲▲ https://bit.ly/30G1ZO1
       Antworten 
    Sind Sie sicher, dass Sie …  Ja  Nein
    Ihre Nachricht erscheint hier
  • You can now be your own boss and get yourself a very generous daily income. START FREE...➤➤ https://tinyurl.com/realmoneystreams2019
       Antworten 
    Sind Sie sicher, dass Sie …  Ja  Nein
    Ihre Nachricht erscheint hier
  • Secrets to making $$$ with paid surveys... ▲▲▲ https://tinyurl.com/realmoneystreams2019
       Antworten 
    Sind Sie sicher, dass Sie …  Ja  Nein
    Ihre Nachricht erscheint hier
  • Hello! Get Your Professional Job-Winning Resume Here - Check our website! https://vk.cc/818RFv
       Antworten 
    Sind Sie sicher, dass Sie …  Ja  Nein
    Ihre Nachricht erscheint hier

London Hashicorp Meetup #8 - Testing Programmable Infrastructure By Matt Long

  1. 1. MATT LONG TESTING PROGRAMMABLE INFRASTRUCTURE
  2. 2. PROGRAMMABLE INFRASTRUCTURE IS GREAT, BUT WE'RE MISSING SOMETHING. TESTING.
  3. 3. I'M A TESTER HELLO, I'M MATT
  4. 4. I WORK HERE ↑
  5. 5. I AM NOT A SYSADMIN
  6. 6. WHAT IS PROGRAMMABLE INFRASTRUCTURE?
  7. 7. TESTING PROGRAMMABLE INFRASTRUCTURE THE APPLICATION OF METHODS AND TOOLING FROM SOFTWARE DEVELOPMENT TO MANAGEMENT OF IT INFRASTRUCTURE PROGRAMMABLE INFRASTRUCTURE IS.. THE INTERNET
  8. 8. TESTING PROGRAMMABLE INFRASTRUCTURE EXAMPLES OF PROGRAMMABLE INFRASTRUCTURE ▸ Automated provisioning & configuration ▸ Configuration as code ▸ Version / source controlled
  9. 9. TESTING PROGRAMMABLE INFRASTRUCTURE TOOLING EXAMPLES
  10. 10. PROGRAMMABLE INFRASTRUCTURE IS AWESOME! Credit: Vault Boy, Bethesda Softworks
  11. 11. IT'S FAST!
  12. 12. IT'S AUTOMATIC!
  13. 13. IT'S ALL CODE!
  14. 14. BUT IT GETS COMPLEX
  15. 15. TESTING IS USED TO MITIGATE COMPLEXITY & RISK
  16. 16. BUT TESTING IS RARE Credit: Gunshow, KC Green
  17. 17. TESTING PROGRAMMABLE INFRASTRUCTURE WHAT I'M GOING TO TALK ABOUT ▸PART 1: Testing a cloud broker ▸PART 2: Building a Kubernetes cluster ▸CONCLUSIONS
  18. 18. TESTING A CLOUD BROKER AN INFRASTRUCTURE HEAVY PRODUCT
  19. 19. THE PROBLEM
  20. 20. TESTING PROGRAMMABLE INFRASTRUCTURE WE WANT TO MOVE TO THE CLOUD... BUT WE'RE WARY OF LOCK IN Large organisation
  21. 21. TESTING PROGRAMMABLE INFRASTRUCTURE USERS USE MULTIPLE CLOUD PROVIDERS
  22. 22. TESTING PROGRAMMABLE INFRASTRUCTURE PROBLEMS ▸ Different interfaces, feature sets & lingo ▸ Can't switch easily ▸ Spending difficult to track ▸ Temptation to fall back on most popular
  23. 23. TESTING PROGRAMMABLE INFRASTRUCTURE USERS CLOUD BROKER
  24. 24. TESTING PROGRAMMABLE INFRASTRUCTURE BENEFITS ▸ Quick, easy provisioning ▸ one team previously took 3 months ▸ Common interface to cloud features ▸ Templates for common dev environments ▸ Built in best practice: monitoring, security ▸ Track spending
  25. 25. THIS IS A REALLY COMPLICATED APPLICATION
  26. 26. TESTING PROGRAMMABLE INFRASTRUCTURE
  27. 27. TESTING PROGRAMMABLE INFRASTRUCTURE WORKFLOW ▸ Log into Web UI ▸ Fill in information about environment ▸ Broker creates and bootstraps resources ▸ SSH into resources
  28. 28. TESTING PROGRAMMABLE INFRASTRUCTURE WEB TESTING ▸ Log into Web UI ▸ Fill in information about environment
  29. 29. TESTING PROGRAMMABLE INFRASTRUCTURE ??? ▸ Broker creates and bootstraps resources ▸ SSH into resources
  30. 30. HOW DO YOU TEST INFRASTRUCTURE?
  31. 31. TESTING PROGRAMMABLE INFRASTRUCTURE WHAT TO TEST? Do our deployment 
 scripts work? Does the VPN server work? Can instances 
 access one another? Are services running? Can I SSH into a server?
  32. 32. THIS SEEMS FAMILIAR..
  33. 33. TESTING PROGRAMMABLE INFRASTRUCTURE Does the VPN box work?
 Can I SSH into a server? Do our deployment scripts work? Are services running? ANOTHER TESTING PYRAMID? credit: Ubuntu dev quality guide
 https://developer.ubuntu.com/en/phone/platform/quality/ Can instances access one another?
  34. 34. TOOLING
  35. 35. TESTING PROGRAMMABLE INFRASTRUCTURE TOOLS AVAILABLE ▸ Bats ▸ ShUnit2 ▸ Goss ▸ ServerSpec / Inspec / TestInfra ▸ Test Kitchen
  36. 36. UNIT TESTING
  37. 37. TESTING PROGRAMMABLE INFRASTRUCTURE BATS ▸ "Bash Automated Testing System" ▸ Unit testing for bash ▸ Like JUnit
  38. 38. TESTING PROGRAMMABLE INFRASTRUCTURE SH UNIT 2 ▸ Shell unit testing framework ▸ Runs on all Bourne shells ▸ sh, BASH, DASH, ksh, zsh ▸ No activity or support?
  39. 39. INTEGRATION TESTING OR: SERVER VALIDATION
  40. 40. TESTING PROGRAMMABLE INFRASTRUCTURE GOSS ▸ Go based ▸ Specs in YAML ▸ Minimal, fast, and simple ▸ Some neat features ▸ .. have to run on the server ▸ .. no Windows support
  41. 41. TESTING PROGRAMMABLE INFRASTRUCTURE SERVERSPEC ▸ Server based assertions ▸ Ruby/RSpec based ▸ Probably the most famous ▸ Can SSH into instances
  42. 42. TESTING PROGRAMMABLE INFRASTRUCTURE INSPEC ▸ Written & maintained by Chef ▸ Very similar to ServerSpec ▸ Different feature set ▸ More focused on compliance
  43. 43. TESTING PROGRAMMABLE INFRASTRUCTURE TESTINFRA ▸ ServerSpec, but in Python
  44. 44. TEST HARNESS
  45. 45. TESTING PROGRAMMABLE INFRASTRUCTURE TEST KITCHEN ▸ Orchestrates setup, test, teardown ▸ Runs BATS, shUnit2, RSpec, Serverspec ▸ Popular in the Chef community ▸ Not suitable for our cloud broker
  46. 46. OUR SOLUTION
  47. 47. TESTING PROGRAMMABLE INFRASTRUCTURE USERS CLOUD BROKER
  48. 48. TESTING PROGRAMMABLE INFRASTRUCTURE USERS WEB TEST FRAMEWORK
  49. 49. TESTING PROGRAMMABLE INFRASTRUCTURE USERS INFRASTRUCTURE TEST FRAMEWORK
  50. 50. TESTING PROGRAMMABLE INFRASTRUCTURE USERS WEB TESTS https://github.com/opencredo/test-automation-quickstart
  51. 51. TESTING PROGRAMMABLE INFRASTRUCTURE INFRASTRUCTURE TESTS Serverspec
  52. 52. TESTING PROGRAMMABLE INFRASTRUCTURE INFRASTRUCTURE TESTING STACK / Serverspec ???
  53. 53. TESTING PROGRAMMABLE INFRASTRUCTURE WHY RUBY? ▸ Fantastic testing community ▸ More suitable for SSHing into boxes ▸ "Win RM" gem ▸ Ops already familiar with it ▸ Reduces tech stack
  54. 54. TESTING PROGRAMMABLE INFRASTRUCTURE SERVERSPEC SMOKE TESTS ▸ Run before everything else ▸ Really quick ▸ Catches obvious errors ▸ Not complex tasks
  55. 55. TESTING PROGRAMMABLE INFRASTRUCTURE SERVERSPEC EXAMPLE describe package('jenkins') do it { should be_installed } end describe service('jenkins') do it { should be_enabled } it { should be_running } end describe port(8080) do it { should be_listening } end
  56. 56. TESTING PROGRAMMABLE INFRASTRUCTURE Background:
 Given environment has been created
 And the following user details:
 | user_alias | username | public_key |
 | userA | envoy | test | 
 
 
 Scenario: IPA - Login via SSH Key authentication succeeds
 Given user "userA" is authorised to access environment vms
 When user "userA" starts ssh session in host "env" 
 Then I should be able to echo "hello world"
 CUCUMBER FOR ACCEPTANCE TESTING
  57. 57. TESTING PROGRAMMABLE INFRASTRUCTURE Background:
 Given environment has been created
 And the following user details:
 | user_alias | username | public_key |
 | userA | envoy | test | 
 
 
 Scenario: IPA - Login via SSH Key authentication succeeds
 Given user "userA" is authorised to access environment vms
 When user "userA" starts ssh session in host "env" 
 Then I should be able to echo "hello world"
 CUCUMBER FOR ACCEPTANCE TESTING Cloud broker APIs
  58. 58. TESTING PROGRAMMABLE INFRASTRUCTURE Background:
 Given environment has been created
 And the following user details:
 | user_alias | username | public_key |
 | userA | envoy | test | 
 
 
 Scenario: IPA - Login via SSH Key authentication succeeds
 Given user "userA" is authorised to access environment vms
 When user "userA" starts ssh session in host "env" 
 Then I should be able to echo "hello world"
 CUCUMBER FOR ACCEPTANCE TESTING Standard Ruby
  59. 59. TESTING PROGRAMMABLE INFRASTRUCTURE Background:
 Given environment has been created
 And the following user details:
 | user_alias | username | public_key |
 | userA | envoy | test | 
 
 
 Scenario: IPA - Login via SSH Key authentication succeeds
 Given user "userA" is authorised to access environment vms
 When user "userA" starts ssh session in host "env" 
 Then I should be able to echo "hello world"
 CUCUMBER FOR ACCEPTANCE TESTING RSpec assertions
  60. 60. TESTING PROGRAMMABLE INFRASTRUCTURE UNDER THE CUCUMBER, PLAIN RUBY Then(/^I should be able to echo "([^"]*)"$/) do |text| cmd = "echo #{text}" output = @session.exec!(cmd) close_ssh(@session) expect(output.to_s.strip).to eql(text) end
  61. 61. THOUGHTS
  62. 62. TESTING PROGRAMMABLE INFRASTRUCTURE THE GOOD ▸ Specialised tests for each layer ▸ Really quick, expressive ServerSpec tests ▸ Power of a full programming language for user tests
  63. 63. TESTING PROGRAMMABLE INFRASTRUCTURE THE BAD ▸ Over reliance on acceptance tests ▸ Awkward switching between two suites ▸ Out of my comfort zone
  64. 64. TESTING PROGRAMMABLE INFRASTRUCTURE THE UGLY ▸ Starting infrastructure is SLOW. ▸ It's expensive...
  65. 65. IT WAS WORTH IT DESPITE ALL THAT
  66. 66. BUILDING A KUBERNETES CLUSTER APPLYING TDD TO INFRASTRUCTURE
  67. 67. INTERNAL DEVOPS TRAINING COURSE
  68. 68. I LEARNED A LOT! Credit: The Simpsons, Fox
  69. 69. TESTING PROGRAMMABLE INFRASTRUCTURE BUILD THIS: WITH THESE:
  70. 70. TESTING PROGRAMMABLE INFRASTRUCTURE NOT A STRAIGHTFORWARD TASK
  71. 71. TESTING PROGRAMMABLE INFRASTRUCTURE BUT HOW TO TEST IT? ▸ This is a dev activity ▸ Want fast feedback ▸ Complexity is mitigated by testing!
  72. 72. TESTING PROGRAMMABLE INFRASTRUCTURE ▸ Provisions cloud infrastructure ▸ Declarative files ▸ Some support for variables TERRAFORM
  73. 73. TESTING PROGRAMMABLE INFRASTRUCTURE TERRAFORM COMMANDS ▸ terraform plan ▸ Tells you what will change ▸ terraform apply ▸ Applies changes ▸ terraform validate ▸ Lints terraform syntax
  74. 74. TESTING PROGRAMMABLE INFRASTRUCTURE TERRAFORM FILE EXAMPLE resource "aws_instance" "etcd-node" { count = 3 ami = "ami-7abd0209" # centos availability_zone = "eu-west-1a" # ireland instance_type = "t2.micro" subnet_id = .... private_ip = .... key_name = "${aws_key_pair.my-key.key_name}" }
  75. 75. TESTING PROGRAMMABLE INFRASTRUCTURE LINT WITH 'TERRAFORM VALIDATE' COMMAND Omitting a variable:
  76. 76. TESTING PROGRAMMABLE INFRASTRUCTURE BUT IT DOESN'T CATCH ALL PROBLEMS Duplicate subnet CIDRS:
  77. 77. TESTING PROGRAMMABLE INFRASTRUCTURE LINTING ISN'T ENOUGH ▸ Devs don’t just rely on compilers ▸ We need something more powerful Credit: Nick Cave, "Soundsuit"
  78. 78. UNIT TESTING
  79. 79. TESTING PROGRAMMABLE INFRASTRUCTURE TERRAFORM_VALIDATE ▸ Python based unit testing ▸ NOT to be confused with 'validate' command ▸ Builds map of resources & properties ▸ Totally offline ▸ New and incomplete https://github.com/elmundio87/terraform_validate
  80. 80. TESTING PROGRAMMABLE INFRASTRUCTURE TERRAFORM_VALIDATE FORK OC has forked the terraform validate repo https://github.com/opencredo/terraform_validate
  81. 81. INTEGRATION TESTING
  82. 82. TESTING PROGRAMMABLE INFRASTRUCTURE GOSS ▸ Easy to get up and running ▸ Doesn’t support remote # example usage: ./goss-test.sh 34.248.91.167 TARGET='centos@'$1 SSH_KEY_PATH=~/.ssh/aws ssh -t -t -i $SSH_KEY_PATH $TARGET 'curl -fsSL https://goss.rocks/install | sudo sh' scp ./goss.json $TARGET:~/goss.yaml ssh -t -t -i $SSH_KEY_PATH $TARGET 'goss validate' https://gist.github.com/burythehammer/081d6ee11cc33c2f4c4729ae67622f5b
  83. 83. TESTING PROGRAMMABLE INFRASTRUCTURE ▸ Terraform compatibility ▸ Already a talk about this ▸ “Untangling Infrastructure Code” by Nell Shamrell-Harrington TEST KITCHEN + INSPEC
  84. 84. TESTING PROGRAMMABLE INFRASTRUCTURE TestCreate Config Destroy ]TEST KITCHEN MANAGES YOUR TEST LIFECYCLE
  85. 85. TESTING PROGRAMMABLE INFRASTRUCTURE TestCreate Config Destroy TEST KITCHEN DOESN'T SUPPORT MULTIPLE PROVISIONERS
  86. 86. TESTING PROGRAMMABLE INFRASTRUCTURE TEST KITCHEN DOESN'T SUPPORT MULTIPLE PROVISIONERS https://github.com/test-kitchen/test-kitchen/issues/329
  87. 87. TESTING PROGRAMMABLE INFRASTRUCTURE TERRAFORM 'NULL RESOURCE' resource "null_resource" "ansible" { triggers { instance_ids = "${join(",", aws_instance.etcd-node.*.id)}" } provisioner "local-exec" { command = "sleep 20 && cd ../ansible/ && ansible-playbook etcd.yaml" } }
  88. 88. TESTING PROGRAMMABLE INFRASTRUCTURE TestCreate Config Destroy
  89. 89. THOUGHTS
  90. 90. TESTING PROGRAMMABLE INFRASTRUCTURE THE GOOD ▸ Tooling exists! ▸ You can totally get a test suite working Credit: Overwatch, Blizzard Entertainment
  91. 91. TESTING PROGRAMMABLE INFRASTRUCTURE THE BAD ▸ Unit testing extremely immature ▸ Tools immature in general Credit: Overwatch, Blizzard Entertainment
  92. 92. TESTING PROGRAMMABLE INFRASTRUCTURE THE HACKY ▸ Be prepared to hack ▸ It might not even be possible Credit: Overwatch, Blizzard Entertainment
  93. 93. THIS IS BRAND NEW GROUND REMEMBER:
  94. 94. TESTING TOOLS DEPEND ON YOUR STACK
  95. 95. CONCLUSIONS
  96. 96. TESTING IS IMPORTANT BUT OFTEN IGNORED
  97. 97. TESTERS AND OPS SHOULD WORK TOGETHER WE NEED TO GET OUT OF OUR COMFORT ZONES
  98. 98. TOOLS EXIST BUT BE PREPARED TO HACK
  99. 99. FINALLY...
  100. 100. TESTING PROGRAMMABLE INFRASTRUCTURE THE APPLICATION OF METHODS AND TOOLING FROM SOFTWARE DEVELOPMENT TO MANAGEMENT OF IT INFRASTRUCTURE PROGRAMMABLE INFRASTRUCTURE IS..
  101. 101. TESTING IS A SOFTWARE DEVELOPMENT METHOD WE SHOULD APPLY IT TO INFRASTRUCTURE
  102. 102. THANKS QUESTIONS? @burythehammer matt.long@opencredo.com

×