Test your cloud as code by Pierre-Luc Dion

1. #CCCEU14 | #CloudStackWorks
#CCCEU14 | #CloudStackWorks
Test your cloud as code
Pierre-Luc Dion (pdion@cloudops.com)

2. #CCCEU14 | #CloudStackWorks
Overview
• introduction
• Why?
• cloudstack_spec: how it work

3. #CCCEU14 | #CloudStackWorks
About Me
• Cloud Architect @ CloudOps
• Work full time with Apache CloudStack
– As user
– As contributor
• Apache CloudStack PMC member
• Ruby and Chef Enthusiast

4. #CCCEU14 | #CloudStackWorks
About CloudOps
• CloudOps builds and operates CloudStack
based clouds of all shapes and sizes
• Develops cloud infrastructure solutions and
operational models
• 24x7x365 managed service for CloudStack
based cloud infrastructures
• Customers are global
• Based in Montreal, Canada

5. #CCCEU14 | #CloudStackWorks
Current state of the project
• Proof of concept
• Support
– VPC
– Port Forwarding
– Create snapshot
– Create template from snapshot
– Various zone features validation.

6. #CCCEU14 | #CloudStackWorks
#CCCEU14 | #CloudStackWorks
Why ?

7. #CCCEU14 | #CloudStackWorks
What I’m trying to solve
• We have multiple LAB CloudStack
installations running multiple versions and
configurations. Need to validate their
usability.
• CloudStack Release Upgrade tests on real
hardware.

8. #CCCEU14 | #CloudStackWorks
Monitoring is not enough
• Monitoring can tell if there is enough
Secondary storage. But cannot tell if it’s
usable in CloudStack.
• Monitoring cannot tell if Featured
templates are usable.

9. #CCCEU14 | #CloudStackWorks
#CCCEU14 | #CloudStackWorks
But what about Marvin?

10. #CCCEU14 | #CloudStackWorks
I did not tried Marvin
1. Look like it provision CloudStack configs
and tests, right?
2. Defining tests require coding (from user
perspective)
3. Could hardly benefit from ServerSpec,
specinfra or test-kitchen
https://cwiki.apache.org/confluence/display/CLOUDSTACK/Marvin+-+Testing+with+Python

11. #CCCEU14 | #CloudStackWorks
#CCCEU14 | #CloudStackWorks
cloudstack_spec

12. #CCCEU14 | #CloudStackWorks
The purposes
• Test existing CloudStack installation
• Validate functionality of a Cloud
• Advanced tests:
– Port-forwarding rules into VPC
– Create Instance from Snapshot
– Create Template from Snapshot

13. #CCCEU14 | #CloudStackWorks
About cloudstack_spec
• Ruby Rspec based
• Based on ServerSpec
• Test case easy to define
• Use:
– cloudstack_ruby_client
– serverspec
– specinfra
• Code maturity: under heavy dev

14. #CCCEU14 | #CloudStackWorks
It is not
• Deployment tool
• Replacement to Marvin
• Monitoring plugin/tool
• Reporting tool

15. #CCCEU14 | #CloudStackWorks
#CCCEU14 | #CloudStackWorks
In Action

16. #CCCEU14 | #CloudStackWorks
#CCCEU14 | #CloudStackWorks
Define tests

17. #CCCEU14 | #CloudStackWorks
Zone
require 'spec_helper'
describe zone do
it { should exist }
it { should be_allocated }
its(:local_storage) { should be_set }
its(:security_group) { should_not be_set }
its(:network_type) { should match("Advanced") }
end

18. #CCCEU14 | #CloudStackWorks
Zone
require 'spec_helper'
describe zone do
it { should exist }
it { should be_allocated }
its(:local_storage) { should be_set }
its(:security_group) { should_not be_set }
its(:network_type) { should match("Advanced") }
end
Do I have a zone?

19. #CCCEU14 | #CloudStackWorks
Zone
require 'spec_helper'
describe zone do
it { should exist }
it { should be_allocated }
its(:local_storage) { should be_set }
its(:security_group) { should_not be_set }
its(:network_type) { should match("Advanced") }
end
The zone is
enabled
(allocated)?

20. #CCCEU14 | #CloudStackWorks
Zone
require 'spec_helper'
describe zone do
it { should exist }
it { should be_allocated }
its(:local_storage) { should be_set }
its(:security_group) { should_not be_set }
its(:network_type) { should match("Advanced") }
end
rspec_its: eval
method attributes

21. #CCCEU14 | #CloudStackWorks
Zone
require 'spec_helper'
describe zone do
it { should exist }
it { should be_allocated }
its(:local_storage) { should be_set }
its(:security_group) { should_not be_set }
its(:network_type) { should match("Advanced") }
end
Easy valid on false

22. #CCCEU14 | #CloudStackWorks
Zone
require 'spec_helper'
describe zone do
it { should exist }
it { should be_allocated }
its(:local_storage) { should be_set }
its(:security_group) { should_not be_set }
its(:network_type) { should match("Advanced") }
end
Rspec string
validation

23. #CCCEU14 | #CloudStackWorks
Output results

24. #CCCEU14 | #CloudStackWorks
Output results

25. #CCCEU14 | #CloudStackWorks
Output results

26. #CCCEU14 | #CloudStackWorks
Output results

27. #CCCEU14 | #CloudStackWorks
#CCCEU14 | #CloudStackWorks
Does my zone have system VMs?

28. #CCCEU14 | #CloudStackWorks
System vm tests
%w(consoleproxy secondarystoragevm).each do |svm|
describe system_vm(svm) do
it { should exist }
it { should be_running }
it { should be_reachable }
end
end

29. #CCCEU14 | #CloudStackWorks
SVMs output

30. #CCCEU14 | #CloudStackWorks
#CCCEU14 | #CloudStackWorks
Create VM inside VPC and test the network

31. #CCCEU14 | #CloudStackWorks
VPC test Scope
• Create VPC
• Create VPC network Tier
• Create Instance
• Enable Port-Forwarding for SSH

32. #CCCEU14 | #CloudStackWorks
vpc_spec.rb
describe vpc('spec-vpc1') do
it { should be_created }
it { should exist }
it { should be_ready }
it { should be_reachable }
describe vpc_tier('tier11') do
it { should be_created }
it { should exist }
describe virtual_machine('vm-test1') do
it { should be_created }
it { should_not be_reachable }
it { should exist }
it { should be_running }
its(:open_pf_ssh) { should be_set }
context 'With port forwarding' do
it { should be_reachable.with( :port => 22, :proto => 'tcp' ) }
end
end
end
end

33. #CCCEU14 | #CloudStackWorks
vpc_spec.rb
describe vpc('spec-vpc1') do
it { should be_created }
it { should exist }
it { should be_ready }
it { should be_reachable }
describe vpc_tier('tier11') do
it { should be_created }
it { should exist }
describe virtual_machine('vm-test1') do
it { should be_created }
it { should_not be_reachable }
it { should exist }
it { should be_running }
its(:open_pf_ssh) { should be_set }
context 'With port forwarding' do
it { should be_reachable.with( :port => 22, :proto => 'tcp' ) }
end
end
end
end
be_created
Create the object

34. #CCCEU14 | #CloudStackWorks
vpc_spec.rb
describe vpc('spec-vpc1') do
it { should be_created }
it { should exist }
it { should be_ready }
it { should be_reachable }
describe vpc_tier('tier11') do
it { should be_created }
it { should exist }
describe virtual_machine('vm-test1') do
it { should be_created }
it { should_not be_reachable }
it { should exist }
it { should be_running }
its(:open_pf_ssh) { should be_set }
context 'With port forwarding' do
it { should be_reachable.with( :port => 22, :proto => 'tcp' ) }
end
end
end
end
be_reachable
Ping test

35. #CCCEU14 | #CloudStackWorks
vpc_spec.rb
describe vpc('spec-vpc1') do
it { should be_created }
it { should exist }
it { should be_ready }
it { should be_reachable }
describe vpc_tier('tier11') do
it { should be_created }
it { should exist }
describe virtual_machine('vm-test1') do
it { should be_created }
it { should_not be_reachable }
it { should exist }
it { should be_running }
its(:open_pf_ssh) { should be_set }
context 'With port forwarding' do
it { should be_reachable.with( :port => 22, :proto => 'tcp' ) }
end
end
end
end
be_reachable
Tcp connection test
Using Public IP

36. #CCCEU14 | #CloudStackWorks
#CCCEU14 | #CloudStackWorks
Results of VPC tests…

37. #CCCEU14 | #CloudStackWorks
test execution (1/4)

38. #CCCEU14 | #CloudStackWorks
test execution (2/4)

39. #CCCEU14 | #CloudStackWorks
test execution (3/4)

40. #CCCEU14 | #CloudStackWorks
test execution (4/4)

41. #CCCEU14 | #CloudStackWorks
#CCCEU14 | #CloudStackWorks
Working with Rspec

42. #CCCEU14 | #CloudStackWorks
Custom Rspec matcher
# If the zone is allocated (Enabled)
RSpec::Matchers.define :be_allocated do |expected|
match do |actual|
actual.allocated? == true
end
failure_message do |actual|
"status: #{actual.allocated?}"
end
description do
"be allocated (Enabled)"
end
failure_message_when_negated do |actual|
"Status: #{actual}"
end
end

43. #CCCEU14 | #CloudStackWorks
resources (Test cases)
• Test cases are:
– Objects to test are Ruby Class:
CloudstackSpec::Resource::Snapshot
– Unit tests are Ruby Methods inside Class:
self.exist?

44. #CCCEU14 | #CloudStackWorks
#CCCEU14 | #CloudStackWorks
What’s next?

45. #CCCEU14 | #CloudStackWorks
Future improvements
• Provide as Gem
• Auto config, same as Serverspec
• More tests
• Better error message
• Add compatibility with ServerSpec
• test-kitchen compatibility?

46. #CCCEU14 | #CloudStackWorks
Help needed
• ServerSpec integration
– Test from within an
instance
• More test scenarios

47. #CCCEU14 | #CloudStackWorks
Future usage
• Validate new CloudStack release
• Cloud heath check
• Post automation tests
• Continuous Integration Tests (CI)

48. #CCCEU14 | #CloudStackWorks
Conclusion
• Test phase in the automated DevOps world
• Tests definition made easy.
• More tests case in progress.

49. #CCCEU14 | #CloudStackWorks
More Information
• Pierre-Luc Dion
– pdion@cloudops.com
– www.cloudops.com
– @CloudOps_
• Github
– https://github.com/pdion891/cloudstack_spec

50. #CCCEU14 | #CloudStackWorks
#CCCEU14 | #CloudStackWorks
Thank you!