MCollective as an Integration Layer by Jon Topper

1. MCollective as
an integration layer
Jon Topper
The Scale Factory

2. MCollective
a framework to build
server orchestration
or parallel job
execution systems.
“
”

3. MCollective
not simply a fancy
SSH “for loop”“
”

4. MCollective
Servers
MiddlewareClient

5. MCollective
Servers
MiddlewareClient

6. Finding Servers

7. Check Service Status

8. Check Package Versions

9. Still looks a bit like a
fancy SSH “for loop”

10. a framework to build
server orchestration
or parallel job
execution systems.
“
”

12. Zabbix
• Distributed monitoring system
• Fetches time series data - can alert on anomaly
• Uses MySQL
• Multiple data sources:
• Zabbix agent / sender
• SNMP poll / traps
• JMX

13. Zabbix

14. Zabbix

15. ZCollective
zcollective —zabbix-api-url !
http://localhost/api_jsonrpc.php

16. ZCollective
include MCollective::RPC!
!
mc = rpcclient("rpcutil", :debug => true)!
!
mc.discover.sort.each do |host|!
!
inventory = mc.custom_request( !
"inventory", {}, host,!
{ "identity" => host }!
).first!
!
do_some_zabbix_magic(inventory)!
!
end

17. ZCollective
def do_some_zabbix_magic(inventory)!
!
# If the host isn’t added to Zabbix!
# Add the host using the API!
# For each Puppet class on this host!
# If there’s a Zabbix template by this name!
# Assign that template to the host.!
# end!
# end!
# end!
!
# If the host is already in Zabbix!
# Make sure all templates correctly assigned!
# end !
!
end

18. ZCollective

19. ZCollective

20. ZCollective
https://github.com/scalefactory/zcollective

21. Traditional Deployment

22. sf-deploy
Latest:
Deployed:
aaa1
aaa1

23. sf-deploy
Latest:
Deployed:
aaa1
aaa1

24. sf-deploy
Latest:
Deployed: aaa1
aaa2

25. sf-deploy
Latest:
Deployed:
aaa2
aaa2

26. Code
class MCollective::Application::Sfdeploy < MCollective::Application!
!
def main!
!
action = configuration[:action]!
!
mc = rpcclient("sfdeploy")!
!
printrpc mc.send(!
action,!
:application => configuration[:application],!
:branch => configuration[:branch],!
:tag => configuration[:tag],!
:groups => configuration[:groups],!
:options => options!
)!
!
printrpcstats!
end

27. Code
module MCollective!
module Agent!
class Sfdeploy < RPC::Agent!
!
all_actions.each do |act|!
action act do!
!
sf_deploy = config.pluginconf["sfdeploy.binary"] || "sf-deploy"!
deploy_user = config.pluginconf["sfdeploy.user"]|| nil!
!
command = "#{sf_deploy} -a #{request[:application]} -v "!
command << "-b #{request[:branch]} " if request[:branch]!
command << "-t #{request[:tag]} " if request[:tag]!
command << "-g #{request[:groups]} " if request[:groups]!
command << act!
!
if deploy_user!
to_run = "su #{deploy_user} -c "#{command}""!
else!
to_run = command!
end

28. ZCollective
https://github.com/scalefactory/sf-deploy

29. Resources
http://docs.puppetlabs.com/mcollective/

30. http://www.scalefactory.com/
jon@scalefactory.com
@jtopper
scalefactory