This talk will be a modified and updated version of the talk given at Puppet Camp Amsterdam. I will discuss some technical examples of how to use Puppet to manage large infrastructures in Amazon's cloud, as well as giving some background in to how Puppet fits in to the AWS ecosystem. Further topics include: - OpsWorks (Amazon's Chef-based configuration management offering) and what it means for Puppet - using Puppet in conjunction with CloudFormation - using Puppet to automate common AWS tasks, such as building AMIs - using Vagrant and Puppet to create an easy path from local development to production - common pitfalls and workarounds Mike Ryan Cloud Infrastructure Consultant, Epitech BV Mike Ryan is the founder of Epitech BV, a cloud technology consultancy based in Amsterdam, and author of the upcoming O'Reilly book AWS System Administration. He is also the founder of Cloudfrag, an internet gaming startup, and is focussed on using efficient engineering practices to help companies accelerate their growth.

1. WELCOME
Puppet and AWS
Getting the Best of BothWorlds
Mike Ryan - Epitech BV
23
August
2013
CONTACT:
mike@epitech.nl
www.epitech.nl
Saturday, August 24, 13

2. EPITECH BV
Hello, I’m Mike
2
CONTACT:
mike@epitech.nl
www.epitech.nl
Sysadmin
with
a
passion
for
automa2on
Londoner
in
Amsterdam
Epitech.nl
-­‐
sysadmin
as
a
service
Saturday, August 24, 13

3. EPITECH BV
AWS - A very brief
introduction
3
CONTACT:
mike@epitech.nl
www.epitech.nl
EC2
-­‐
Elas2c
Compute
Cloud
AMI
-­‐
Amazon
Machine
Images
User
Data
Saturday, August 24, 13

4. EPITECH BV
EC2 - Auto Scaling
4
CONTACT:
mike@epitech.nl
www.epitech.nl
Saturday, August 24, 13

5. EPITECH BV
CloudFormation
5
CONTACT:
mike@epitech.nl
www.epitech.nl
{
"AWSTemplateFormatVersion" : "2010-09-09",
"Description" : "EC2 instance",
"Resources" : {
"MyEC2Instance" : {
"Type" : "AWS::EC2::Instance",
"Properties" : {
"ImageId" : "ami-79fd7eee",
"KeyName" : "my-ssh-key",
}
}
}
}
Saturday, August 24, 13

6. EPITECH BV
6
CONTACT:
mike@epitech.nl
www.epitech.nl
}
}
}
},
"asgMyAutoScalingGroup": {
"Type": "AWS::AutoScaling::AutoScalingGroup",
"Properties": {
"AvailabilityZones": [
"us-east-1b",
"us-east-1c"
],
"Cooldown": "300",
"DesiredCapacity": "1",
"MaxSize": "1",
"MinSize": "1",
"LaunchConfigurationName": {
"Ref": "lcMyLC"
},
"LoadBalancerNames": [
{
"Ref": "elbMyLB"
}
]
}
},
"s3webapplication": {
"Type": "AWS::S3::Bucket"
},
"sgwebappsecuritygroup": {
"Type": "AWS::EC2::SecurityGroup",
"Properties": {
"GroupDescription": "for web app",
"SecurityGroupIngress": [
Saturday, August 24, 13

7. EPITECH BV
7
CONTACT:
mike@epitech.nl
www.epitech.nl
Bootstrapping an
Infrastructure
www.infrastructures.org
Saturday, August 24, 13

8. EPITECH BV
8
CONTACT:
mike@epitech.nl
www.epitech.nl
CloudFormation or Puppet
Saturday, August 24, 13

9. EPITECH BV
9
CONTACT:
mike@epitech.nl
www.epitech.nl
Files, Services and Packages
with cfn-init
"Metadata": {
"AWS::CloudFormation::Init": {
"config": {
"sources" : {
"/etc/puppet" : "http://example.com/puppet.tar.gz"
},
"packages": {
"yum": {
"puppet-server": [],
},
},
"services": {
"sysvinit": {
"puppetmaster": {
"ensureRunning": "true",
"enabled": "true"
Saturday, August 24, 13

10. EPITECH BV
10
CONTACT:
mike@epitech.nl
www.epitech.nl
Run a script at launch time
with User Data
{ "AWSTemplateFormatVersion" : "2010-09-09",
"Description" : "EC2 instance",
"Resources" : {
"PuppetMasterInstance": {
"Type": "AWS::EC2::Instance",
"Metadata": { },
"Properties": {
"UserData": {
"Fn::Base64": {
"Fn::Join": [ "",
[
"#!/bin/bashn",
"/opt/aws/bin/cfn-init --region ", "n",
"/usr/bin/puppet apply site.pp, "n"
]
...
Saturday, August 24, 13

11. EPITECH BV
11
CONTACT:
mike@epitech.nl
www.epitech.nl
cfn-create-stack
cfn-create-stack puppetmaster
--template-file=puppetmaster.json
http://aws.amazon.com/developertools/
2555753788650372
http://aws.amazon.com/cli/
Saturday, August 24, 13

12. EPITECH BV
12
CONTACT:
mike@epitech.nl
www.epitech.nl
Auto Scale = Autosign
Saturday, August 24, 13

13. EPITECH BV
13
CONTACT:
mike@epitech.nl
www.epitech.nl
The Hostname Issue
mike@ip-10-32-34-116:~$ hostname -f
ip-10-32-34-116.eu-west-1.compute.internal
Saturday, August 24, 13

14. EPITECH BV
14
CONTACT:
mike@epitech.nl
www.epitech.nl
User Data - Web Console
Saturday, August 24, 13

15. EPITECH BV
15
CONTACT:
mike@epitech.nl
www.epitech.nl
User Data - CloudFormation},
"UserData": {
"Fn::Base64": {
"Fn::Join": [
"",
[
“{”role”: ”web”, ”,
“”env”: ”staging”} ”
]
]
}
}
Saturday, August 24, 13

16. EPITECH BV
16
CONTACT:
mike@epitech.nl
www.epitech.nl
include stdlib
node default {
$userdata = parsejson($ec2_userdata)
$role = userdata[‘role’]
$environment = userdata[‘env’]
case $role {
‘web’: {
include nginx
}
‘db’: {
include postgresql
}
}
}
Saturday, August 24, 13

17. EPITECH BV
17
CONTACT:
mike@epitech.nl
www.epitech.nl
Vagrant -> EC2
Saturday, August 24, 13

18. EPITECH BV
18
CONTACT:
mike@epitech.nl
www.epitech.nl
Saturday, August 24, 13

19. EPITECH BV
19
CONTACT:
mike@epitech.nl
www.epitech.nl
Vagrant::Config.run do |config|
config.vm.provision :puppet do |puppet|
puppet.manifests_path = "../puppet/manifests"
puppet.module_path = "../puppet/modules"
puppet.manifest_file = "site.pp"
puppet.options = "--verbose --debug"
puppet.facter = {
:ec2_userdata => {
:role => "database",
:env => "vagrant",
}.to_json,
:vagrant => "true"
}
end
end
Saturday, August 24, 13

20. EPITECH BV
20
CONTACT:
mike@epitech.nl
www.epitech.nl
Decoupling
Saturday, August 24, 13

21. EPITECH BV
21
CONTACT:
mike@epitech.nl
www.epitech.nl
Broken Puppet Master =
Broken Auto Scaling
No
Saturday, August 24, 13

22. EPITECH BV
22
CONTACT:
mike@epitech.nl
www.epitech.nl
#!/bin/bash
/usr/local/bin/update.sh
/usr/bin/puppet apply site.pp
/etc/rc.local
Saturday, August 24, 13

23. EPITECH BV
23
CONTACT:
mike@epitech.nl
www.epitech.nl
But you lose...
Puppet
Dashboard
Exported
Resources
Stored
configs
Saturday, August 24, 13

24. EPITECH BV
24
CONTACT:
mike@epitech.nl
www.epitech.nl
env.roledefs = {
'web': ['www1', 'www2', 'www3'],
'db': ['db1', 'db1']
}
@roles('db', ‘web’)
def run_puppet():
sudo('puppet apply site.pp')
$fab run_puppet #run puppet everywhere
$fab run_puppet --roles db # role-specific
Triggering Puppet with
Fabric
Saturday, August 24, 13

25. EPITECH BV
25
CONTACT:
mike@epitech.nl
www.epitech.nl
def configure_roles():
tags = EC2TagManager(AWS_KEY, AWS_SECRET,
regions=['eu-west-1'])
roles = {}
for role in [‘db’, ‘web’]:
roles[role] =
tags.get_instances(role=role)
return roles
Triggering Puppet with Fabric
https://github.com/mikery/fabric-ec2
Saturday, August 24, 13

26. THANK YOU
BACK X
CONTACT:
Mike Ryan - Epitech BV
mike@epitech.nl
mike@epitech.nl
www.epitech.nl
Saturday, August 24, 13