Submit Search
Upload
Puppet conf 2013
•
0 likes
•
442 views
Sam Bashton
Follow
Technology
Entertainment & Humor
Report
Share
Report
Share
1 of 51
Download now
Download to read offline
Recommended
Hands-On with Heat: Service Orchestration in SUSE Cloud
Hands-On with Heat: Service Orchestration in SUSE Cloud
Rick Ashford
Hosting a Custom Forge with Pulp
Hosting a Custom Forge with Pulp
Puppet
Continuously-Integrated Puppet in a Dynamic Environment
Continuously-Integrated Puppet in a Dynamic Environment
Puppet
Dev to Delivery with Puppet, Vagrant and AWS
Dev to Delivery with Puppet, Vagrant and AWS
Puppet
Securedata - Build a virtualization service on standard hardware
Securedata - Build a virtualization service on standard hardware
GARL
Serving the Internet of Things (IoT) with Drupal - DrupalCamp Baltics 2015:
Serving the Internet of Things (IoT) with Drupal - DrupalCamp Baltics 2015:
Exove
Ammeon See OpenStack in Action
Ammeon See OpenStack in Action
Ammeon HR
MesosCon_Asia_2016_DevOps_at_GSShop
MesosCon_Asia_2016_DevOps_at_GSShop
Vivek Juneja
Recommended
Hands-On with Heat: Service Orchestration in SUSE Cloud
Hands-On with Heat: Service Orchestration in SUSE Cloud
Rick Ashford
Hosting a Custom Forge with Pulp
Hosting a Custom Forge with Pulp
Puppet
Continuously-Integrated Puppet in a Dynamic Environment
Continuously-Integrated Puppet in a Dynamic Environment
Puppet
Dev to Delivery with Puppet, Vagrant and AWS
Dev to Delivery with Puppet, Vagrant and AWS
Puppet
Securedata - Build a virtualization service on standard hardware
Securedata - Build a virtualization service on standard hardware
GARL
Serving the Internet of Things (IoT) with Drupal - DrupalCamp Baltics 2015:
Serving the Internet of Things (IoT) with Drupal - DrupalCamp Baltics 2015:
Exove
Ammeon See OpenStack in Action
Ammeon See OpenStack in Action
Ammeon HR
MesosCon_Asia_2016_DevOps_at_GSShop
MesosCon_Asia_2016_DevOps_at_GSShop
Vivek Juneja
GPSTEC319-Build Once Deploy Many Architecting and Building Automated Reusable...
GPSTEC319-Build Once Deploy Many Architecting and Building Automated Reusable...
Amazon Web Services
Agile Transformation as a Catalyst for Cloud Adoption AWS Summit SG 2017
Agile Transformation as a Catalyst for Cloud Adoption AWS Summit SG 2017
Amazon Web Services
Cloud Adoption
Cloud Adoption
Amazon Web Services
Herding cats in the Cloud
Herding cats in the Cloud
Dewey Sasser
MySQL HA Presentation
MySQL HA Presentation
papablues
Introduction to Puppet Enterprise 2016.2
Introduction to Puppet Enterprise 2016.2
Puppet
Ansible Case Studies
Ansible Case Studies
Greg DeKoenigsberg
Find out what's new at Puppet - products, programs, and more!
Find out what's new at Puppet - products, programs, and more!
Puppet
From Mainframe to Microservices
From Mainframe to Microservices
Amazon Web Services
GPSWKS301_Comprehensive Big Data Architecture Made Easy
GPSWKS301_Comprehensive Big Data Architecture Made Easy
Amazon Web Services
Comprehensive Big Data Analytics Architecture Made Easy - The AWS Marketplace...
Comprehensive Big Data Analytics Architecture Made Easy - The AWS Marketplace...
Amazon Web Services
Control-with-Ansible-Tower
Control-with-Ansible-Tower
Mark Costin
Tempo’s Journey Into the Cloud
Tempo’s Journey Into the Cloud
Atlassian
Scalable Distributed System Architecture
Scalable Distributed System Architecture
Nicholas van de Walle
Running Oracle Databases on Amazon RDS - DAT313 - re:Invent 2017
Running Oracle Databases on Amazon RDS - DAT313 - re:Invent 2017
Amazon Web Services
DevOps at VAST
DevOps at VAST
upasanip
Bosh - Configuring Services
Bosh - Configuring Services
Andrew Shafer
Join Us to Explore DevOps on AWS with REAN Cloud
Join Us to Explore DevOps on AWS with REAN Cloud
Amazon Web Services
Harnessing the Power of Apache Hadoop
Harnessing the Power of Apache Hadoop
Cloudera, Inc.
Clickability Puppet Case Study
Clickability Puppet Case Study
Puppet
Apidays New York 2024 - The value of a flexible API Management solution for O...
Apidays New York 2024 - The value of a flexible API Management solution for O...
apidays
Driving Behavioral Change for Information Management through Data-Driven Gree...
Driving Behavioral Change for Information Management through Data-Driven Gree...
Enterprise Knowledge
More Related Content
Similar to Puppet conf 2013
GPSTEC319-Build Once Deploy Many Architecting and Building Automated Reusable...
GPSTEC319-Build Once Deploy Many Architecting and Building Automated Reusable...
Amazon Web Services
Agile Transformation as a Catalyst for Cloud Adoption AWS Summit SG 2017
Agile Transformation as a Catalyst for Cloud Adoption AWS Summit SG 2017
Amazon Web Services
Cloud Adoption
Cloud Adoption
Amazon Web Services
Herding cats in the Cloud
Herding cats in the Cloud
Dewey Sasser
MySQL HA Presentation
MySQL HA Presentation
papablues
Introduction to Puppet Enterprise 2016.2
Introduction to Puppet Enterprise 2016.2
Puppet
Ansible Case Studies
Ansible Case Studies
Greg DeKoenigsberg
Find out what's new at Puppet - products, programs, and more!
Find out what's new at Puppet - products, programs, and more!
Puppet
From Mainframe to Microservices
From Mainframe to Microservices
Amazon Web Services
GPSWKS301_Comprehensive Big Data Architecture Made Easy
GPSWKS301_Comprehensive Big Data Architecture Made Easy
Amazon Web Services
Comprehensive Big Data Analytics Architecture Made Easy - The AWS Marketplace...
Comprehensive Big Data Analytics Architecture Made Easy - The AWS Marketplace...
Amazon Web Services
Control-with-Ansible-Tower
Control-with-Ansible-Tower
Mark Costin
Tempo’s Journey Into the Cloud
Tempo’s Journey Into the Cloud
Atlassian
Scalable Distributed System Architecture
Scalable Distributed System Architecture
Nicholas van de Walle
Running Oracle Databases on Amazon RDS - DAT313 - re:Invent 2017
Running Oracle Databases on Amazon RDS - DAT313 - re:Invent 2017
Amazon Web Services
DevOps at VAST
DevOps at VAST
upasanip
Bosh - Configuring Services
Bosh - Configuring Services
Andrew Shafer
Join Us to Explore DevOps on AWS with REAN Cloud
Join Us to Explore DevOps on AWS with REAN Cloud
Amazon Web Services
Harnessing the Power of Apache Hadoop
Harnessing the Power of Apache Hadoop
Cloudera, Inc.
Clickability Puppet Case Study
Clickability Puppet Case Study
Puppet
Similar to Puppet conf 2013
(20)
GPSTEC319-Build Once Deploy Many Architecting and Building Automated Reusable...
GPSTEC319-Build Once Deploy Many Architecting and Building Automated Reusable...
Agile Transformation as a Catalyst for Cloud Adoption AWS Summit SG 2017
Agile Transformation as a Catalyst for Cloud Adoption AWS Summit SG 2017
Cloud Adoption
Cloud Adoption
Herding cats in the Cloud
Herding cats in the Cloud
MySQL HA Presentation
MySQL HA Presentation
Introduction to Puppet Enterprise 2016.2
Introduction to Puppet Enterprise 2016.2
Ansible Case Studies
Ansible Case Studies
Find out what's new at Puppet - products, programs, and more!
Find out what's new at Puppet - products, programs, and more!
From Mainframe to Microservices
From Mainframe to Microservices
GPSWKS301_Comprehensive Big Data Architecture Made Easy
GPSWKS301_Comprehensive Big Data Architecture Made Easy
Comprehensive Big Data Analytics Architecture Made Easy - The AWS Marketplace...
Comprehensive Big Data Analytics Architecture Made Easy - The AWS Marketplace...
Control-with-Ansible-Tower
Control-with-Ansible-Tower
Tempo’s Journey Into the Cloud
Tempo’s Journey Into the Cloud
Scalable Distributed System Architecture
Scalable Distributed System Architecture
Running Oracle Databases on Amazon RDS - DAT313 - re:Invent 2017
Running Oracle Databases on Amazon RDS - DAT313 - re:Invent 2017
DevOps at VAST
DevOps at VAST
Bosh - Configuring Services
Bosh - Configuring Services
Join Us to Explore DevOps on AWS with REAN Cloud
Join Us to Explore DevOps on AWS with REAN Cloud
Harnessing the Power of Apache Hadoop
Harnessing the Power of Apache Hadoop
Clickability Puppet Case Study
Clickability Puppet Case Study
Recently uploaded
Apidays New York 2024 - The value of a flexible API Management solution for O...
Apidays New York 2024 - The value of a flexible API Management solution for O...
apidays
Driving Behavioral Change for Information Management through Data-Driven Gree...
Driving Behavioral Change for Information Management through Data-Driven Gree...
Enterprise Knowledge
Strategies for Landing an Oracle DBA Job as a Fresher
Strategies for Landing an Oracle DBA Job as a Fresher
Remote DBA Services
Finology Group – Insurtech Innovation Award 2024
Finology Group – Insurtech Innovation Award 2024
The Digital Insurer
🐬 The future of MySQL is Postgres 🐘
🐬 The future of MySQL is Postgres 🐘
RTylerCroy
Histor y of HAM Radio presentation slide
Histor y of HAM Radio presentation slide
vu2urc
+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...
+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...
?#DUbAI#??##{{(☎️+971_581248768%)**%*]'#abortion pills for sale in dubai@
ProductAnonymous-April2024-WinProductDiscovery-MelissaKlemke
ProductAnonymous-April2024-WinProductDiscovery-MelissaKlemke
Product Anonymous
Automating Google Workspace (GWS) & more with Apps Script
Automating Google Workspace (GWS) & more with Apps Script
wesley chun
Tata AIG General Insurance Company - Insurer Innovation Award 2024
Tata AIG General Insurance Company - Insurer Innovation Award 2024
The Digital Insurer
Boost PC performance: How more available memory can improve productivity
Boost PC performance: How more available memory can improve productivity
Principled Technologies
What Are The Drone Anti-jamming Systems Technology?
What Are The Drone Anti-jamming Systems Technology?
Antenna Manufacturer Coco
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
Miguel Araújo
TrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
TrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
TrustArc
Developing An App To Navigate The Roads of Brazil
Developing An App To Navigate The Roads of Brazil
V3cube
GenCyber Cyber Security Day Presentation
GenCyber Cyber Security Day Presentation
Michael W. Hawkins
Apidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, Adobe
Apidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, Adobe
apidays
How to Troubleshoot Apps for the Modern Connected Worker
How to Troubleshoot Apps for the Modern Connected Worker
ThousandEyes
2024: Domino Containers - The Next Step. News from the Domino Container commu...
2024: Domino Containers - The Next Step. News from the Domino Container commu...
Martijn de Jong
Understanding Discord NSFW Servers A Guide for Responsible Users.pdf
Understanding Discord NSFW Servers A Guide for Responsible Users.pdf
UK Journal
Recently uploaded
(20)
Apidays New York 2024 - The value of a flexible API Management solution for O...
Apidays New York 2024 - The value of a flexible API Management solution for O...
Driving Behavioral Change for Information Management through Data-Driven Gree...
Driving Behavioral Change for Information Management through Data-Driven Gree...
Strategies for Landing an Oracle DBA Job as a Fresher
Strategies for Landing an Oracle DBA Job as a Fresher
Finology Group – Insurtech Innovation Award 2024
Finology Group – Insurtech Innovation Award 2024
🐬 The future of MySQL is Postgres 🐘
🐬 The future of MySQL is Postgres 🐘
Histor y of HAM Radio presentation slide
Histor y of HAM Radio presentation slide
+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...
+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...
ProductAnonymous-April2024-WinProductDiscovery-MelissaKlemke
ProductAnonymous-April2024-WinProductDiscovery-MelissaKlemke
Automating Google Workspace (GWS) & more with Apps Script
Automating Google Workspace (GWS) & more with Apps Script
Tata AIG General Insurance Company - Insurer Innovation Award 2024
Tata AIG General Insurance Company - Insurer Innovation Award 2024
Boost PC performance: How more available memory can improve productivity
Boost PC performance: How more available memory can improve productivity
What Are The Drone Anti-jamming Systems Technology?
What Are The Drone Anti-jamming Systems Technology?
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
TrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
TrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
Developing An App To Navigate The Roads of Brazil
Developing An App To Navigate The Roads of Brazil
GenCyber Cyber Security Day Presentation
GenCyber Cyber Security Day Presentation
Apidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, Adobe
Apidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, Adobe
How to Troubleshoot Apps for the Modern Connected Worker
How to Troubleshoot Apps for the Modern Connected Worker
2024: Domino Containers - The Next Step. News from the Domino Container commu...
2024: Domino Containers - The Next Step. News from the Domino Container commu...
Understanding Discord NSFW Servers A Guide for Responsible Users.pdf
Understanding Discord NSFW Servers A Guide for Responsible Users.pdf
Puppet conf 2013
1.
CONTINUOUSLY INTEGRATED PUPPET IN A DYNAMIC ENVIRONMENT SAM BASHTON, BASHTON LTD
2.
CONTINUOUSLY INTEGRATED PUPPET IN A DYNAMIC ENVIRONMENT MASTERLESS PUPPET: WHY AND HOW SAM BASHTON, BASHTON LTD
3.
ABOUT ME Linux guy since Slackware, floppy disks and root + boot Using Puppet since 2007 Run a company in Manchester, North West England We provide outsourced ops for other companies
4.
OUR FULLY MANAGED ENVIRONMENTS Primarily transactional websites (ecommerce) Majority (70%+) on Amazon Web Services (AWS) Majority using CentOS
5.
HOW WE WORK Simple is better than complex Complexity is worth adding only if it provides obvious functional benefits Reusability Resilience
6.
WHY DID WE PICK AWS? Featureset and toolset massively in advance of any other cloud provider, public or private #1 customer reason for switching to AWS? The ability to scale on demand
7.
8.
TOOLS WE USE FOR BUILDING AND MANAGING Do one thing and do it well CloudFormation Amazon tool to manage infrastructure Puppet Manage system configuration Pulp centralised repository, manages package revisions Jenkins
9.
HOW WE USE PUPPET No Puppetmaster Puppet manifests, hieradata and modules distributed to all machines via RPM All machines boot with a common, blank image and get configured at first boot
10.
WHAT'S WRONG WITH MASTER BASED PUPPET? Pets vs Cattle Puppet designed for a world of servers as pets We do not live in that world
11.
PUPPET DESIGNED FOR PETS Many assumptions in Puppet presume that your servers are pets Some of these work against us when managing a herd
12.
MANUAL CERTIFICATE SIGNING Clearly unsuitable when machines are automatically provisioned
13.
POTENTIAL WORKAROUNDS: Autosign Use/write another automated certificate generation mechanism Possibly tied in with autoscaling
14.
NO MECHANISM FOR CLEANING OLD HOSTS Likely to have hostnames reused, causing machines to fail to configure Puppetmaster will fill with certificates for machines that ran for a few hours and went away again
15.
POTENTIAL WORKAROUNDS: Use UUID certificates Agree not to look in the certificate directory Write mechanism for cleaning up old certificates
16.
HOSTS CONFIGURED BASED ON HOSTNAME Our machines have names like ip172265123 How does Puppet know what type of machine this is?
17.
POTENTIAL WORKAROUNDS Use an external node classifier Use some mechanism for giving a better hostname, eg web172265123 and use regex for node names
18.
PUPPETMASTER IS A SINGLE POINT OF FAILURE If the Puppetmaster fails, we can no longer autoscale up In particular, this could be a problem if there is availability zone failure
19.
POTENTIAL WORKAROUNDS Clustered Puppetmasters
20.
WORKAROUND RECAP Use/write alternative certificate management software Write an external node classifier / mechanism for setting hostname appropriately Cluster multiple Puppetmasters
21.
WHAT WE DID INSTEAD Decided using a Puppetmaster was trying to fit a square peg into a round hole Instead, decided to run Puppet without a master
22.
APPLYING LOCAL PUPPET MANIFESTS puppet apply --modulepath=/etc/puppet/modules
example.pp
23.
DISTRIBUTING MANIFESTS Use RPM Distribute full set of manifests/modules to each machine Apply only the manifest relevant to that machine
24.
PACKING PUPPET MANIFESTS IN RPM Build an RPM containing everything under /etc/puppet Make files readable only by root
25.
APPLY PUPPET MANIFESTS Have an RPM %postinst command apply the Puppet config This isn't as straightforward as running the puppet apply from %postinst Puppet needs to install packages via yum, but yum is running installing the Puppet package Instead, we work around with a dirty hack: have the %postinst create an at script which checks if yum has finished and then runs the puppet apply
26.
RPM INSTALLATION AND MANAGEMENT How do we get these RPMs on our machines?
27.
PULP We were already using Pulp Provides yum repository management Used for managing security updates and deploying application code http://pulpproject.org/
28.
WHAT IS PULP Repository manager Allows us to easily audit what packages and versions are installed where Allows us to push package installations Uses qpid message queue Has concept of 'content distrubtion servers' for easy replication and clustering
29.
HOW WE USE PULP Puppet contains details of what packages should be installed Pulp manages which version of the package should be installed Pulp allows us to clone repos and copy packages between them for easy qa>stage>live environment management
30.
DEPLOYING CONFIGURATION AS CODE Allows us to reuse our existing code deployment infrastructure Manage configuration deployment from Jenkins
31.
HOW WE DEPLOY CODE Everything managed via the Jenkins continuous integration server Jenkins uses Pulp to install code on remote machines
32.
DETAILS ON HOW WE DEPLOY CODE Jenkins fetches code from source control (git) An RPM is built Tests are run If tests pass, the RPM is added to the relevant Pulp repository RPM installed on the target machine(s)
33.
DEPLOYMENT LIFECYCLE Jenkins also manages deployment lifecycle RPMs are installed on staging Promoted Builds plugin then used to install the same RPMs on live once testing is complete
34.
PUPPET DEPLOYMENT PROCESS Puppet manifests are checked into git Lint tests via Jenkins pulls in modules with librarianpuppet, then builds an RPM Deployment to test environments, functional tests for wider codebase run Jenkins Warnings plugin
35.
PUTTING IT INTO PRODUCTION Once suitable tests (automated and manual) have been carried out, we promote Puppet config into production We use the Jenkins 'Promoted Builds' plugin for this
36.
JENKINS: PROMOTION
37.
EXCEPT.. How does a machine get from a bare image to the state where we can push packages to it from Pulp? How does a machine know what type of machine it is? How do we find other resources, eg database hostname?
38.
CLOUDFORMATION Amazon tool for specifying infrastructure Everything* we provision inside AWS is provisioned via CloudFormation JSON templates * Everything except for the things Amazon doesn't expose via CloudFormation..
39.
CLOUDINIT Works with multiple cloud types Sorts out things like SSH keys, allows us to configure host names Also allows us to provide a bash script to run on startup
40.
PROVISIONING A BARE INSTANCE cloudinit automatically manually adds the pulp repo which contains Pulp, Puppet and our Puppet manifests/modules Installs appropriate RPMs Puppet runs, subscribing the machine to the relevant Pulp repos, and installing packages in the usual Puppet way
41.
HOW DOES IT KNOW WHAT TYPE OF MACHINE IT IS? We tell it! Use an environmental variable $HOSTTYPE Simply run puppet apply --modulepath=/etc/puppet/modules
${HOSTTYPE}.pp
42.
EXTRA FACTS Custom facter facts Also specified in an environmental variable Data comes from within the CloudFormation template On our list of things to look at: FACTER_HOSTENVIRONMENT=live FACTER_STACKNAME=customer-web-live https://github.com/fanduel/hieracloudformation
43.
OTHER RESOURCES We either: Provide details as a facter fact `FACTER_DBHOST=xyz Also use this approach to limit distribution of secure details, eg DB passwords Discover via the EC2 API Eg Varnish servers discover web backends by calling API and finding hosts tagged appropriately
44.
FREE WINS!
45.
46.
FREE WINS! Greater control over the timing of Puppet runs Improved visibility for ops and devs Configuration changes now have to be deployed to testing/staging first
47.
MORE FREE WINS! Puppet configs now have a version Easy to find config version on the machine itself Config changelogs accessible on every machine (Git changelog added to RPM)
48.
THE DOWNSIDES Puppet manifests and modules on all machines Potentially a security issue? Mitigated by CloudFormation holding most sensitive data
49.
ALTERNATIVE IMPLEMENTATIONS Don't want to use Pulp? Could do basically the same thing with yum s3 plugin Use mcollective to push package updates https://github.com/jbraeuer/yums3plugin
50.
FUTURE IMPROVEMENTS Build AMIs using Packer instead of configuring at boot time Decrease time to autoscale Would probably still need to run Puppet at first boot to configure machine specific settings
51.
QUESTIONS? COMMENTS? Sam Bashton Twitter: (Psst.. ) sam@bashton.com @bashtoni http://www.bashton.com/jobs/
Download now