Integrating Puppet with Cloud Infrastructures-Remco Overdijk
•
2 gefällt mir•1,034 views
Presentation Puppet Camp Amsterdam 2016
Empfohlen
Empfohlen
Weitere ähnliche Inhalte
Was ist angesagt?
Was ist angesagt? (20)
Ähnlich wie Integrating Puppet with Cloud Infrastructures-Remco Overdijk
Ähnlich wie Integrating Puppet with Cloud Infrastructures-Remco Overdijk (20)
Kürzlich hochgeladen
Kürzlich hochgeladen (20)
Integrating Puppet with Cloud Infrastructures-Remco Overdijk
- 1. Remco Overdijk LEAD OPERATIONS ENGINEER Automating the Cloud Integrating Puppet with Cloud Infrastructures @MaxServ @RemzJay
- 3. 3AUTOMATING THE CLOUD Traditional Website Hosting MySQL
- 4. AUTOMATING THE CLOUD Single Server Infrastructure Issues PROBLEM Limited headroom Service issues affect other services One outage means downtime Maintenance during deployment 4
- 5. Image credit: https://commons.wikimedia.org/wiki/File:Grumpy-Cat.jpg - Rjommabolli (CC 4.0)
- 6. AUTOMATING THE CLOUD Single Server Infrastructure Issues 6 Scalability Service Isolation Redundancy Zero-Downtime Deployments SOLUTIONPROBLEM Limited headroom Affected services One outage means downtime Maintenance during deployment
- 7. • AWS specific, but applies to most (if not any) Cloud platforms. AUTOMATING THE CLOUD Scope 7 • LAMP stack, but works for most stacks. • Mix & match for best results. • The method that works best depends on your own setup. • Based on Puppet Open Source. • Things may be different in Puppet Enterprise (Orchestrator). • Ready-to-go AMI’s may be faster to launch, but harder to maintain.
- 8. AUTOMATING THE CLOUD AWS Infrastructure Principles Read more https://media.amazonwebservices.com/AWS_Cloud_Best_Practices.pdf 8 • Infrastructure is failure-prone; Service robustness is achieved through redundancy. • EC2 instances should be considered volatile. • Scaling should be both vertical and horizontal. • Legacy applications aren’t magically cloud-ready. • Loose coupling improves scalability. • Isolation increases security and decreases dependencies.
- 9. AUTOMATING THE CLOUD AWS Infra: A lot of moving parts 9
- 11. • IAM Server Certificates • IAM Instance Profiles • IAM Roles • IAM Policies • IAM Role Policies • CodeDeploy Apps • CodeDeploy Deployment Groups • EC2 Instances • EC2 Elastic IP’s • CloudWatch Metrics • CloudWatch Alarms • EIP Attachments • ElastiCache Subnet Groups Provisioning Infrastructure AWS Resources • ElastiCache Clusters • ElastiCache Parameter Groups • EC2 Elastic Load Balancers • ELB Health Checks • ELB App Cookie Stickiness Policies • Key Pairs • RDS Subnet Groups • RDS Parameter Groups • RDS DB Instances • Route53 Zones • Route53 Records • S3 Buckets • S3 Policies • S3 CORS Configuration • Security Groups • SNS Topics • SNS Topic Subscriptions • VPC’s • VPC Subnets • VPC Internet Gateways • VPC (S3) Endpoints • VPC Route Tables • VPC Customer Gateways • VPC VPN Gateways • VPC DHCP Option Sets • VPC VPN Routes 11
- 13. Provisioning Infrastructure That’s a lot of clicking 13 • Manual changes are extremely error-prone. • Manual changes result in an inconsistent platform. • Collaboration is difficult; People are scared to break things. • Changes are scattered throughout the AWS console.
- 15. • Uses the same DSL as your ”regular” Puppet code. Provisioning Infrastructure Puppetlabs-AWS 15 • Most benefits from ”regular” Puppet; like relationships, defined types and the graph. • At the moment this module only supports a few of the resources in the AWS API. • Does NOT run using agents / puppet masters, but manually from your workstation using `puppet apply`. • Not as idempotent as you’d like at times. Read more https://github.com/puppetlabs/puppetlabs-aws https://forge.puppetlabs.com/puppetlabs/aws
- 16. `AWS_PROFILE=my-aws-account AWS_REGION=eu-west-1 puppet apply aws-stack.pp --templatedir ./templates/`
- 17. Provisioning Infrastructure Hashicorp Terraform • Uses HCL, which looks a lot like Puppet DSL. • Maintains a dependency graph, just like Puppet. • Runs from your workstation. • Is not AWS API feature complete, but covers most services. • Allows flexible scaling and destruction. 17 Read more https://github.com/hashicorp/terraform https://www.terraform.io/
- 19. View & try the full source https://github.com/MaxServ/Terraform-Puppet-Demo/tree/master/Terraform
- 20. Provisioning Infrastructure Puppet? 20 • Automated Infrastructure is awesome. • We like Puppet too. • Empty EC2 instances don’t serve websites. • How do we connect Puppet to the infrastructure we’ve just provisioned?
- 22. Bootstrapping Puppet Machine Configuration States 22
- 23. SLIDE CREDIT: Tim Bell, CERN – PuppetConf 2012 https://speakerdeck.com/puppetlabs/cern-accelerating-science-with-puppet https://youtu.be/-Ykb2j2ojYU?t=19m33s
- 24. Image credit: Geert Orye, https://flic.kr/p/7zDqXY (CC BY 2.0)
- 25. Bootstrapping Puppet So, Cattle. • Configuration should be at type level. Not node level. • Nodes should be replaceable. • Data on the nodes should be considered volatile. • Only versioned and cached content should be present. 25 • Provisioning should be automated; No manual intervention should be required.
- 26. Bootstrapping Puppet Puppet: Autosigning • Automates indoctrination for new nodes. • Multiple mechanisms available: Naïve, Basic and Policy-based Autosigning. 26 View & try https://github.com/MaxServ/Terraform-Puppet-Demo/blob/master/Puppet/puppet.conf Read more https://docs.puppetlabs.com/puppet/latest/reference/ssl_autosign.html Be very careful with naïve autosigning. Don’t do it in production without strict firewalls.
- 28. Bootstrapping Puppet Node Manifests using Regex 28 Read more https://docs.puppetlabs.com/puppet/latest/reference/lang_node_definitions.html#regular-expression-names
- 29. Bootstrapping Puppet Hiera • Configuration at `type` level instead of `clientcert`. • Use node-level overrides when required. • Combine with host `%H` mount points for master/slave. 29 View & try https://github.com/MaxServ/Terraform-Puppet-Demo/tree/master/Puppet/hieradata Read more https://docs.puppetlabs.com/hiera/latest/ https://docs.puppetlabs.com/guides/file_serving.html#file-server-configuration
- 31. Bootstrapping Puppet ENC’s in Foreman 31 Read more https://docs.puppetlabs.com/guides/external_nodes.html http://theforeman.org/manuals/1.10/index.html#2.Quickstart
- 32. Bootstrapping Puppet Foreman: Default Host Groups • The same idea as Autosigning in Puppet. • Uses a Foreman Plugin. • Use facts for assigning host groups. 32 Read more https://github.com/theforeman/foreman_default_hostgroup
- 33. Bootstrapping Puppet Foreman: Default Host Groups 33
- 34. Bootstrapping Puppet Assigning Configuration: Facter 34 View & try https://github.com/MaxServ/Terraform-Puppet-Demo/blob/master/Terraform/templates/userdata.tpl https://github.com/MaxServ/Terraform-Puppet-Demo/blob/master/Puppet/Vagrantfile Read more https://docs.puppetlabs.com/facter/2.4/custom_facts.html#structured-data-facts
- 35. Bootstrapping Puppet AWS: User data • User data allows you to pass configuration data to an instance during launch. • User data can be used to provide cloud-init configuration. 35
- 36. Bootstrapping Puppet AWS: User data 36
- 37. Bootstrapping Puppet Cloud-init 37 • Handles early initialization of cloud instances. • Supports EC2, CloudStack, OpenStack, OnApp, OpenNebula, RHEVm, vSphere & more. • Can install packages, do basic config and more. • Is able to bootstrap Puppet. Read more http://cloudinit.readthedocs.org/en/latest/index.html http://cloudinit.readthedocs.org/en/latest/topics/examples.html#setup-and-run-puppet
- 38. Bootstrapping Puppet Cloud-init with Puppetlabs-Aws 38 Read more https://docs.puppetlabs.com/puppet/latest/reference/lang_template_erb.html
- 39. Bootstrapping Puppet Cloud-init with Terraform 39 View & try https://github.com/MaxServ/Terraform-Puppet-Demo/blob/master/Terraform/ec2.tf Read more https://docs.puppetlabs.com/puppet/latest/reference/lang_template_erb.html
- 40. Bootstrapping Puppet Don’t forget to clean up! • Revoke Puppet-CA certificates for decommissioned nodes. • Clean up Salt keys. • Remove old reports, exported resources and catalogs from PuppetDB. • Clean connected resources like Load Balancers. 40 Sloppiness will catch up with you when it hurts the most.
- 41. Demo
- 42. Demo Terraform & Puppet THIS DEMONSTRATION INCLUDES: VIEW & TRY THE FULL SOURCE OF THIS DEMO https://github.com/MaxServ/Terraform-Puppet-Demo A B C Docker container running a Puppetmaster. AWS Stack with 4 webservers using Terraform. Webserver provisioning using Puppet based on type. 42
- 44. 44
- 45. 0416 - 30 10 00 Remco Overdijk LEAD OPERATIONS ENGINEER info@maxserv.com Questions? @MaxServ @RemzJay