Diese Präsentation wurde erfolgreich gemeldet.
Wir verwenden Ihre LinkedIn Profilangaben und Informationen zu Ihren Aktivitäten, um Anzeigen zu personalisieren und Ihnen relevantere Inhalte anzuzeigen. Sie können Ihre Anzeigeneinstellungen jederzeit ändern.

EC2 AMI Factory with Chef, Berkshelf, and Packer

12.034 Aufrufe

Veröffentlicht am

Presentation accompanying a Live Demo at the AWS Pop-Up Loft in San Francisco on using Chef + Berks + Packer to create an AWS EC2 AMI Factory.

Demo Repo available here -- https://github.com/gmiranda23/chef-ami-factory

  • DOWNLOAD FULL eBOOK INTO AVAILABLE FORMAT ......................................................................................................................... ......................................................................................................................... 1.DOWNLOAD FULL. PDF eBook here { https://tinyurl.com/y6a5rkg5 } ......................................................................................................................... 1.DOWNLOAD FULL. EPUB eBook here { https://tinyurl.com/y6a5rkg5 } ......................................................................................................................... 1.DOWNLOAD FULL. doc eBook here { https://tinyurl.com/y6a5rkg5 } ......................................................................................................................... 1.DOWNLOAD FULL. PDF eBook here { https://tinyurl.com/y6a5rkg5 } ......................................................................................................................... 1.DOWNLOAD FULL. EPUB eBook here { https://tinyurl.com/y6a5rkg5 } ......................................................................................................................... 1.DOWNLOAD FULL. doc eBook here { https://tinyurl.com/y6a5rkg5 } ......................................................................................................................... ......................................................................................................................... ......................................................................................................................... .............. Browse by Genre Available eBooks ......................................................................................................................... Art, Biography, Business, Chick Lit, Children's, Christian, Classics, Comics, Contemporary, CookeBOOK Crime, eeBOOK Fantasy, Fiction, Graphic Novels, Historical Fiction, History, Horror, Humor And Comedy, Manga, Memoir, Music, Mystery, Non Fiction, Paranormal, Philosophy, Poetry, Psychology, Religion, Romance, Science, Science Fiction, Self Help, Suspense, Spirituality, Sports, Thriller, Travel, Young Adult,
       Antworten 
    Sind Sie sicher, dass Sie …  Ja  Nein
    Ihre Nachricht erscheint hier
  • DOWNLOAD FULL BOOKS, INTO AVAILABLE FORMAT ......................................................................................................................... ......................................................................................................................... 1.DOWNLOAD FULL. PDF EBOOK here { https://tinyurl.com/y3nhqquc } ......................................................................................................................... 1.DOWNLOAD FULL. EPUB Ebook here { https://tinyurl.com/y3nhqquc } ......................................................................................................................... 1.DOWNLOAD FULL. doc Ebook here { https://tinyurl.com/y3nhqquc } ......................................................................................................................... 1.DOWNLOAD FULL. PDF EBOOK here { https://tinyurl.com/y3nhqquc } ......................................................................................................................... 1.DOWNLOAD FULL. EPUB Ebook here { https://tinyurl.com/y3nhqquc } ......................................................................................................................... 1.DOWNLOAD FULL. doc Ebook here { https://tinyurl.com/y3nhqquc } ......................................................................................................................... ......................................................................................................................... ......................................................................................................................... .............. Browse by Genre Available eBooks ......................................................................................................................... Art, Biography, Business, Chick Lit, Children's, Christian, Classics, Comics, Contemporary, Cookbooks, Crime, Ebooks, Fantasy, Fiction, Graphic Novels, Historical Fiction, History, Horror, Humor And Comedy, Manga, Memoir, Music, Mystery, Non Fiction, Paranormal, Philosophy, Poetry, Psychology, Religion, Romance, Science, Science Fiction, Self Help, Suspense, Spirituality, Sports, Thriller, Travel, Young Adult,
       Antworten 
    Sind Sie sicher, dass Sie …  Ja  Nein
    Ihre Nachricht erscheint hier
  • DOWNLOAD FULL. BOOKS INTO AVAILABLE FORMAT ......................................................................................................................... ......................................................................................................................... 1.DOWNLOAD FULL. PDF EBOOK here { https://tinyurl.com/y8nn3gmc } ......................................................................................................................... 1.DOWNLOAD FULL. EPUB Ebook here { https://tinyurl.com/y8nn3gmc } ......................................................................................................................... 1.DOWNLOAD FULL. doc Ebook here { https://tinyurl.com/y8nn3gmc } ......................................................................................................................... 1.DOWNLOAD FULL. PDF EBOOK here { https://tinyurl.com/y8nn3gmc } ......................................................................................................................... 1.DOWNLOAD FULL. EPUB Ebook here { https://tinyurl.com/y8nn3gmc } ......................................................................................................................... 1.DOWNLOAD FULL. doc Ebook here { https://tinyurl.com/y8nn3gmc } ......................................................................................................................... ......................................................................................................................... ......................................................................................................................... .............. Browse by Genre Available eBooks ......................................................................................................................... Art, Biography, Business, Chick Lit, Children's, Christian, Classics, Comics, Contemporary, Cookbooks, Crime, Ebooks, Fantasy, Fiction, Graphic Novels, Historical Fiction, History, Horror, Humor And Comedy, Manga, Memoir, Music, Mystery, Non Fiction, Paranormal, Philosophy, Poetry, Psychology, Religion, Romance, Science, Science Fiction, Self Help, Suspense, Spirituality, Sports, Thriller, Travel, Young Adult,
       Antworten 
    Sind Sie sicher, dass Sie …  Ja  Nein
    Ihre Nachricht erscheint hier
  • Hello! Get Your Professional Job-Winning Resume Here - Check our website! https://vk.cc/818RFv
       Antworten 
    Sind Sie sicher, dass Sie …  Ja  Nein
    Ihre Nachricht erscheint hier

EC2 AMI Factory with Chef, Berkshelf, and Packer

  1. 1. AWS Provisioning with Chef Creating an AMI Factory George Miranda Partner Integration Engineer AWS Pop-up Loft June 23, 2014
  2. 2. Introductions • George Miranda ! • Partner Integration Engineer • Consultant • Infrastructure guy ! • gmiranda@getchef.com • gmiranda23 Twitter | GitHub | IRC | *:*
  3. 3. Deploying to AWS EC2 • A pretty BIG topic • FAQ: where does Chef fit? • Here’s what we’ll cover • Brief intro to Chef • Basic intro to provisioning options • AWS specific challenge: auto-scaling • Focus on AMI management • Live Demo • Next Steps
  4. 4. Chef Basics In 90 seconds or less • Configuration Management & Automation Framework • Scalable, Extensible, Testable • Infrastructure as Code • Data Driven Cookbooks • Run the same code: Dev/Stg/Prod, On-Prem or AWS • “Recipes” are a collection of “resources” • “Cookbooks” are a collection of “recipes” • 1,400+ Community Cookbooks Available • “Roles” are an ordered list of recipes to apply • “Nodes” can have one or more “roles” • “Knife” is a Chef’s go-to tool
  5. 5. Provisioning a node with knife-ec2 ! knife  ec2  server  create  -­‐r  "role[webserver]"  -­‐I  ami-­‐2d4aa444          -­‐-­‐flavor  m1.small  -­‐S  aws_ssh_key  -­‐Z  us-­‐east-­‐1b  -­‐N  my_webserver1 • Provisions a new m1.small in us-east-1b • Waits for instance-id & network settings from AWS • Waits for ssh or winrm • Issues remote commands • Install Chef • Configure Chef • Run chef-client
  6. 6. Provisioning with chef-metal require  'chef_metal_fog'   ! with_machine_options  :bootstrap_options  =>  {      :key_name  =>  'aws_ssh_key',      :image_id  =>  'ami-­‐2d4aa444',      :flavor_id  =>  'm1.small',      :region  =>  'us-­‐east-­‐1b'   }   ! machine  'my_database1'  do      role  'database'      tag  'mysql_master'   end   ! num_webservers  =  5   ! 1.upto(num_webservers)  do  |i|      machine  "my_webserver#{i}"  do          role  'webserver'    end   end • Declare infrastructure topology in a recipe • Configure multiple VM "drivers" • Ensure my_database1 is present • Then ensure 5 web servers are present • Self-healing (convergent) • Version your infrastructure • Almost 1.0.0
  7. 7. {        ~500  lines  of  JSON   !      "UserData"  :  {  "Fn::Base64"  :  {  "Fn::Join"  :  ["",  [                "#!/bin/bash  -­‐vn",                "function  error_exitn",                "{n",                "  cfn-­‐signal  -­‐e  1  -­‐r  "$1"  '",  {  "Ref"  :   "ChefClientWaitHandle"  },  "'n",                "exit  1n",                "}n",     !            #{Bunch  of  stuff  to  get  Chef  installed/configured}   !            "#  If  all  went  well,  signal  successn",                "cfn-­‐signal  -­‐e  $?  -­‐r  'Chef  Server  configuration'   '",  {  "Ref"  :  "ChefClientWaitHandle"  },  "'n"            ]]}},   !      ~500  more  lines  of  JSON   } Provisioning with CloudFormation
  8. 8. Why CloudFormation? JSON All The Things! • Create new Security Groups • Provision EC2 stack topology • Elastic Load Balancers • Elastic IPs • Provision AWS PaaS connections • RDS • SQS • S3 Buckets • etc • Create EC2 Auto-Scaling Groups
  9. 9. The Auto-Scaling Challenge
 Meeting Demand Fast • Typically triggered by utilization alerts • You are already falling behind by the time your infrastructure responds • You need capacity… FAST! • You already lose response time during instantiation • 3-5 minutes is an eternity when your application is choking
  10. 10. • Flexibility • Managing Configuration Drift • Rapid incremental change • Avoid golden image sprawl • Systems are not always disposable • Avoid asynchronous VM configuration • Infrastructure as Code Deploying fully-baked AMIs
 Why Would You Still Use Configuration Management?
  11. 11. Pre-Loaded AMIs
 Flexibility and Performance • Bake in the most expensive components • Minimal Configuration occurs after initialization (e.g. systems integration tasks) • Manage your pre-loaded AMI creation in a Chef Recipe • Chef Recipes are idempotent • Include those same recipes in your node’s run_list
  12. 12. Today’s Demo
 A Live Demo? SRSLY? • Creating an AMI Factory with Chef • Chef Recipe • Packer • Berkshelf ! • For today’s Demo, we’ll pre-bake a simple Nginx Web Server* ! * (you would probably never bake in something that inexpensive in “the real world”)
  13. 13. Packer • Packer is a tool for creating identical machine images for multiple platforms from a single source configuration • A HashiCorp Project • http://www.packer.io
  14. 14. Berkshelf • Berkshelf is a bundler-like dependency manager for Chef cookbooks • Encourages treating your cookbooks as Libraries or Applications • Makes it very easy to automatically download and use Chef Community Cookbooks • Started at Riot Games • http://berkshelf.com
  15. 15. Live Demo Creating a Pre-Loaded AMI Instance-Store AMI & EBS-Backed AMI: building in parallel
  16. 16. A Tour of Today’s Demo
 The Recipe include_recipe  'apt'   include_recipe  'nginx'   ! directory  '/var/www/nginx-­‐default'  do      owner  'www-­‐data'      group  'www-­‐data'      mode  '0755'      recursive  true      action  :create   end   ! file  '/var/www/nginx-­‐default/index.html'  do      owner  'www-­‐data'      group  'www-­‐data'      mode  '0755'      content  'Hello  World  from  the  AWS  Pop-­‐up  Loft!'      action  :create   end
  17. 17. A Tour of Today’s Demo
 The Berksfile site  :opscode   ! cookbook  'apt'   cookbook  'nginx'   ! metadata
  18. 18. A Tour of Today’s Demo
 The packer.json file — building Instance-Store AMIs    "builders":  [          {  "type":  "amazon-­‐instance",              "access_key":  "{{user  `aws_access_key_id`}}",              "secret_key":  "{{user  `aws_secret_key`}}",              "region":  "us-­‐east-­‐1",              "source_ami":  "ami-­‐6f969506",              "instance_type":  "m1.xlarge",              "ssh_username":  "ubuntu",              "ami_name":  "packer-­‐instance_store_nginx_{{timestamp}}",   !            "bundle_vol_command":  "sudo  -­‐n  #{very  long  ami  tool  e2-­‐bundle-­‐vol  command}”,              "bundle_upload_command":  "sudo  -­‐n  #{very  long  ami  tool  e2-­‐upload-­‐bundle  command}”,   !            "account_id":  "{{user  `account_id`}}",              "s3_bucket":  "{{user  `s3_bucket`}}",              "x509_cert_path":  "{{user  `x509_cert_path`}}",              "x509_key_path":  "{{user  `x509_key_path`}}"          },
  19. 19. A Tour of Today’s Demo
 The packer.json file — building EBS-backed AMIs builders  (continued)   !        {  "type":  "amazon-­‐ebs",              "access_key":  "{{user  `aws_access_key_id`}}",              "secret_key":  "{{user  `aws_secret_key`}}",              "region":  "us-­‐east-­‐1",              "source_ami":  "ami-­‐2b0b1442",              "instance_type":  "m1.xlarge",              "ssh_username":  "ubuntu",              "ami_name":  "packer-­‐ebs_nginx_{{timestamp}}"          }      ],
  20. 20. A Tour of Today’s Demo
 The packer.json file — provisioners    "provisioners"  :  [          {  "type":  "chef-­‐solo",              "cookbook_paths":  [  "../vendor/cookbooks"  ],              "run_list":  [  "packer::default"  ]          },   !        {  "type":  "shell",              "inline":  [                  "sudo  apt-­‐get  install  -­‐y  ruby  unzip  kpartx",                  "wget  http://s3.amazonaws.com/ec2-­‐downloads/ec2-­‐ami-­‐tools.zip",                  "sudo  mkdir  -­‐p  /usr/local/ec2",                  "sudo  unzip  ec2-­‐ami-­‐tools.zip  -­‐d  /usr/local/ec2",                  "sudo  mv  /usr/local/ec2/ec2-­‐ami-­‐tools-­‐*  /usr/local/ec2/ec2-­‐ami-­‐tools/"              ],              "only":  [  "amazon-­‐instance"  ]          }    ]
  21. 21. A Tour of Today’s Demo
 The packer command (script) #!/bin/bash   rm  -­‐r  ../vendor/cookbooks   berks  install  -­‐-­‐path  ../vendor/cookbooks   packer  build        -­‐var  "account_id=$AWS_ACCOUNT_ID"        -­‐var  "aws_access_key_id=$AWS_ACCESS_KEY_ID"        -­‐var  "aws_secret_key=$AWS_SECRET_ACCESS_KEY"        -­‐var  "x509_cert_path=$AWS_X509_CERT_PATH"        -­‐var  "x509_key_path=$AWS_X509_KEY_PATH"        -­‐var  "s3_bucket=chef-­‐packer-­‐bucket"        packer.json
  22. 22. Live Demo Checkpoint Creating a Pre-Loaded AMI Instance-Store AMI & EBS-Backed AMI: building in parallel
  23. 23. Ready to Use Our AMIs
 Run it the best way for your environment • Use these AMIs in your CloudFormation Templates • Define UserData to run "chef-client -j run_list.json" • Example run_list.json { "run_list": [ "role[webserver]" ] } ! • Leave integration tasks to Chef • Recommended: run chef-client periodically
  24. 24. Pitfalls
 Or How I Learned to Stop Worrying and Not Rage Against Pre-Baked AMIs • Beware Image Sprawl • Pre-Load AMIs only for critical Autoscaling Instances • Just Enough OS everywhere else • Generate AMIs with a CI Pipeline • Purge previous AMIs methodically • Cycle running instances methodically ! • Note: I am NOT suggesting everyone start making pre-loaded AMIs
  25. 25. Further Reading Next Steps From here • https://github.com/gmiranda23/chef-ami-factory • Unit testing your AMIs (e.g. serverspec) • AWS Chef Cookbook • ebs_volume • ebs_raid • elastic_ip • elastic_lb • resource_tag • http://packer.io • http://community.opscode.com/cookbooks • http://learnchef.com/
  26. 26. Thank You! George Miranda ! gmiranda@getchef.com @gmiranda23 ! Demo Repo — https://github.com/gmiranda23/chef-ami-factory ! Packer Demo primer found at — http://engineering.cotap.com

×