SlideShare ist ein Scribd-Unternehmen logo
1 von 13
Baking Custom AMIs Made Easy
June 12, 2014
Joe Block <jpb@numenta.com>
Why not just use configuration
management?
Instances would get the most current version of all
rpms, Python eggs, Ruby gems, etc
Reliability
If any of your upstream repositories have an issue,
suddenly you can’t scale up with configuration
management. Or move to a new AZ.
Rapid Response
When we scale up, we really want the
performance boost 5 minutes ago, not 10 minutes
from now.
Cost
A ten minute configuration run means you’re
paying for an extra hour of EC2 time every 6
instances.
So how do we make it less painful?
AMI Toolkit
• chef-solo / masterless puppet / ansible etc
• fpm - one line rpm creation
• packer - easy AMI building
• jenkins - trigger ami build every commit
chef solo
• I’m not crazy, we use configuration
management to create the master AMIs.
• Keep the instances used in bake process out of
your chef server = 1 less SPOF in your life.
• No need to purge certs from resulting AMI
fpm
• Packing your app in rpm/deb files simplifies your
chef recipes - keep the app-specific things in the
app’s git repo
• Creating packages doesn’t have to be a PITA. fpm
reduces package creation to a literal one-liner -
fpm --verbose -s dir -t rpm --architecture
noarch --name yourrpm.rpm --version 1.1.1
—iteration 3 -C /path/to/fakeroot etc
packer
• Supports multiple output formats - AWS,
VMWare, etc
• Simple JSON configuration files
• Can automatically copy the resulting AMIs to
the regions you’re deployed in
• Written in Go, no dependency hell to interfere
with the other software installed on your build
box
jenkins
• Keep the humans out of the loop
• Generate AMIs every commit
• Integration test every AMI build
AMI Toolkit Summary
• chef-solo / masterless puppet /ansible etc
• fpm - https://github.com/jordansissel/fpm
• packer - http://www.packer.io/
• S3 yum plugin - https://github.com/jbraeuer/yum-
s3-plugin
• jenkins - http://jenkins-ci.org/
Joe Block <jpb@numenta.com>

Weitere ähnliche Inhalte

Was ist angesagt?

Talk About Configuration Management
Talk About Configuration ManagementTalk About Configuration Management
Talk About Configuration ManagementGuixing Bai
 
Scale like a pro with Gearman
Scale like a pro with GearmanScale like a pro with Gearman
Scale like a pro with GearmanAmal Raghav
 
BAM experiences in large scale deployments
BAM experiences in large scale deploymentsBAM experiences in large scale deployments
BAM experiences in large scale deploymentsAIMS Innovation
 
Immutable infrastructure isn’t the answer
Immutable infrastructure isn’t the answerImmutable infrastructure isn’t the answer
Immutable infrastructure isn’t the answerSam Bashton
 
Ansible meetup-0915
Ansible meetup-0915Ansible meetup-0915
Ansible meetup-0915Pierre Mavro
 
uPerform implementation roadmap
uPerform implementation roadmapuPerform implementation roadmap
uPerform implementation roadmapKumar Kolaganti
 
Sử dụng Gulp trong phát triển theme
Sử dụng Gulp trong phát triển themeSử dụng Gulp trong phát triển theme
Sử dụng Gulp trong phát triển themeĐược Nguyễn
 
What's new in Symfony3
What's new in Symfony3What's new in Symfony3
What's new in Symfony3Yuki MAEJIMA
 
Open analytics meetup alex poon (1)
Open analytics meetup   alex poon (1)Open analytics meetup   alex poon (1)
Open analytics meetup alex poon (1)Open Analytics
 
Scholarly Publishing with Omeka - part 1: Local installation
Scholarly Publishing with Omeka - part 1: Local installationScholarly Publishing with Omeka - part 1: Local installation
Scholarly Publishing with Omeka - part 1: Local installationBrian Zelip
 
Growing MongoDB on AWS
Growing MongoDB on AWSGrowing MongoDB on AWS
Growing MongoDB on AWScolinthehowe
 
Coffee script grunt
Coffee script gruntCoffee script grunt
Coffee script gruntKien Pham
 
JIT Compilation for VA Smalltalk
JIT Compilation for VA SmalltalkJIT Compilation for VA Smalltalk
JIT Compilation for VA SmalltalkESUG
 
Micro services architecture and service fabric
Micro services architecture and service fabricMicro services architecture and service fabric
Micro services architecture and service fabricLuis Valencia
 
Build & test once, deploy anywhere - Vday.hu 2016
Build & test once, deploy anywhere - Vday.hu 2016Build & test once, deploy anywhere - Vday.hu 2016
Build & test once, deploy anywhere - Vday.hu 2016Zsolt Molnar
 

Was ist angesagt? (20)

Talk About Configuration Management
Talk About Configuration ManagementTalk About Configuration Management
Talk About Configuration Management
 
Ansible
AnsibleAnsible
Ansible
 
Scale like a pro with Gearman
Scale like a pro with GearmanScale like a pro with Gearman
Scale like a pro with Gearman
 
BAM experiences in large scale deployments
BAM experiences in large scale deploymentsBAM experiences in large scale deployments
BAM experiences in large scale deployments
 
Immutable infrastructure isn’t the answer
Immutable infrastructure isn’t the answerImmutable infrastructure isn’t the answer
Immutable infrastructure isn’t the answer
 
Ansible meetup-0915
Ansible meetup-0915Ansible meetup-0915
Ansible meetup-0915
 
uPerform implementation roadmap
uPerform implementation roadmapuPerform implementation roadmap
uPerform implementation roadmap
 
Ansible
AnsibleAnsible
Ansible
 
Ansible, PHP and EC2
Ansible, PHP and EC2Ansible, PHP and EC2
Ansible, PHP and EC2
 
Sử dụng Gulp trong phát triển theme
Sử dụng Gulp trong phát triển themeSử dụng Gulp trong phát triển theme
Sử dụng Gulp trong phát triển theme
 
What's new in Symfony3
What's new in Symfony3What's new in Symfony3
What's new in Symfony3
 
Open analytics meetup alex poon (1)
Open analytics meetup   alex poon (1)Open analytics meetup   alex poon (1)
Open analytics meetup alex poon (1)
 
Scholarly Publishing with Omeka - part 1: Local installation
Scholarly Publishing with Omeka - part 1: Local installationScholarly Publishing with Omeka - part 1: Local installation
Scholarly Publishing with Omeka - part 1: Local installation
 
Pegando carona no Command Bus
Pegando carona no Command BusPegando carona no Command Bus
Pegando carona no Command Bus
 
Scholarly Publishing with Omeka - part 1: Local installation
Scholarly Publishing with Omeka - part 1: Local installationScholarly Publishing with Omeka - part 1: Local installation
Scholarly Publishing with Omeka - part 1: Local installation
 
Growing MongoDB on AWS
Growing MongoDB on AWSGrowing MongoDB on AWS
Growing MongoDB on AWS
 
Coffee script grunt
Coffee script gruntCoffee script grunt
Coffee script grunt
 
JIT Compilation for VA Smalltalk
JIT Compilation for VA SmalltalkJIT Compilation for VA Smalltalk
JIT Compilation for VA Smalltalk
 
Micro services architecture and service fabric
Micro services architecture and service fabricMicro services architecture and service fabric
Micro services architecture and service fabric
 
Build & test once, deploy anywhere - Vday.hu 2016
Build & test once, deploy anywhere - Vday.hu 2016Build & test once, deploy anywhere - Vday.hu 2016
Build & test once, deploy anywhere - Vday.hu 2016
 

Ähnlich wie Lightning overview of creating custom AMIs

Running CentOS on the Facebook fleet
Running CentOS on the Facebook fleetRunning CentOS on the Facebook fleet
Running CentOS on the Facebook fleetDavide Cavalca
 
Continuous Integration
Continuous IntegrationContinuous Integration
Continuous IntegrationJoerg Henning
 
CNPM: Private NPM for Company / 企業級私有NPM
CNPM: Private NPM for Company / 企業級私有NPMCNPM: Private NPM for Company / 企業級私有NPM
CNPM: Private NPM for Company / 企業級私有NPMFeng Yuan
 
Deploying software at Scale
Deploying software at ScaleDeploying software at Scale
Deploying software at ScaleKris Buytaert
 
Puppet Development Workflow
Puppet Development WorkflowPuppet Development Workflow
Puppet Development WorkflowJeffery Smith
 
Distributed Queue System using Gearman
Distributed Queue System using GearmanDistributed Queue System using Gearman
Distributed Queue System using GearmanEric Cho
 
Symfony 2 under control
Symfony 2 under controlSymfony 2 under control
Symfony 2 under controlMax Romanovsky
 
How Percolate uses CFEngine to Manage AWS Stateless Infrastructure
How Percolate uses CFEngine to Manage AWS Stateless InfrastructureHow Percolate uses CFEngine to Manage AWS Stateless Infrastructure
How Percolate uses CFEngine to Manage AWS Stateless InfrastructurePercolate
 
perlbrew yapcasia 2010
perlbrew yapcasia 2010perlbrew yapcasia 2010
perlbrew yapcasia 2010Kang-min Liu
 
Build and deployment
Build and deploymentBuild and deployment
Build and deploymentWO Community
 
Automated Deployment and Configuration Engines. Ansible
Automated Deployment and Configuration Engines. AnsibleAutomated Deployment and Configuration Engines. Ansible
Automated Deployment and Configuration Engines. AnsibleAlberto Molina Coballes
 
DCRUG: Achieving Development-Production Parity
DCRUG: Achieving Development-Production ParityDCRUG: Achieving Development-Production Parity
DCRUG: Achieving Development-Production ParityGeoff Harcourt
 
VMware studio practice in TIM
VMware studio practice in TIMVMware studio practice in TIM
VMware studio practice in TIMYi-Huan Chan
 
Puppet Camp New York 2014: Streamlining Puppet Development Workflow
Puppet Camp New York 2014: Streamlining Puppet Development Workflow Puppet Camp New York 2014: Streamlining Puppet Development Workflow
Puppet Camp New York 2014: Streamlining Puppet Development Workflow Puppet
 
Steamlining your puppet development workflow
Steamlining your puppet development workflowSteamlining your puppet development workflow
Steamlining your puppet development workflowTomas Doran
 
Cloud-powered Continuous Integration and Deployment architectures - Jinesh Varia
Cloud-powered Continuous Integration and Deployment architectures - Jinesh VariaCloud-powered Continuous Integration and Deployment architectures - Jinesh Varia
Cloud-powered Continuous Integration and Deployment architectures - Jinesh VariaAmazon Web Services
 
Continuous Integration at Mollie
Continuous Integration at MollieContinuous Integration at Mollie
Continuous Integration at Molliewillemstuursma
 
Towards Continuous Deployment with Django
Towards Continuous Deployment with DjangoTowards Continuous Deployment with Django
Towards Continuous Deployment with DjangoRoger Barnes
 

Ähnlich wie Lightning overview of creating custom AMIs (20)

Running CentOS on the Facebook fleet
Running CentOS on the Facebook fleetRunning CentOS on the Facebook fleet
Running CentOS on the Facebook fleet
 
Continuous Integration
Continuous IntegrationContinuous Integration
Continuous Integration
 
CNPM: Private NPM for Company / 企業級私有NPM
CNPM: Private NPM for Company / 企業級私有NPMCNPM: Private NPM for Company / 企業級私有NPM
CNPM: Private NPM for Company / 企業級私有NPM
 
Deploying software at Scale
Deploying software at ScaleDeploying software at Scale
Deploying software at Scale
 
Chef
ChefChef
Chef
 
Puppet Development Workflow
Puppet Development WorkflowPuppet Development Workflow
Puppet Development Workflow
 
Distributed Queue System using Gearman
Distributed Queue System using GearmanDistributed Queue System using Gearman
Distributed Queue System using Gearman
 
Symfony 2 under control
Symfony 2 under controlSymfony 2 under control
Symfony 2 under control
 
How Percolate uses CFEngine to Manage AWS Stateless Infrastructure
How Percolate uses CFEngine to Manage AWS Stateless InfrastructureHow Percolate uses CFEngine to Manage AWS Stateless Infrastructure
How Percolate uses CFEngine to Manage AWS Stateless Infrastructure
 
perlbrew yapcasia 2010
perlbrew yapcasia 2010perlbrew yapcasia 2010
perlbrew yapcasia 2010
 
Build and deployment
Build and deploymentBuild and deployment
Build and deployment
 
Automated Deployment and Configuration Engines. Ansible
Automated Deployment and Configuration Engines. AnsibleAutomated Deployment and Configuration Engines. Ansible
Automated Deployment and Configuration Engines. Ansible
 
DCRUG: Achieving Development-Production Parity
DCRUG: Achieving Development-Production ParityDCRUG: Achieving Development-Production Parity
DCRUG: Achieving Development-Production Parity
 
VMware studio practice in TIM
VMware studio practice in TIMVMware studio practice in TIM
VMware studio practice in TIM
 
Puppet Camp New York 2014: Streamlining Puppet Development Workflow
Puppet Camp New York 2014: Streamlining Puppet Development Workflow Puppet Camp New York 2014: Streamlining Puppet Development Workflow
Puppet Camp New York 2014: Streamlining Puppet Development Workflow
 
Steamlining your puppet development workflow
Steamlining your puppet development workflowSteamlining your puppet development workflow
Steamlining your puppet development workflow
 
Cloud-powered Continuous Integration and Deployment architectures - Jinesh Varia
Cloud-powered Continuous Integration and Deployment architectures - Jinesh VariaCloud-powered Continuous Integration and Deployment architectures - Jinesh Varia
Cloud-powered Continuous Integration and Deployment architectures - Jinesh Varia
 
Continuous Integration at Mollie
Continuous Integration at MollieContinuous Integration at Mollie
Continuous Integration at Mollie
 
CommandBox at CFCamp 2014
CommandBox at CFCamp 2014CommandBox at CFCamp 2014
CommandBox at CFCamp 2014
 
Towards Continuous Deployment with Django
Towards Continuous Deployment with DjangoTowards Continuous Deployment with Django
Towards Continuous Deployment with Django
 

Kürzlich hochgeladen

"I see eyes in my soup": How Delivery Hero implemented the safety system for ...
"I see eyes in my soup": How Delivery Hero implemented the safety system for ..."I see eyes in my soup": How Delivery Hero implemented the safety system for ...
"I see eyes in my soup": How Delivery Hero implemented the safety system for ...Zilliz
 
FWD Group - Insurer Innovation Award 2024
FWD Group - Insurer Innovation Award 2024FWD Group - Insurer Innovation Award 2024
FWD Group - Insurer Innovation Award 2024The Digital Insurer
 
Rising Above_ Dubai Floods and the Fortitude of Dubai International Airport.pdf
Rising Above_ Dubai Floods and the Fortitude of Dubai International Airport.pdfRising Above_ Dubai Floods and the Fortitude of Dubai International Airport.pdf
Rising Above_ Dubai Floods and the Fortitude of Dubai International Airport.pdfOrbitshub
 
DEV meet-up UiPath Document Understanding May 7 2024 Amsterdam
DEV meet-up UiPath Document Understanding May 7 2024 AmsterdamDEV meet-up UiPath Document Understanding May 7 2024 Amsterdam
DEV meet-up UiPath Document Understanding May 7 2024 AmsterdamUiPathCommunity
 
WSO2's API Vision: Unifying Control, Empowering Developers
WSO2's API Vision: Unifying Control, Empowering DevelopersWSO2's API Vision: Unifying Control, Empowering Developers
WSO2's API Vision: Unifying Control, Empowering DevelopersWSO2
 
CNIC Information System with Pakdata Cf In Pakistan
CNIC Information System with Pakdata Cf In PakistanCNIC Information System with Pakdata Cf In Pakistan
CNIC Information System with Pakdata Cf In Pakistandanishmna97
 
Elevate Developer Efficiency & build GenAI Application with Amazon Q​
Elevate Developer Efficiency & build GenAI Application with Amazon Q​Elevate Developer Efficiency & build GenAI Application with Amazon Q​
Elevate Developer Efficiency & build GenAI Application with Amazon Q​Bhuvaneswari Subramani
 
DBX First Quarter 2024 Investor Presentation
DBX First Quarter 2024 Investor PresentationDBX First Quarter 2024 Investor Presentation
DBX First Quarter 2024 Investor PresentationDropbox
 
Apidays New York 2024 - Passkeys: Developing APIs to enable passwordless auth...
Apidays New York 2024 - Passkeys: Developing APIs to enable passwordless auth...Apidays New York 2024 - Passkeys: Developing APIs to enable passwordless auth...
Apidays New York 2024 - Passkeys: Developing APIs to enable passwordless auth...apidays
 
ICT role in 21st century education and its challenges
ICT role in 21st century education and its challengesICT role in 21st century education and its challenges
ICT role in 21st century education and its challengesrafiqahmad00786416
 
Polkadot JAM Slides - Token2049 - By Dr. Gavin Wood
Polkadot JAM Slides - Token2049 - By Dr. Gavin WoodPolkadot JAM Slides - Token2049 - By Dr. Gavin Wood
Polkadot JAM Slides - Token2049 - By Dr. Gavin WoodJuan lago vázquez
 
Why Teams call analytics are critical to your entire business
Why Teams call analytics are critical to your entire businessWhy Teams call analytics are critical to your entire business
Why Teams call analytics are critical to your entire businesspanagenda
 
Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...
Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...
Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...apidays
 
Platformless Horizons for Digital Adaptability
Platformless Horizons for Digital AdaptabilityPlatformless Horizons for Digital Adaptability
Platformless Horizons for Digital AdaptabilityWSO2
 
Strategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff
Strategize a Smooth Tenant-to-tenant Migration and Copilot TakeoffStrategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff
Strategize a Smooth Tenant-to-tenant Migration and Copilot Takeoffsammart93
 
Apidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, Adobe
Apidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, AdobeApidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, Adobe
Apidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, Adobeapidays
 
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers:  A Deep Dive into Serverless Spatial Data and FMECloud Frontiers:  A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FMESafe Software
 
Apidays New York 2024 - APIs in 2030: The Risk of Technological Sleepwalk by ...
Apidays New York 2024 - APIs in 2030: The Risk of Technological Sleepwalk by ...Apidays New York 2024 - APIs in 2030: The Risk of Technological Sleepwalk by ...
Apidays New York 2024 - APIs in 2030: The Risk of Technological Sleepwalk by ...apidays
 
Finding Java's Hidden Performance Traps @ DevoxxUK 2024
Finding Java's Hidden Performance Traps @ DevoxxUK 2024Finding Java's Hidden Performance Traps @ DevoxxUK 2024
Finding Java's Hidden Performance Traps @ DevoxxUK 2024Victor Rentea
 

Kürzlich hochgeladen (20)

"I see eyes in my soup": How Delivery Hero implemented the safety system for ...
"I see eyes in my soup": How Delivery Hero implemented the safety system for ..."I see eyes in my soup": How Delivery Hero implemented the safety system for ...
"I see eyes in my soup": How Delivery Hero implemented the safety system for ...
 
FWD Group - Insurer Innovation Award 2024
FWD Group - Insurer Innovation Award 2024FWD Group - Insurer Innovation Award 2024
FWD Group - Insurer Innovation Award 2024
 
Rising Above_ Dubai Floods and the Fortitude of Dubai International Airport.pdf
Rising Above_ Dubai Floods and the Fortitude of Dubai International Airport.pdfRising Above_ Dubai Floods and the Fortitude of Dubai International Airport.pdf
Rising Above_ Dubai Floods and the Fortitude of Dubai International Airport.pdf
 
DEV meet-up UiPath Document Understanding May 7 2024 Amsterdam
DEV meet-up UiPath Document Understanding May 7 2024 AmsterdamDEV meet-up UiPath Document Understanding May 7 2024 Amsterdam
DEV meet-up UiPath Document Understanding May 7 2024 Amsterdam
 
WSO2's API Vision: Unifying Control, Empowering Developers
WSO2's API Vision: Unifying Control, Empowering DevelopersWSO2's API Vision: Unifying Control, Empowering Developers
WSO2's API Vision: Unifying Control, Empowering Developers
 
CNIC Information System with Pakdata Cf In Pakistan
CNIC Information System with Pakdata Cf In PakistanCNIC Information System with Pakdata Cf In Pakistan
CNIC Information System with Pakdata Cf In Pakistan
 
Elevate Developer Efficiency & build GenAI Application with Amazon Q​
Elevate Developer Efficiency & build GenAI Application with Amazon Q​Elevate Developer Efficiency & build GenAI Application with Amazon Q​
Elevate Developer Efficiency & build GenAI Application with Amazon Q​
 
DBX First Quarter 2024 Investor Presentation
DBX First Quarter 2024 Investor PresentationDBX First Quarter 2024 Investor Presentation
DBX First Quarter 2024 Investor Presentation
 
Apidays New York 2024 - Passkeys: Developing APIs to enable passwordless auth...
Apidays New York 2024 - Passkeys: Developing APIs to enable passwordless auth...Apidays New York 2024 - Passkeys: Developing APIs to enable passwordless auth...
Apidays New York 2024 - Passkeys: Developing APIs to enable passwordless auth...
 
ICT role in 21st century education and its challenges
ICT role in 21st century education and its challengesICT role in 21st century education and its challenges
ICT role in 21st century education and its challenges
 
Polkadot JAM Slides - Token2049 - By Dr. Gavin Wood
Polkadot JAM Slides - Token2049 - By Dr. Gavin WoodPolkadot JAM Slides - Token2049 - By Dr. Gavin Wood
Polkadot JAM Slides - Token2049 - By Dr. Gavin Wood
 
Why Teams call analytics are critical to your entire business
Why Teams call analytics are critical to your entire businessWhy Teams call analytics are critical to your entire business
Why Teams call analytics are critical to your entire business
 
Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...
Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...
Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...
 
Understanding the FAA Part 107 License ..
Understanding the FAA Part 107 License ..Understanding the FAA Part 107 License ..
Understanding the FAA Part 107 License ..
 
Platformless Horizons for Digital Adaptability
Platformless Horizons for Digital AdaptabilityPlatformless Horizons for Digital Adaptability
Platformless Horizons for Digital Adaptability
 
Strategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff
Strategize a Smooth Tenant-to-tenant Migration and Copilot TakeoffStrategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff
Strategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff
 
Apidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, Adobe
Apidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, AdobeApidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, Adobe
Apidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, Adobe
 
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers:  A Deep Dive into Serverless Spatial Data and FMECloud Frontiers:  A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
 
Apidays New York 2024 - APIs in 2030: The Risk of Technological Sleepwalk by ...
Apidays New York 2024 - APIs in 2030: The Risk of Technological Sleepwalk by ...Apidays New York 2024 - APIs in 2030: The Risk of Technological Sleepwalk by ...
Apidays New York 2024 - APIs in 2030: The Risk of Technological Sleepwalk by ...
 
Finding Java's Hidden Performance Traps @ DevoxxUK 2024
Finding Java's Hidden Performance Traps @ DevoxxUK 2024Finding Java's Hidden Performance Traps @ DevoxxUK 2024
Finding Java's Hidden Performance Traps @ DevoxxUK 2024
 

Lightning overview of creating custom AMIs

  • 1. Baking Custom AMIs Made Easy June 12, 2014 Joe Block <jpb@numenta.com>
  • 2.
  • 3. Why not just use configuration management? Instances would get the most current version of all rpms, Python eggs, Ruby gems, etc
  • 4. Reliability If any of your upstream repositories have an issue, suddenly you can’t scale up with configuration management. Or move to a new AZ.
  • 5. Rapid Response When we scale up, we really want the performance boost 5 minutes ago, not 10 minutes from now.
  • 6. Cost A ten minute configuration run means you’re paying for an extra hour of EC2 time every 6 instances.
  • 7. So how do we make it less painful?
  • 8. AMI Toolkit • chef-solo / masterless puppet / ansible etc • fpm - one line rpm creation • packer - easy AMI building • jenkins - trigger ami build every commit
  • 9. chef solo • I’m not crazy, we use configuration management to create the master AMIs. • Keep the instances used in bake process out of your chef server = 1 less SPOF in your life. • No need to purge certs from resulting AMI
  • 10. fpm • Packing your app in rpm/deb files simplifies your chef recipes - keep the app-specific things in the app’s git repo • Creating packages doesn’t have to be a PITA. fpm reduces package creation to a literal one-liner - fpm --verbose -s dir -t rpm --architecture noarch --name yourrpm.rpm --version 1.1.1 —iteration 3 -C /path/to/fakeroot etc
  • 11. packer • Supports multiple output formats - AWS, VMWare, etc • Simple JSON configuration files • Can automatically copy the resulting AMIs to the regions you’re deployed in • Written in Go, no dependency hell to interfere with the other software installed on your build box
  • 12. jenkins • Keep the humans out of the loop • Generate AMIs every commit • Integration test every AMI build
  • 13. AMI Toolkit Summary • chef-solo / masterless puppet /ansible etc • fpm - https://github.com/jordansissel/fpm • packer - http://www.packer.io/ • S3 yum plugin - https://github.com/jbraeuer/yum- s3-plugin • jenkins - http://jenkins-ci.org/ Joe Block <jpb@numenta.com>

Hinweis der Redaktion

  1. Installing at run time means you rely on upstream being performant SPOF that is out of your control Immutable instance AMIs = scale any time
  2. The longer your configuration management takes to run, the longer the lag between starting to scale up and seeing actual performance improvement in your infrastructure. Bad enough we have to wait for the instances to start up.
  3. Using rpms lets you minimize what you have to specify in your chef recipes fpm lets you specify dependencies, versions, all the good stuff without dealing with spec files
  4. One line AMI builds
  5. More often the integration tests run, the less painful fixing the failures becomes Nobody ever complained that they found a problem too soon after it was introduced