SlideShare ist ein Scribd-Unternehmen logo
1 von 59
Downloaden Sie, um offline zu lesen
Scaling Rails Applications in the Cloud




                            100 new users
                             in < 2 hours


       The hashtag for this talk is:
              #SX09-573
A Typical Case Study
                                                                                                            32
The Inevitable Mass Adoption of Our Product




                                                                                                              Public
                                                                                                               Beta




                                                                                                    8
                                                                                                     Techcrunch50
                                               Me, Josh,   Friends & Family
                                                and a                                 4
                                                                                            Private Beta
                                              few other
                                                people
                                                                 The Inexorable March of Time
I’m Mike Subelsky
@subelsky / @otherinbox
What is Cloud
Computing?
Solving problems
across a flexible grid
      of nodes
...which you consume
      as a utility
The nodes are
   elastic
The nodes are
        loosely coupled
                 Tasking System




Computing   Computing    Computing   Computing
 Instance    Instance     Instance    Instance

               Permanent Storage
The nodes are
 distributed
The nodes are
 ephemeral
The nodes are
 disposable
What are the
 benefits?
Lower cost
Greater efficiency
Greater flexibility
You want flexibility
My Cloud Grows With Me!
                                                                                                                32
Heady, Viral, Asymptomtotic Growth of Userbase




                                                                                                                  Public
                                                                                                                   Beta




                                                                                                        8
                                                                                                         Techcrunch50
                                                  Me, Josh,     Friends & Family
                                                   and a                                   4
                                                                                                Private Beta
                                                 few other
                                                   people
                                                                          The Blink of an Eye
What are the costs?
Some loss of control
Potential for lock-in
Greater complexity
You chose a language
  that sublimates
     complexity
aws-S3
  RightAWS

                  Ruby/EventMachine


     Ruby Shines Here
                  Hpricot
memcache-client
                            ActiveSupport
Capistrano
     Chef
   Rubber
  PoolParty
     Scalr
Roll Your Own
What do you need
 to get started?
Tasking System




Computing   Computing    Computing   Computing
 Instance    Instance     Instance    Instance

               Permanent Storage




     Bonus: Non-RDBMS
Our Sky
        SMTP                 Sync




                                    Staging
                                     Staging
                                       Staging
                                         Staging
                                          Staging
                                            Staging



Processing       Master DB


                                    Staging
                                     Staging
                                       Staging
                                         Staging
                                          Staging
                                          Experiment




       Workers               Web
Web Cloud
                                        Mongrels &
                                        Memcached
                  JSON




                                        Mongrels &
    HTML            Apache    HAProxy                                    Master DB
                                        Memcached




                 Atom                   Mongrels &
                                        Memcached
Message Bodies
                                                     Mailbox Export Requests,
                                                       Domain Registration




                                                                  SQS

                         S3
Staging &
Experiments
Sync Cloud
Autoscaling
SMTP Cloud
                           o
                    et                          Ta
                                       SQS
                  m                                s
                na e                                  kin
              e
           Fil ars                                         g
         S3 P


Postfix   MailSender                                         MailReceiver   MailParser   Master DB




                                                           le
             Ra




                                                       Fi
                  w
                      Mb




                                                       x
                                                    bo
                        ox




                                                M
                               Fil


                                                w
                                  e

                                             Ra
                                       S3
Loose Coupling Is How I Sleep
         At Night
  US-EAST-1C
                               to                     Ta
                         e                 SQS
                      am                                 skin
                    en e
                 Fil ars                                      g
               S3 P


      Postfix   MailSender                                       MailReceiver   MailParser   Master DB




                                                                le
                   Ra




                                                             Fi
                      w
                          Mb




                                                          x
                                                       bo
                              ox




                                                      M
                                    Fil




                                                  w
                                       e




                                                 Ra
                                           S3




  US-EAST-1B
                               to                     Ta
                         me                SQS           s
                       na e                               kin
                    ile rs                                    g
                 3 F Pa
               S


      Postfix   MailSender                                       MailReceiver   MailParser   Master DB
                                                               le




                   Ra
                                                             Fi




                      w
                          Mb
                                                          x
                                                       bo




                              ox
                                                      M




                                    Fil
                                                   w




                                       e
                                                 Ra




                                           S3
Loose Coupling Is How I Sleep
         At Night
   US-EAST-1C
                                to                     Ta
                          me                SQS           s
                        na e                               kin
                     ile rs                                    g
                  3 F Pa
                S


       Postfix   MailSender                                       MailReceiver   MailParser   Master DB




                                                                 le
                    Ra




                                                              Fi
                       w
                           Mb




                                                           x
                                                        bo
                               ox




                                                       M
                                     Fil




                                                   w
                                        e




                                                  Ra
                                            S3




  X XX
   US-EAST-1B
                                o
                           t                           Ta
                        me                  SQS           s
                      a                                    kin
                    en e
                 Fil ars                                       g
                 3P
                S


       Postfix   MailSender                                       MailReceiver   MailParser   Master DB
                                                                le




                    Ra
                                                              Fi




                       w
                           Mb
                                                           x
                                                        bo




                               ox
                                                       M




                                     Fil
                                                    w




                                        e
                                                  Ra




                                            S3
Loose Coupling Is How I Sleep
         At Night


                                                                                            X
  US-EAST-1C
                               to                     Ta
                         me                SQS           s
                       na e                               kin
                    ile rs                                    g
                 3 F Pa
               S


      Postfix   MailSender                                       MailReceiver   MailParser   Master DB




                                                                le
                   Ra




                                                             Fi
                      w
                          Mb




                                                          x
                                                       bo
                              ox




                                                      M
                                    Fil




                                                  w
                                       e




                                                 Ra
                                           S3




                                                                                            X
  US-EAST-1B
                               o
                          t                           Ta
                       me                  SQS           s
                     a                                    kin
                   en e
                Fil ars                                       g
                3P
               S


      Postfix   MailSender                                       MailReceiver   MailParser   Master DB
                                                               le




                   Ra
                                                             Fi




                      w
                          Mb
                                                          x
                                                       bo




                              ox
                                                      M




                                    Fil
                                                   w




                                       e
                                                 Ra




                                           S3
Loose Coupling Is How I Sleep
         At Night


                                                       X
   US-EAST-1C
                                to                     Ta
                          me                SQS           s
                        na e                               kin
                     ile rs                                    g
                  3 F Pa
                S


       Postfix   MailSender                                       MailReceiver   MailParser   Master DB




                                                                 le
                    Ra




                                                              Fi
                       w
                           Mb




                                                           x
                                                        bo
                               ox




                                                       M
                                     Fil




                                                   w
                                        e




                                                  Ra
                                            S3




                                                       X
   US-EAST-1B
                                o
                           t                           Ta
                        me                  SQS           s
                      a                                    kin
                    en e
                 Fil ars                                       g
                 3P
                S


       Postfix   MailSender                                       MailReceiver   MailParser   Master DB
                                                                le




                    Ra
                                                              Fi




                       w
                           Mb
                                                           x
                                                        bo




                               ox
                                                       M




                                     Fil
                                                    w




                                        e
                                                  Ra




                                            S3
Loose Coupling Is How I Sleep
         At Night


                                      X
   US-EAST-1C
                                to                     Ta
                          me                SQS           s
                        na e                               kin
                     ile rs                                    g
                  3 F Pa
                S


       Postfix   MailSender                                       MailReceiver   MailParser   Master DB




                                                                 le
                    Ra




                                                              Fi
                       w
                           Mb




                                                           x
                                                        bo
                               ox




                                                       M
                                     Fil




                                                   w
                                        e




                                                  Ra
                                            S3




                                      X
   US-EAST-1B
                                o
                           t                           Ta
                        me                  SQS           s
                      a                                    kin
                    en e
                 Fil ars                                       g
                 3P
                S


       Postfix   MailSender                                       MailReceiver   MailParser   Master DB
                                                                le




                    Ra
                                                              Fi




                       w
                           Mb
                                                           x
                                                        bo




                               ox
                                                       M




                                     Fil
                                                    w




                                        e
                                                  Ra




                                            S3
Loose Coupling Is How I Sleep
         At Night
What It Looks Like
Processing Cloud
                                                                                     MessageParser
                                               h
                                           s
                                        pu
                                  cks                                       Tasks
                                                    MessageParsing Queue
                                a
                            allb e                  MessageClassfier Queue           MessageClassifier
                          c
                        rd nam
                      o                            AnyFeatureWeWant Queue
                    ec file
                  R
               ive S3
            ct
           A
                                                                                    AnyFeatureWorker


Message Created
Workers
•   MessageExporter

•   DailyUpdateSender

•   RegistrationWorker

•   BackupWorker
The Database Is a
 Single Point of
     Failure
How We Cope
                            Hourly
              EBS Volume
                           Snapshots


                                       Weekly
Master DB
                                       Dumps




                                                Expensive Queries
                                                   Memcached
A Dark But
Illustrative Tale
with QuickTime
Manual Stuff

    mkdir /vol
 mount /vol /dev/sdj

sudo god start mysqld
Manual Stuff

change database.yml by hand (yuck)

     queue mail to redeliver
(Wasted)
Troubleshooting:
     15 mins
Provisioning new
   database:
    15 mins
If there was a next
       time...
(standard programmer
   optimism applies)
     < 10 minutes
Replication?
 Sharding?
Key-Value?
Columns?
Logging gem        Splunk




              Logging


                   Good ol’ fashion tail
Zenoss
          Hoptoad

                 Slow query log
   Dash


            Monitoring
          RightScale

Alertra                           New Relic
Questions?
 mike@oib.com

   @subelsky
  @otherinbox
Join our team
Upgrade your job            Open Positions
•   Smart, fun people
                            • Ruby on Rails Developer
•   Startup environment
                            • Ruby on Rails Intern
•   Cutting edge tech
                            • Product Manager
•   Competitive pay
•   Stock options




We pay $1,000 for successful referrals!
rate.sxsw.com
Thank
 you!

Weitere ähnliche Inhalte

Ähnlich wie Scaling Rails Applications In The Cloud

A Methodology for the Emulation of Boolean Logic that Paved the Way for the S...
A Methodology for the Emulation of Boolean Logic that Paved the Way for the S...A Methodology for the Emulation of Boolean Logic that Paved the Way for the S...
A Methodology for the Emulation of Boolean Logic that Paved the Way for the S...ricky_pi_tercios
 
Http Jaoo.Com.Au Sydney 2008 File Path= Jaoo Aus2008 Slides Dave Thomas Lif...
Http   Jaoo.Com.Au Sydney 2008 File Path= Jaoo Aus2008 Slides Dave Thomas Lif...Http   Jaoo.Com.Au Sydney 2008 File Path= Jaoo Aus2008 Slides Dave Thomas Lif...
Http Jaoo.Com.Au Sydney 2008 File Path= Jaoo Aus2008 Slides Dave Thomas Lif...qedanne
 
SAM SIG: Hadoop architecture, MapReduce patterns, and best practices with Cas...
SAM SIG: Hadoop architecture, MapReduce patterns, and best practices with Cas...SAM SIG: Hadoop architecture, MapReduce patterns, and best practices with Cas...
SAM SIG: Hadoop architecture, MapReduce patterns, and best practices with Cas...cwensel
 
雲端與Big data
雲端與Big data雲端與Big data
雲端與Big dataYa-hui Lin
 
Bangalore cloudstack user group
Bangalore cloudstack user groupBangalore cloudstack user group
Bangalore cloudstack user groupShapeBlue
 
Learn OpenStack from trystack.cn ——Folsom in practice
Learn OpenStack from trystack.cn  ——Folsom in practiceLearn OpenStack from trystack.cn  ——Folsom in practice
Learn OpenStack from trystack.cn ——Folsom in practiceOpenCity Community
 
Microsoft HPC User Group
Microsoft HPC User Group Microsoft HPC User Group
Microsoft HPC User Group sjwoodman
 
Embracing concurrency for fun utility and simpler code
Embracing concurrency for fun utility and simpler codeEmbracing concurrency for fun utility and simpler code
Embracing concurrency for fun utility and simpler codekamaelian
 
Challenges in Maintaining a High Performance Search Engine Written in Java
Challenges in Maintaining a High Performance Search Engine Written in JavaChallenges in Maintaining a High Performance Search Engine Written in Java
Challenges in Maintaining a High Performance Search Engine Written in Javalucenerevolution
 
Future of WCM - CM Forum Belgium
Future of WCM - CM Forum BelgiumFuture of WCM - CM Forum Belgium
Future of WCM - CM Forum BelgiumDavid Nuescheler
 
Cms forum, future of Web Content Management
Cms forum, future of Web Content ManagementCms forum, future of Web Content Management
Cms forum, future of Web Content Managementguest88136a
 
Creating Cloud Communities
Creating Cloud CommunitiesCreating Cloud Communities
Creating Cloud CommunitiesPeter Coffee
 
Joyent Cloud Advantages
Joyent Cloud AdvantagesJoyent Cloud Advantages
Joyent Cloud Advantagesdmasciorini
 
OpenStack and OpenFlow Demos
OpenStack and OpenFlow DemosOpenStack and OpenFlow Demos
OpenStack and OpenFlow DemosBrent Salisbury
 
Automating Oracle Database deployment with Amazon Web Services, fabric, and boto
Automating Oracle Database deployment with Amazon Web Services, fabric, and botoAutomating Oracle Database deployment with Amazon Web Services, fabric, and boto
Automating Oracle Database deployment with Amazon Web Services, fabric, and botomjbommar
 
Behind the Scenes at LiveJournal: Scaling Storytime
Behind the Scenes at LiveJournal: Scaling StorytimeBehind the Scenes at LiveJournal: Scaling Storytime
Behind the Scenes at LiveJournal: Scaling StorytimeSergeyChernyshev
 
vBrownBag OpenStack Networking Talk
vBrownBag OpenStack Networking TalkvBrownBag OpenStack Networking Talk
vBrownBag OpenStack Networking Talkmestery
 

Ähnlich wie Scaling Rails Applications In The Cloud (20)

A Methodology for the Emulation of Boolean Logic that Paved the Way for the S...
A Methodology for the Emulation of Boolean Logic that Paved the Way for the S...A Methodology for the Emulation of Boolean Logic that Paved the Way for the S...
A Methodology for the Emulation of Boolean Logic that Paved the Way for the S...
 
Http Jaoo.Com.Au Sydney 2008 File Path= Jaoo Aus2008 Slides Dave Thomas Lif...
Http   Jaoo.Com.Au Sydney 2008 File Path= Jaoo Aus2008 Slides Dave Thomas Lif...Http   Jaoo.Com.Au Sydney 2008 File Path= Jaoo Aus2008 Slides Dave Thomas Lif...
Http Jaoo.Com.Au Sydney 2008 File Path= Jaoo Aus2008 Slides Dave Thomas Lif...
 
近未来的並列 LL
近未来的並列 LL近未来的並列 LL
近未来的並列 LL
 
SAM SIG: Hadoop architecture, MapReduce patterns, and best practices with Cas...
SAM SIG: Hadoop architecture, MapReduce patterns, and best practices with Cas...SAM SIG: Hadoop architecture, MapReduce patterns, and best practices with Cas...
SAM SIG: Hadoop architecture, MapReduce patterns, and best practices with Cas...
 
雲端與Big data
雲端與Big data雲端與Big data
雲端與Big data
 
Tom Krcha - Future of Flash
Tom Krcha - Future of FlashTom Krcha - Future of Flash
Tom Krcha - Future of Flash
 
Bangalore cloudstack user group
Bangalore cloudstack user groupBangalore cloudstack user group
Bangalore cloudstack user group
 
Learn OpenStack from trystack.cn ——Folsom in practice
Learn OpenStack from trystack.cn  ——Folsom in practiceLearn OpenStack from trystack.cn  ——Folsom in practice
Learn OpenStack from trystack.cn ——Folsom in practice
 
Microsoft HPC User Group
Microsoft HPC User Group Microsoft HPC User Group
Microsoft HPC User Group
 
Delay Tolerant Streaming Services, Thomas Plagemann, UiO
Delay Tolerant Streaming Services, Thomas Plagemann, UiODelay Tolerant Streaming Services, Thomas Plagemann, UiO
Delay Tolerant Streaming Services, Thomas Plagemann, UiO
 
Embracing concurrency for fun utility and simpler code
Embracing concurrency for fun utility and simpler codeEmbracing concurrency for fun utility and simpler code
Embracing concurrency for fun utility and simpler code
 
Challenges in Maintaining a High Performance Search Engine Written in Java
Challenges in Maintaining a High Performance Search Engine Written in JavaChallenges in Maintaining a High Performance Search Engine Written in Java
Challenges in Maintaining a High Performance Search Engine Written in Java
 
Future of WCM - CM Forum Belgium
Future of WCM - CM Forum BelgiumFuture of WCM - CM Forum Belgium
Future of WCM - CM Forum Belgium
 
Cms forum, future of Web Content Management
Cms forum, future of Web Content ManagementCms forum, future of Web Content Management
Cms forum, future of Web Content Management
 
Creating Cloud Communities
Creating Cloud CommunitiesCreating Cloud Communities
Creating Cloud Communities
 
Joyent Cloud Advantages
Joyent Cloud AdvantagesJoyent Cloud Advantages
Joyent Cloud Advantages
 
OpenStack and OpenFlow Demos
OpenStack and OpenFlow DemosOpenStack and OpenFlow Demos
OpenStack and OpenFlow Demos
 
Automating Oracle Database deployment with Amazon Web Services, fabric, and boto
Automating Oracle Database deployment with Amazon Web Services, fabric, and botoAutomating Oracle Database deployment with Amazon Web Services, fabric, and boto
Automating Oracle Database deployment with Amazon Web Services, fabric, and boto
 
Behind the Scenes at LiveJournal: Scaling Storytime
Behind the Scenes at LiveJournal: Scaling StorytimeBehind the Scenes at LiveJournal: Scaling Storytime
Behind the Scenes at LiveJournal: Scaling Storytime
 
vBrownBag OpenStack Networking Talk
vBrownBag OpenStack Networking TalkvBrownBag OpenStack Networking Talk
vBrownBag OpenStack Networking Talk
 

Mehr von Mike Subelsky

STAQ Development Manual (Redacted)
STAQ Development Manual (Redacted)STAQ Development Manual (Redacted)
STAQ Development Manual (Redacted)Mike Subelsky
 
Coding for uncertainty
Coding for uncertaintyCoding for uncertainty
Coding for uncertaintyMike Subelsky
 
Ruby Concurrency Realities
Ruby Concurrency RealitiesRuby Concurrency Realities
Ruby Concurrency RealitiesMike Subelsky
 
Generating Good Ideas
Generating Good IdeasGenerating Good Ideas
Generating Good IdeasMike Subelsky
 
Baltimore: A Great Place to Pick Up Ideas and Run WIth Them (Ignite)
Baltimore: A Great Place to Pick Up Ideas and Run WIth Them (Ignite)Baltimore: A Great Place to Pick Up Ideas and Run WIth Them (Ignite)
Baltimore: A Great Place to Pick Up Ideas and Run WIth Them (Ignite)Mike Subelsky
 
Let's Make Baltimore More Innovative (TEDxBaltimore)
Let's Make Baltimore More Innovative (TEDxBaltimore)Let's Make Baltimore More Innovative (TEDxBaltimore)
Let's Make Baltimore More Innovative (TEDxBaltimore)Mike Subelsky
 
Social Media for Everybody
Social Media for EverybodySocial Media for Everybody
Social Media for EverybodyMike Subelsky
 
It's Not Always Sunny in the Clouds
It's Not Always Sunny in the CloudsIt's Not Always Sunny in the Clouds
It's Not Always Sunny in the CloudsMike Subelsky
 
Introduction to SproutCore at JSConf
Introduction to SproutCore at JSConfIntroduction to SproutCore at JSConf
Introduction to SproutCore at JSConfMike Subelsky
 
SproutCore and the Future of Web Apps
SproutCore and the Future of Web AppsSproutCore and the Future of Web Apps
SproutCore and the Future of Web AppsMike Subelsky
 
SproutCore and the Future of Web Apps
SproutCore and the Future of Web AppsSproutCore and the Future of Web Apps
SproutCore and the Future of Web AppsMike Subelsky
 

Mehr von Mike Subelsky (12)

STAQ Development Manual (Redacted)
STAQ Development Manual (Redacted)STAQ Development Manual (Redacted)
STAQ Development Manual (Redacted)
 
Coding for uncertainty
Coding for uncertaintyCoding for uncertainty
Coding for uncertainty
 
Ruby Concurrency Realities
Ruby Concurrency RealitiesRuby Concurrency Realities
Ruby Concurrency Realities
 
Generating Good Ideas
Generating Good IdeasGenerating Good Ideas
Generating Good Ideas
 
Baltimore: A Great Place to Pick Up Ideas and Run WIth Them (Ignite)
Baltimore: A Great Place to Pick Up Ideas and Run WIth Them (Ignite)Baltimore: A Great Place to Pick Up Ideas and Run WIth Them (Ignite)
Baltimore: A Great Place to Pick Up Ideas and Run WIth Them (Ignite)
 
Let's Make Baltimore More Innovative (TEDxBaltimore)
Let's Make Baltimore More Innovative (TEDxBaltimore)Let's Make Baltimore More Innovative (TEDxBaltimore)
Let's Make Baltimore More Innovative (TEDxBaltimore)
 
Social Media for Everybody
Social Media for EverybodySocial Media for Everybody
Social Media for Everybody
 
Ruby For Startups
Ruby For StartupsRuby For Startups
Ruby For Startups
 
It's Not Always Sunny in the Clouds
It's Not Always Sunny in the CloudsIt's Not Always Sunny in the Clouds
It's Not Always Sunny in the Clouds
 
Introduction to SproutCore at JSConf
Introduction to SproutCore at JSConfIntroduction to SproutCore at JSConf
Introduction to SproutCore at JSConf
 
SproutCore and the Future of Web Apps
SproutCore and the Future of Web AppsSproutCore and the Future of Web Apps
SproutCore and the Future of Web Apps
 
SproutCore and the Future of Web Apps
SproutCore and the Future of Web AppsSproutCore and the Future of Web Apps
SproutCore and the Future of Web Apps
 

Kürzlich hochgeladen

MINDCTI Revenue Release Quarter One 2024
MINDCTI Revenue Release Quarter One 2024MINDCTI Revenue Release Quarter One 2024
MINDCTI Revenue Release Quarter One 2024MIND CTI
 
TrustArc Webinar - Unlock the Power of AI-Driven Data Discovery
TrustArc Webinar - Unlock the Power of AI-Driven Data DiscoveryTrustArc Webinar - Unlock the Power of AI-Driven Data Discovery
TrustArc Webinar - Unlock the Power of AI-Driven Data DiscoveryTrustArc
 
Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...
Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...
Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...apidays
 
Top 10 Most Downloaded Games on Play Store in 2024
Top 10 Most Downloaded Games on Play Store in 2024Top 10 Most Downloaded Games on Play Store in 2024
Top 10 Most Downloaded Games on Play Store in 2024SynarionITSolutions
 
Bajaj Allianz Life Insurance Company - Insurer Innovation Award 2024
Bajaj Allianz Life Insurance Company - Insurer Innovation Award 2024Bajaj Allianz Life Insurance Company - Insurer Innovation Award 2024
Bajaj Allianz Life Insurance Company - Insurer Innovation Award 2024The Digital Insurer
 
2024: Domino Containers - The Next Step. News from the Domino Container commu...
2024: Domino Containers - The Next Step. News from the Domino Container commu...2024: Domino Containers - The Next Step. News from the Domino Container commu...
2024: Domino Containers - The Next Step. News from the Domino Container commu...Martijn de Jong
 
ProductAnonymous-April2024-WinProductDiscovery-MelissaKlemke
ProductAnonymous-April2024-WinProductDiscovery-MelissaKlemkeProductAnonymous-April2024-WinProductDiscovery-MelissaKlemke
ProductAnonymous-April2024-WinProductDiscovery-MelissaKlemkeProduct Anonymous
 
AWS Community Day CPH - Three problems of Terraform
AWS Community Day CPH - Three problems of TerraformAWS Community Day CPH - Three problems of Terraform
AWS Community Day CPH - Three problems of TerraformAndrey Devyatkin
 
Strategies for Landing an Oracle DBA Job as a Fresher
Strategies for Landing an Oracle DBA Job as a FresherStrategies for Landing an Oracle DBA Job as a Fresher
Strategies for Landing an Oracle DBA Job as a FresherRemote DBA Services
 
Scaling API-first – The story of a global engineering organization
Scaling API-first – The story of a global engineering organizationScaling API-first – The story of a global engineering organization
Scaling API-first – The story of a global engineering organizationRadu Cotescu
 
Deploy with confidence: VMware Cloud Foundation 5.1 on next gen Dell PowerEdg...
Deploy with confidence: VMware Cloud Foundation 5.1 on next gen Dell PowerEdg...Deploy with confidence: VMware Cloud Foundation 5.1 on next gen Dell PowerEdg...
Deploy with confidence: VMware Cloud Foundation 5.1 on next gen Dell PowerEdg...Principled Technologies
 
Polkadot JAM Slides - Token2049 - By Dr. Gavin Wood
Polkadot JAM Slides - Token2049 - By Dr. Gavin WoodPolkadot JAM Slides - Token2049 - By Dr. Gavin Wood
Polkadot JAM Slides - Token2049 - By Dr. Gavin WoodJuan lago vázquez
 
Apidays New York 2024 - The value of a flexible API Management solution for O...
Apidays New York 2024 - The value of a flexible API Management solution for O...Apidays New York 2024 - The value of a flexible API Management solution for O...
Apidays New York 2024 - The value of a flexible API Management solution for O...apidays
 
The 7 Things I Know About Cyber Security After 25 Years | April 2024
The 7 Things I Know About Cyber Security After 25 Years | April 2024The 7 Things I Know About Cyber Security After 25 Years | April 2024
The 7 Things I Know About Cyber Security After 25 Years | April 2024Rafal Los
 
Apidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, Adobe
Apidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, AdobeApidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, Adobe
Apidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, Adobeapidays
 
From Event to Action: Accelerate Your Decision Making with Real-Time Automation
From Event to Action: Accelerate Your Decision Making with Real-Time AutomationFrom Event to Action: Accelerate Your Decision Making with Real-Time Automation
From Event to Action: Accelerate Your Decision Making with Real-Time AutomationSafe Software
 
Workshop - Best of Both Worlds_ Combine KG and Vector search for enhanced R...
Workshop - Best of Both Worlds_ Combine  KG and Vector search for  enhanced R...Workshop - Best of Both Worlds_ Combine  KG and Vector search for  enhanced R...
Workshop - Best of Both Worlds_ Combine KG and Vector search for enhanced R...Neo4j
 
presentation ICT roal in 21st century education
presentation ICT roal in 21st century educationpresentation ICT roal in 21st century education
presentation ICT roal in 21st century educationjfdjdjcjdnsjd
 
Boost Fertility New Invention Ups Success Rates.pdf
Boost Fertility New Invention Ups Success Rates.pdfBoost Fertility New Invention Ups Success Rates.pdf
Boost Fertility New Invention Ups Success Rates.pdfsudhanshuwaghmare1
 

Kürzlich hochgeladen (20)

MINDCTI Revenue Release Quarter One 2024
MINDCTI Revenue Release Quarter One 2024MINDCTI Revenue Release Quarter One 2024
MINDCTI Revenue Release Quarter One 2024
 
TrustArc Webinar - Unlock the Power of AI-Driven Data Discovery
TrustArc Webinar - Unlock the Power of AI-Driven Data DiscoveryTrustArc Webinar - Unlock the Power of AI-Driven Data Discovery
TrustArc Webinar - Unlock the Power of AI-Driven Data Discovery
 
+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...
+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...
+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...
 
Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...
Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...
Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...
 
Top 10 Most Downloaded Games on Play Store in 2024
Top 10 Most Downloaded Games on Play Store in 2024Top 10 Most Downloaded Games on Play Store in 2024
Top 10 Most Downloaded Games on Play Store in 2024
 
Bajaj Allianz Life Insurance Company - Insurer Innovation Award 2024
Bajaj Allianz Life Insurance Company - Insurer Innovation Award 2024Bajaj Allianz Life Insurance Company - Insurer Innovation Award 2024
Bajaj Allianz Life Insurance Company - Insurer Innovation Award 2024
 
2024: Domino Containers - The Next Step. News from the Domino Container commu...
2024: Domino Containers - The Next Step. News from the Domino Container commu...2024: Domino Containers - The Next Step. News from the Domino Container commu...
2024: Domino Containers - The Next Step. News from the Domino Container commu...
 
ProductAnonymous-April2024-WinProductDiscovery-MelissaKlemke
ProductAnonymous-April2024-WinProductDiscovery-MelissaKlemkeProductAnonymous-April2024-WinProductDiscovery-MelissaKlemke
ProductAnonymous-April2024-WinProductDiscovery-MelissaKlemke
 
AWS Community Day CPH - Three problems of Terraform
AWS Community Day CPH - Three problems of TerraformAWS Community Day CPH - Three problems of Terraform
AWS Community Day CPH - Three problems of Terraform
 
Strategies for Landing an Oracle DBA Job as a Fresher
Strategies for Landing an Oracle DBA Job as a FresherStrategies for Landing an Oracle DBA Job as a Fresher
Strategies for Landing an Oracle DBA Job as a Fresher
 
Scaling API-first – The story of a global engineering organization
Scaling API-first – The story of a global engineering organizationScaling API-first – The story of a global engineering organization
Scaling API-first – The story of a global engineering organization
 
Deploy with confidence: VMware Cloud Foundation 5.1 on next gen Dell PowerEdg...
Deploy with confidence: VMware Cloud Foundation 5.1 on next gen Dell PowerEdg...Deploy with confidence: VMware Cloud Foundation 5.1 on next gen Dell PowerEdg...
Deploy with confidence: VMware Cloud Foundation 5.1 on next gen Dell PowerEdg...
 
Polkadot JAM Slides - Token2049 - By Dr. Gavin Wood
Polkadot JAM Slides - Token2049 - By Dr. Gavin WoodPolkadot JAM Slides - Token2049 - By Dr. Gavin Wood
Polkadot JAM Slides - Token2049 - By Dr. Gavin Wood
 
Apidays New York 2024 - The value of a flexible API Management solution for O...
Apidays New York 2024 - The value of a flexible API Management solution for O...Apidays New York 2024 - The value of a flexible API Management solution for O...
Apidays New York 2024 - The value of a flexible API Management solution for O...
 
The 7 Things I Know About Cyber Security After 25 Years | April 2024
The 7 Things I Know About Cyber Security After 25 Years | April 2024The 7 Things I Know About Cyber Security After 25 Years | April 2024
The 7 Things I Know About Cyber Security After 25 Years | April 2024
 
Apidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, Adobe
Apidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, AdobeApidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, Adobe
Apidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, Adobe
 
From Event to Action: Accelerate Your Decision Making with Real-Time Automation
From Event to Action: Accelerate Your Decision Making with Real-Time AutomationFrom Event to Action: Accelerate Your Decision Making with Real-Time Automation
From Event to Action: Accelerate Your Decision Making with Real-Time Automation
 
Workshop - Best of Both Worlds_ Combine KG and Vector search for enhanced R...
Workshop - Best of Both Worlds_ Combine  KG and Vector search for  enhanced R...Workshop - Best of Both Worlds_ Combine  KG and Vector search for  enhanced R...
Workshop - Best of Both Worlds_ Combine KG and Vector search for enhanced R...
 
presentation ICT roal in 21st century education
presentation ICT roal in 21st century educationpresentation ICT roal in 21st century education
presentation ICT roal in 21st century education
 
Boost Fertility New Invention Ups Success Rates.pdf
Boost Fertility New Invention Ups Success Rates.pdfBoost Fertility New Invention Ups Success Rates.pdf
Boost Fertility New Invention Ups Success Rates.pdf
 

Scaling Rails Applications In The Cloud

  • 1. Scaling Rails Applications in the Cloud 100 new users in < 2 hours The hashtag for this talk is: #SX09-573
  • 2. A Typical Case Study 32 The Inevitable Mass Adoption of Our Product Public Beta 8 Techcrunch50 Me, Josh, Friends & Family and a 4 Private Beta few other people The Inexorable March of Time
  • 5. Solving problems across a flexible grid of nodes
  • 6. ...which you consume as a utility
  • 7. The nodes are elastic
  • 8. The nodes are loosely coupled Tasking System Computing Computing Computing Computing Instance Instance Instance Instance Permanent Storage
  • 9. The nodes are distributed
  • 10. The nodes are ephemeral
  • 11. The nodes are disposable
  • 12. What are the benefits?
  • 15. My Cloud Grows With Me! 32 Heady, Viral, Asymptomtotic Growth of Userbase Public Beta 8 Techcrunch50 Me, Josh, Friends & Family and a 4 Private Beta few other people The Blink of an Eye
  • 16. What are the costs?
  • 17. Some loss of control Potential for lock-in Greater complexity
  • 18. You chose a language that sublimates complexity
  • 19. aws-S3 RightAWS Ruby/EventMachine Ruby Shines Here Hpricot memcache-client ActiveSupport
  • 20.
  • 21. Capistrano Chef Rubber PoolParty Scalr Roll Your Own
  • 22. What do you need to get started?
  • 23. Tasking System Computing Computing Computing Computing Instance Instance Instance Instance Permanent Storage Bonus: Non-RDBMS
  • 24. Our Sky SMTP Sync Staging Staging Staging Staging Staging Staging Processing Master DB Staging Staging Staging Staging Staging Experiment Workers Web
  • 25. Web Cloud Mongrels & Memcached JSON Mongrels & HTML Apache HAProxy Master DB Memcached Atom Mongrels & Memcached Message Bodies Mailbox Export Requests, Domain Registration SQS S3
  • 29. SMTP Cloud o et Ta SQS m s na e kin e Fil ars g S3 P Postfix MailSender MailReceiver MailParser Master DB le Ra Fi w Mb x bo ox M Fil w e Ra S3
  • 30. Loose Coupling Is How I Sleep At Night US-EAST-1C to Ta e SQS am skin en e Fil ars g S3 P Postfix MailSender MailReceiver MailParser Master DB le Ra Fi w Mb x bo ox M Fil w e Ra S3 US-EAST-1B to Ta me SQS s na e kin ile rs g 3 F Pa S Postfix MailSender MailReceiver MailParser Master DB le Ra Fi w Mb x bo ox M Fil w e Ra S3
  • 31. Loose Coupling Is How I Sleep At Night US-EAST-1C to Ta me SQS s na e kin ile rs g 3 F Pa S Postfix MailSender MailReceiver MailParser Master DB le Ra Fi w Mb x bo ox M Fil w e Ra S3 X XX US-EAST-1B o t Ta me SQS s a kin en e Fil ars g 3P S Postfix MailSender MailReceiver MailParser Master DB le Ra Fi w Mb x bo ox M Fil w e Ra S3
  • 32. Loose Coupling Is How I Sleep At Night X US-EAST-1C to Ta me SQS s na e kin ile rs g 3 F Pa S Postfix MailSender MailReceiver MailParser Master DB le Ra Fi w Mb x bo ox M Fil w e Ra S3 X US-EAST-1B o t Ta me SQS s a kin en e Fil ars g 3P S Postfix MailSender MailReceiver MailParser Master DB le Ra Fi w Mb x bo ox M Fil w e Ra S3
  • 33. Loose Coupling Is How I Sleep At Night X US-EAST-1C to Ta me SQS s na e kin ile rs g 3 F Pa S Postfix MailSender MailReceiver MailParser Master DB le Ra Fi w Mb x bo ox M Fil w e Ra S3 X US-EAST-1B o t Ta me SQS s a kin en e Fil ars g 3P S Postfix MailSender MailReceiver MailParser Master DB le Ra Fi w Mb x bo ox M Fil w e Ra S3
  • 34. Loose Coupling Is How I Sleep At Night X US-EAST-1C to Ta me SQS s na e kin ile rs g 3 F Pa S Postfix MailSender MailReceiver MailParser Master DB le Ra Fi w Mb x bo ox M Fil w e Ra S3 X US-EAST-1B o t Ta me SQS s a kin en e Fil ars g 3P S Postfix MailSender MailReceiver MailParser Master DB le Ra Fi w Mb x bo ox M Fil w e Ra S3
  • 35. Loose Coupling Is How I Sleep At Night
  • 37. Processing Cloud MessageParser h s pu cks Tasks MessageParsing Queue a allb e MessageClassfier Queue MessageClassifier c rd nam o AnyFeatureWeWant Queue ec file R ive S3 ct A AnyFeatureWorker Message Created
  • 38. Workers • MessageExporter • DailyUpdateSender • RegistrationWorker • BackupWorker
  • 39. The Database Is a Single Point of Failure
  • 40. How We Cope Hourly EBS Volume Snapshots Weekly Master DB Dumps Expensive Queries Memcached
  • 41. A Dark But Illustrative Tale with QuickTime
  • 42.
  • 43.
  • 44.
  • 45.
  • 46. Manual Stuff mkdir /vol mount /vol /dev/sdj sudo god start mysqld
  • 47. Manual Stuff change database.yml by hand (yuck) queue mail to redeliver
  • 49. Provisioning new database: 15 mins
  • 50. If there was a next time...
  • 51. (standard programmer optimism applies) < 10 minutes
  • 54. Logging gem Splunk Logging Good ol’ fashion tail
  • 55. Zenoss Hoptoad Slow query log Dash Monitoring RightScale Alertra New Relic
  • 56. Questions? mike@oib.com @subelsky @otherinbox
  • 57. Join our team Upgrade your job Open Positions • Smart, fun people • Ruby on Rails Developer • Startup environment • Ruby on Rails Intern • Cutting edge tech • Product Manager • Competitive pay • Stock options We pay $1,000 for successful referrals!