SlideShare ist ein Scribd-Unternehmen logo
1 von 59
Downloaden Sie, um offline zu lesen
Scaling Rails Applications in the Cloud




                            100 new users
                             in < 2 hours


       The hashtag for this talk is:
              #SX09-573
A Typical Case Study
                                                                                                            32
The Inevitable Mass Adoption of Our Product




                                                                                                              Public
                                                                                                               Beta




                                                                                                    8
                                                                                                     Techcrunch50
                                               Me, Josh,   Friends & Family
                                                and a                                 4
                                                                                            Private Beta
                                              few other
                                                people
                                                                 The Inexorable March of Time
I’m Mike Subelsky
@subelsky / @otherinbox
What is Cloud
Computing?
Solving problems
across a flexible grid
      of nodes
...which you consume
      as a utility
The nodes are
   elastic
The nodes are
        loosely coupled
                 Tasking System




Computing   Computing    Computing   Computing
 Instance    Instance     Instance    Instance

               Permanent Storage
The nodes are
 distributed
The nodes are
 ephemeral
The nodes are
 disposable
What are the
 benefits?
Lower cost
Greater efficiency
Greater flexibility
You want flexibility
My Cloud Grows With Me!
                                                                                                                32
Heady, Viral, Asymptomtotic Growth of Userbase




                                                                                                                  Public
                                                                                                                   Beta




                                                                                                        8
                                                                                                         Techcrunch50
                                                  Me, Josh,     Friends & Family
                                                   and a                                   4
                                                                                                Private Beta
                                                 few other
                                                   people
                                                                          The Blink of an Eye
What are the costs?
Some loss of control
Potential for lock-in
Greater complexity
You chose a language
  that sublimates
     complexity
aws-S3
  RightAWS

                  Ruby/EventMachine


     Ruby Shines Here
                  Hpricot
memcache-client
                            ActiveSupport
Capistrano
     Chef
   Rubber
  PoolParty
     Scalr
Roll Your Own
What do you need
 to get started?
Tasking System




Computing   Computing    Computing   Computing
 Instance    Instance     Instance    Instance

               Permanent Storage




     Bonus: Non-RDBMS
Our Sky
        SMTP                 Sync




                                    Staging
                                     Staging
                                       Staging
                                         Staging
                                          Staging
                                            Staging



Processing       Master DB


                                    Staging
                                     Staging
                                       Staging
                                         Staging
                                          Staging
                                          Experiment




       Workers               Web
Web Cloud
                                        Mongrels &
                                        Memcached
                  JSON




                                        Mongrels &
    HTML            Apache    HAProxy                                    Master DB
                                        Memcached




                 Atom                   Mongrels &
                                        Memcached
Message Bodies
                                                     Mailbox Export Requests,
                                                       Domain Registration




                                                                  SQS

                         S3
Staging &
Experiments
Sync Cloud
Autoscaling
SMTP Cloud
                           o
                    et                          Ta
                                       SQS
                  m                                s
                na e                                  kin
              e
           Fil ars                                         g
         S3 P


Postfix   MailSender                                         MailReceiver   MailParser   Master DB




                                                           le
             Ra




                                                       Fi
                  w
                      Mb




                                                       x
                                                    bo
                        ox




                                                M
                               Fil


                                                w
                                  e

                                             Ra
                                       S3
Loose Coupling Is How I Sleep
         At Night
  US-EAST-1C
                               to                     Ta
                         e                 SQS
                      am                                 skin
                    en e
                 Fil ars                                      g
               S3 P


      Postfix   MailSender                                       MailReceiver   MailParser   Master DB




                                                                le
                   Ra




                                                             Fi
                      w
                          Mb




                                                          x
                                                       bo
                              ox




                                                      M
                                    Fil




                                                  w
                                       e




                                                 Ra
                                           S3




  US-EAST-1B
                               to                     Ta
                         me                SQS           s
                       na e                               kin
                    ile rs                                    g
                 3 F Pa
               S


      Postfix   MailSender                                       MailReceiver   MailParser   Master DB
                                                               le




                   Ra
                                                             Fi




                      w
                          Mb
                                                          x
                                                       bo




                              ox
                                                      M




                                    Fil
                                                   w




                                       e
                                                 Ra




                                           S3
Loose Coupling Is How I Sleep
         At Night
   US-EAST-1C
                                to                     Ta
                          me                SQS           s
                        na e                               kin
                     ile rs                                    g
                  3 F Pa
                S


       Postfix   MailSender                                       MailReceiver   MailParser   Master DB




                                                                 le
                    Ra




                                                              Fi
                       w
                           Mb




                                                           x
                                                        bo
                               ox




                                                       M
                                     Fil




                                                   w
                                        e




                                                  Ra
                                            S3




  X XX
   US-EAST-1B
                                o
                           t                           Ta
                        me                  SQS           s
                      a                                    kin
                    en e
                 Fil ars                                       g
                 3P
                S


       Postfix   MailSender                                       MailReceiver   MailParser   Master DB
                                                                le




                    Ra
                                                              Fi




                       w
                           Mb
                                                           x
                                                        bo




                               ox
                                                       M




                                     Fil
                                                    w




                                        e
                                                  Ra




                                            S3
Loose Coupling Is How I Sleep
         At Night


                                                                                            X
  US-EAST-1C
                               to                     Ta
                         me                SQS           s
                       na e                               kin
                    ile rs                                    g
                 3 F Pa
               S


      Postfix   MailSender                                       MailReceiver   MailParser   Master DB




                                                                le
                   Ra




                                                             Fi
                      w
                          Mb




                                                          x
                                                       bo
                              ox




                                                      M
                                    Fil




                                                  w
                                       e




                                                 Ra
                                           S3




                                                                                            X
  US-EAST-1B
                               o
                          t                           Ta
                       me                  SQS           s
                     a                                    kin
                   en e
                Fil ars                                       g
                3P
               S


      Postfix   MailSender                                       MailReceiver   MailParser   Master DB
                                                               le




                   Ra
                                                             Fi




                      w
                          Mb
                                                          x
                                                       bo




                              ox
                                                      M




                                    Fil
                                                   w




                                       e
                                                 Ra




                                           S3
Loose Coupling Is How I Sleep
         At Night


                                                       X
   US-EAST-1C
                                to                     Ta
                          me                SQS           s
                        na e                               kin
                     ile rs                                    g
                  3 F Pa
                S


       Postfix   MailSender                                       MailReceiver   MailParser   Master DB




                                                                 le
                    Ra




                                                              Fi
                       w
                           Mb




                                                           x
                                                        bo
                               ox




                                                       M
                                     Fil




                                                   w
                                        e




                                                  Ra
                                            S3




                                                       X
   US-EAST-1B
                                o
                           t                           Ta
                        me                  SQS           s
                      a                                    kin
                    en e
                 Fil ars                                       g
                 3P
                S


       Postfix   MailSender                                       MailReceiver   MailParser   Master DB
                                                                le




                    Ra
                                                              Fi




                       w
                           Mb
                                                           x
                                                        bo




                               ox
                                                       M




                                     Fil
                                                    w




                                        e
                                                  Ra




                                            S3
Loose Coupling Is How I Sleep
         At Night


                                      X
   US-EAST-1C
                                to                     Ta
                          me                SQS           s
                        na e                               kin
                     ile rs                                    g
                  3 F Pa
                S


       Postfix   MailSender                                       MailReceiver   MailParser   Master DB




                                                                 le
                    Ra




                                                              Fi
                       w
                           Mb




                                                           x
                                                        bo
                               ox




                                                       M
                                     Fil




                                                   w
                                        e




                                                  Ra
                                            S3




                                      X
   US-EAST-1B
                                o
                           t                           Ta
                        me                  SQS           s
                      a                                    kin
                    en e
                 Fil ars                                       g
                 3P
                S


       Postfix   MailSender                                       MailReceiver   MailParser   Master DB
                                                                le




                    Ra
                                                              Fi




                       w
                           Mb
                                                           x
                                                        bo




                               ox
                                                       M




                                     Fil
                                                    w




                                        e
                                                  Ra




                                            S3
Loose Coupling Is How I Sleep
         At Night
What It Looks Like
Processing Cloud
                                                                                     MessageParser
                                               h
                                           s
                                        pu
                                  cks                                       Tasks
                                                    MessageParsing Queue
                                a
                            allb e                  MessageClassfier Queue           MessageClassifier
                          c
                        rd nam
                      o                            AnyFeatureWeWant Queue
                    ec file
                  R
               ive S3
            ct
           A
                                                                                    AnyFeatureWorker


Message Created
Workers
•   MessageExporter

•   DailyUpdateSender

•   RegistrationWorker

•   BackupWorker
The Database Is a
 Single Point of
     Failure
How We Cope
                            Hourly
              EBS Volume
                           Snapshots


                                       Weekly
Master DB
                                       Dumps




                                                Expensive Queries
                                                   Memcached
A Dark But
Illustrative Tale
with QuickTime
Manual Stuff

    mkdir /vol
 mount /vol /dev/sdj

sudo god start mysqld
Manual Stuff

change database.yml by hand (yuck)

     queue mail to redeliver
(Wasted)
Troubleshooting:
     15 mins
Provisioning new
   database:
    15 mins
If there was a next
       time...
(standard programmer
   optimism applies)
     < 10 minutes
Replication?
 Sharding?
Key-Value?
Columns?
Logging gem        Splunk




              Logging


                   Good ol’ fashion tail
Zenoss
          Hoptoad

                 Slow query log
   Dash


            Monitoring
          RightScale

Alertra                           New Relic
Questions?
 mike@oib.com

   @subelsky
  @otherinbox
Join our team
Upgrade your job            Open Positions
•   Smart, fun people
                            • Ruby on Rails Developer
•   Startup environment
                            • Ruby on Rails Intern
•   Cutting edge tech
                            • Product Manager
•   Competitive pay
•   Stock options




We pay $1,000 for successful referrals!
rate.sxsw.com
Thank
 you!

Weitere ähnliche Inhalte

Ähnlich wie Scaling Rails Applications In The Cloud

A Methodology for the Emulation of Boolean Logic that Paved the Way for the S...
A Methodology for the Emulation of Boolean Logic that Paved the Way for the S...A Methodology for the Emulation of Boolean Logic that Paved the Way for the S...
A Methodology for the Emulation of Boolean Logic that Paved the Way for the S...
ricky_pi_tercios
 
Http Jaoo.Com.Au Sydney 2008 File Path= Jaoo Aus2008 Slides Dave Thomas Lif...
Http   Jaoo.Com.Au Sydney 2008 File Path= Jaoo Aus2008 Slides Dave Thomas Lif...Http   Jaoo.Com.Au Sydney 2008 File Path= Jaoo Aus2008 Slides Dave Thomas Lif...
Http Jaoo.Com.Au Sydney 2008 File Path= Jaoo Aus2008 Slides Dave Thomas Lif...
qedanne
 
Cms forum, future of Web Content Management
Cms forum, future of Web Content ManagementCms forum, future of Web Content Management
Cms forum, future of Web Content Management
guest88136a
 
Joyent Cloud Advantages
Joyent Cloud AdvantagesJoyent Cloud Advantages
Joyent Cloud Advantages
dmasciorini
 

Ähnlich wie Scaling Rails Applications In The Cloud (20)

A Methodology for the Emulation of Boolean Logic that Paved the Way for the S...
A Methodology for the Emulation of Boolean Logic that Paved the Way for the S...A Methodology for the Emulation of Boolean Logic that Paved the Way for the S...
A Methodology for the Emulation of Boolean Logic that Paved the Way for the S...
 
Http Jaoo.Com.Au Sydney 2008 File Path= Jaoo Aus2008 Slides Dave Thomas Lif...
Http   Jaoo.Com.Au Sydney 2008 File Path= Jaoo Aus2008 Slides Dave Thomas Lif...Http   Jaoo.Com.Au Sydney 2008 File Path= Jaoo Aus2008 Slides Dave Thomas Lif...
Http Jaoo.Com.Au Sydney 2008 File Path= Jaoo Aus2008 Slides Dave Thomas Lif...
 
近未来的並列 LL
近未来的並列 LL近未来的並列 LL
近未来的並列 LL
 
SAM SIG: Hadoop architecture, MapReduce patterns, and best practices with Cas...
SAM SIG: Hadoop architecture, MapReduce patterns, and best practices with Cas...SAM SIG: Hadoop architecture, MapReduce patterns, and best practices with Cas...
SAM SIG: Hadoop architecture, MapReduce patterns, and best practices with Cas...
 
雲端與Big data
雲端與Big data雲端與Big data
雲端與Big data
 
Tom Krcha - Future of Flash
Tom Krcha - Future of FlashTom Krcha - Future of Flash
Tom Krcha - Future of Flash
 
Bangalore cloudstack user group
Bangalore cloudstack user groupBangalore cloudstack user group
Bangalore cloudstack user group
 
Learn OpenStack from trystack.cn ——Folsom in practice
Learn OpenStack from trystack.cn  ——Folsom in practiceLearn OpenStack from trystack.cn  ——Folsom in practice
Learn OpenStack from trystack.cn ——Folsom in practice
 
Microsoft HPC User Group
Microsoft HPC User Group Microsoft HPC User Group
Microsoft HPC User Group
 
Delay Tolerant Streaming Services, Thomas Plagemann, UiO
Delay Tolerant Streaming Services, Thomas Plagemann, UiODelay Tolerant Streaming Services, Thomas Plagemann, UiO
Delay Tolerant Streaming Services, Thomas Plagemann, UiO
 
Embracing concurrency for fun utility and simpler code
Embracing concurrency for fun utility and simpler codeEmbracing concurrency for fun utility and simpler code
Embracing concurrency for fun utility and simpler code
 
Challenges in Maintaining a High Performance Search Engine Written in Java
Challenges in Maintaining a High Performance Search Engine Written in JavaChallenges in Maintaining a High Performance Search Engine Written in Java
Challenges in Maintaining a High Performance Search Engine Written in Java
 
Future of WCM - CM Forum Belgium
Future of WCM - CM Forum BelgiumFuture of WCM - CM Forum Belgium
Future of WCM - CM Forum Belgium
 
Cms forum, future of Web Content Management
Cms forum, future of Web Content ManagementCms forum, future of Web Content Management
Cms forum, future of Web Content Management
 
Creating Cloud Communities
Creating Cloud CommunitiesCreating Cloud Communities
Creating Cloud Communities
 
Joyent Cloud Advantages
Joyent Cloud AdvantagesJoyent Cloud Advantages
Joyent Cloud Advantages
 
OpenStack and OpenFlow Demos
OpenStack and OpenFlow DemosOpenStack and OpenFlow Demos
OpenStack and OpenFlow Demos
 
Automating Oracle Database deployment with Amazon Web Services, fabric, and boto
Automating Oracle Database deployment with Amazon Web Services, fabric, and botoAutomating Oracle Database deployment with Amazon Web Services, fabric, and boto
Automating Oracle Database deployment with Amazon Web Services, fabric, and boto
 
Behind the Scenes at LiveJournal: Scaling Storytime
Behind the Scenes at LiveJournal: Scaling StorytimeBehind the Scenes at LiveJournal: Scaling Storytime
Behind the Scenes at LiveJournal: Scaling Storytime
 
vBrownBag OpenStack Networking Talk
vBrownBag OpenStack Networking TalkvBrownBag OpenStack Networking Talk
vBrownBag OpenStack Networking Talk
 

Mehr von Mike Subelsky

SproutCore and the Future of Web Apps
SproutCore and the Future of Web AppsSproutCore and the Future of Web Apps
SproutCore and the Future of Web Apps
Mike Subelsky
 

Mehr von Mike Subelsky (12)

STAQ Development Manual (Redacted)
STAQ Development Manual (Redacted)STAQ Development Manual (Redacted)
STAQ Development Manual (Redacted)
 
Coding for uncertainty
Coding for uncertaintyCoding for uncertainty
Coding for uncertainty
 
Ruby Concurrency Realities
Ruby Concurrency RealitiesRuby Concurrency Realities
Ruby Concurrency Realities
 
Generating Good Ideas
Generating Good IdeasGenerating Good Ideas
Generating Good Ideas
 
Baltimore: A Great Place to Pick Up Ideas and Run WIth Them (Ignite)
Baltimore: A Great Place to Pick Up Ideas and Run WIth Them (Ignite)Baltimore: A Great Place to Pick Up Ideas and Run WIth Them (Ignite)
Baltimore: A Great Place to Pick Up Ideas and Run WIth Them (Ignite)
 
Let's Make Baltimore More Innovative (TEDxBaltimore)
Let's Make Baltimore More Innovative (TEDxBaltimore)Let's Make Baltimore More Innovative (TEDxBaltimore)
Let's Make Baltimore More Innovative (TEDxBaltimore)
 
Social Media for Everybody
Social Media for EverybodySocial Media for Everybody
Social Media for Everybody
 
Ruby For Startups
Ruby For StartupsRuby For Startups
Ruby For Startups
 
It's Not Always Sunny in the Clouds
It's Not Always Sunny in the CloudsIt's Not Always Sunny in the Clouds
It's Not Always Sunny in the Clouds
 
Introduction to SproutCore at JSConf
Introduction to SproutCore at JSConfIntroduction to SproutCore at JSConf
Introduction to SproutCore at JSConf
 
SproutCore and the Future of Web Apps
SproutCore and the Future of Web AppsSproutCore and the Future of Web Apps
SproutCore and the Future of Web Apps
 
SproutCore and the Future of Web Apps
SproutCore and the Future of Web AppsSproutCore and the Future of Web Apps
SproutCore and the Future of Web Apps
 

Kürzlich hochgeladen

Histor y of HAM Radio presentation slide
Histor y of HAM Radio presentation slideHistor y of HAM Radio presentation slide
Histor y of HAM Radio presentation slide
vu2urc
 

Kürzlich hochgeladen (20)

Understanding Discord NSFW Servers A Guide for Responsible Users.pdf
Understanding Discord NSFW Servers A Guide for Responsible Users.pdfUnderstanding Discord NSFW Servers A Guide for Responsible Users.pdf
Understanding Discord NSFW Servers A Guide for Responsible Users.pdf
 
Tech Trends Report 2024 Future Today Institute.pdf
Tech Trends Report 2024 Future Today Institute.pdfTech Trends Report 2024 Future Today Institute.pdf
Tech Trends Report 2024 Future Today Institute.pdf
 
Apidays New York 2024 - The value of a flexible API Management solution for O...
Apidays New York 2024 - The value of a flexible API Management solution for O...Apidays New York 2024 - The value of a flexible API Management solution for O...
Apidays New York 2024 - The value of a flexible API Management solution for O...
 
What Are The Drone Anti-jamming Systems Technology?
What Are The Drone Anti-jamming Systems Technology?What Are The Drone Anti-jamming Systems Technology?
What Are The Drone Anti-jamming Systems Technology?
 
presentation ICT roal in 21st century education
presentation ICT roal in 21st century educationpresentation ICT roal in 21st century education
presentation ICT roal in 21st century education
 
How to Troubleshoot Apps for the Modern Connected Worker
How to Troubleshoot Apps for the Modern Connected WorkerHow to Troubleshoot Apps for the Modern Connected Worker
How to Troubleshoot Apps for the Modern Connected Worker
 
Exploring the Future Potential of AI-Enabled Smartphone Processors
Exploring the Future Potential of AI-Enabled Smartphone ProcessorsExploring the Future Potential of AI-Enabled Smartphone Processors
Exploring the Future Potential of AI-Enabled Smartphone Processors
 
How to Troubleshoot Apps for the Modern Connected Worker
How to Troubleshoot Apps for the Modern Connected WorkerHow to Troubleshoot Apps for the Modern Connected Worker
How to Troubleshoot Apps for the Modern Connected Worker
 
Handwritten Text Recognition for manuscripts and early printed texts
Handwritten Text Recognition for manuscripts and early printed textsHandwritten Text Recognition for manuscripts and early printed texts
Handwritten Text Recognition for manuscripts and early printed texts
 
Boost Fertility New Invention Ups Success Rates.pdf
Boost Fertility New Invention Ups Success Rates.pdfBoost Fertility New Invention Ups Success Rates.pdf
Boost Fertility New Invention Ups Success Rates.pdf
 
Real Time Object Detection Using Open CV
Real Time Object Detection Using Open CVReal Time Object Detection Using Open CV
Real Time Object Detection Using Open CV
 
Boost PC performance: How more available memory can improve productivity
Boost PC performance: How more available memory can improve productivityBoost PC performance: How more available memory can improve productivity
Boost PC performance: How more available memory can improve productivity
 
HTML Injection Attacks: Impact and Mitigation Strategies
HTML Injection Attacks: Impact and Mitigation StrategiesHTML Injection Attacks: Impact and Mitigation Strategies
HTML Injection Attacks: Impact and Mitigation Strategies
 
TrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
TrustArc Webinar - Stay Ahead of US State Data Privacy Law DevelopmentsTrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
TrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
 
Automating Google Workspace (GWS) & more with Apps Script
Automating Google Workspace (GWS) & more with Apps ScriptAutomating Google Workspace (GWS) & more with Apps Script
Automating Google Workspace (GWS) & more with Apps Script
 
Histor y of HAM Radio presentation slide
Histor y of HAM Radio presentation slideHistor y of HAM Radio presentation slide
Histor y of HAM Radio presentation slide
 
TrustArc Webinar - Unlock the Power of AI-Driven Data Discovery
TrustArc Webinar - Unlock the Power of AI-Driven Data DiscoveryTrustArc Webinar - Unlock the Power of AI-Driven Data Discovery
TrustArc Webinar - Unlock the Power of AI-Driven Data Discovery
 
AWS Community Day CPH - Three problems of Terraform
AWS Community Day CPH - Three problems of TerraformAWS Community Day CPH - Three problems of Terraform
AWS Community Day CPH - Three problems of Terraform
 
Workshop - Best of Both Worlds_ Combine KG and Vector search for enhanced R...
Workshop - Best of Both Worlds_ Combine  KG and Vector search for  enhanced R...Workshop - Best of Both Worlds_ Combine  KG and Vector search for  enhanced R...
Workshop - Best of Both Worlds_ Combine KG and Vector search for enhanced R...
 
Finology Group – Insurtech Innovation Award 2024
Finology Group – Insurtech Innovation Award 2024Finology Group – Insurtech Innovation Award 2024
Finology Group – Insurtech Innovation Award 2024
 

Scaling Rails Applications In The Cloud

  • 1. Scaling Rails Applications in the Cloud 100 new users in < 2 hours The hashtag for this talk is: #SX09-573
  • 2. A Typical Case Study 32 The Inevitable Mass Adoption of Our Product Public Beta 8 Techcrunch50 Me, Josh, Friends & Family and a 4 Private Beta few other people The Inexorable March of Time
  • 5. Solving problems across a flexible grid of nodes
  • 6. ...which you consume as a utility
  • 7. The nodes are elastic
  • 8. The nodes are loosely coupled Tasking System Computing Computing Computing Computing Instance Instance Instance Instance Permanent Storage
  • 9. The nodes are distributed
  • 10. The nodes are ephemeral
  • 11. The nodes are disposable
  • 12. What are the benefits?
  • 15. My Cloud Grows With Me! 32 Heady, Viral, Asymptomtotic Growth of Userbase Public Beta 8 Techcrunch50 Me, Josh, Friends & Family and a 4 Private Beta few other people The Blink of an Eye
  • 16. What are the costs?
  • 17. Some loss of control Potential for lock-in Greater complexity
  • 18. You chose a language that sublimates complexity
  • 19. aws-S3 RightAWS Ruby/EventMachine Ruby Shines Here Hpricot memcache-client ActiveSupport
  • 20.
  • 21. Capistrano Chef Rubber PoolParty Scalr Roll Your Own
  • 22. What do you need to get started?
  • 23. Tasking System Computing Computing Computing Computing Instance Instance Instance Instance Permanent Storage Bonus: Non-RDBMS
  • 24. Our Sky SMTP Sync Staging Staging Staging Staging Staging Staging Processing Master DB Staging Staging Staging Staging Staging Experiment Workers Web
  • 25. Web Cloud Mongrels & Memcached JSON Mongrels & HTML Apache HAProxy Master DB Memcached Atom Mongrels & Memcached Message Bodies Mailbox Export Requests, Domain Registration SQS S3
  • 29. SMTP Cloud o et Ta SQS m s na e kin e Fil ars g S3 P Postfix MailSender MailReceiver MailParser Master DB le Ra Fi w Mb x bo ox M Fil w e Ra S3
  • 30. Loose Coupling Is How I Sleep At Night US-EAST-1C to Ta e SQS am skin en e Fil ars g S3 P Postfix MailSender MailReceiver MailParser Master DB le Ra Fi w Mb x bo ox M Fil w e Ra S3 US-EAST-1B to Ta me SQS s na e kin ile rs g 3 F Pa S Postfix MailSender MailReceiver MailParser Master DB le Ra Fi w Mb x bo ox M Fil w e Ra S3
  • 31. Loose Coupling Is How I Sleep At Night US-EAST-1C to Ta me SQS s na e kin ile rs g 3 F Pa S Postfix MailSender MailReceiver MailParser Master DB le Ra Fi w Mb x bo ox M Fil w e Ra S3 X XX US-EAST-1B o t Ta me SQS s a kin en e Fil ars g 3P S Postfix MailSender MailReceiver MailParser Master DB le Ra Fi w Mb x bo ox M Fil w e Ra S3
  • 32. Loose Coupling Is How I Sleep At Night X US-EAST-1C to Ta me SQS s na e kin ile rs g 3 F Pa S Postfix MailSender MailReceiver MailParser Master DB le Ra Fi w Mb x bo ox M Fil w e Ra S3 X US-EAST-1B o t Ta me SQS s a kin en e Fil ars g 3P S Postfix MailSender MailReceiver MailParser Master DB le Ra Fi w Mb x bo ox M Fil w e Ra S3
  • 33. Loose Coupling Is How I Sleep At Night X US-EAST-1C to Ta me SQS s na e kin ile rs g 3 F Pa S Postfix MailSender MailReceiver MailParser Master DB le Ra Fi w Mb x bo ox M Fil w e Ra S3 X US-EAST-1B o t Ta me SQS s a kin en e Fil ars g 3P S Postfix MailSender MailReceiver MailParser Master DB le Ra Fi w Mb x bo ox M Fil w e Ra S3
  • 34. Loose Coupling Is How I Sleep At Night X US-EAST-1C to Ta me SQS s na e kin ile rs g 3 F Pa S Postfix MailSender MailReceiver MailParser Master DB le Ra Fi w Mb x bo ox M Fil w e Ra S3 X US-EAST-1B o t Ta me SQS s a kin en e Fil ars g 3P S Postfix MailSender MailReceiver MailParser Master DB le Ra Fi w Mb x bo ox M Fil w e Ra S3
  • 35. Loose Coupling Is How I Sleep At Night
  • 37. Processing Cloud MessageParser h s pu cks Tasks MessageParsing Queue a allb e MessageClassfier Queue MessageClassifier c rd nam o AnyFeatureWeWant Queue ec file R ive S3 ct A AnyFeatureWorker Message Created
  • 38. Workers • MessageExporter • DailyUpdateSender • RegistrationWorker • BackupWorker
  • 39. The Database Is a Single Point of Failure
  • 40. How We Cope Hourly EBS Volume Snapshots Weekly Master DB Dumps Expensive Queries Memcached
  • 41. A Dark But Illustrative Tale with QuickTime
  • 42.
  • 43.
  • 44.
  • 45.
  • 46. Manual Stuff mkdir /vol mount /vol /dev/sdj sudo god start mysqld
  • 47. Manual Stuff change database.yml by hand (yuck) queue mail to redeliver
  • 49. Provisioning new database: 15 mins
  • 50. If there was a next time...
  • 51. (standard programmer optimism applies) < 10 minutes
  • 54. Logging gem Splunk Logging Good ol’ fashion tail
  • 55. Zenoss Hoptoad Slow query log Dash Monitoring RightScale Alertra New Relic
  • 56. Questions? mike@oib.com @subelsky @otherinbox
  • 57. Join our team Upgrade your job Open Positions • Smart, fun people • Ruby on Rails Developer • Startup environment • Ruby on Rails Intern • Cutting edge tech • Product Manager • Competitive pay • Stock options We pay $1,000 for successful referrals!