SlideShare ist ein Scribd-Unternehmen logo
1 von 63
Scaling an API
From Reboots to Redis, how (not) to do it
{
    "fullName": "Ciarán Rooney",
    "workLife": {
        "company": "Skimlinks",
        "position": "CTO",
        "website": "http://skimlinks.com"
    },
    "homeLife": {
        "email": "ciaran.rooney@gmail.com",
        "twitter": "@CiaranR",
        "website": "http://ciaran.ie"
        "hobby": "http://phplondon.org"
    }
}
{
    "dataYear": "2012",
    "dataMonth": "august",
    "dataPeriod": "31 days",
    "dataRequests": {
        "apiRequests": "14,256,654,541",
        "impressions": "4,073,142,857",
        "uniqueUsers": "145,137,679"
    },
    "customers": {
        "websitesPowered": "987,348",
        "customers": "26,379",
    }
}
Scaling an API
Scaling an API
Scaling an API
Scaling an API
Scaling an API
Scaling an API
Scaling an API
Scaling an API
Scaling an API
Scaling an API
Scaling an API
Scaling an API
BIND / DNS

   eMail

imageStorage
eMail

imageStorage
imageStorage
Scaling an API
Scaling an API
Scaling an API
Scaling an API
Scaling an API
Scaling an API
APIs
Database
Master/
Slave
Client App
Client App


Database
Master/
Slave




APIs
Scaling an API
WordPress
WordPress   Pinterest
Scaling an API
Black Friday '09
Black Friday '09

              Black Friday '10
Black Friday '09            Black Friday '11

              Black Friday '10
Scaling an API
Scaling an API
Client App


Database
Master/
Slave




APIs
WordPress
Scaling an API
Scaling an API
SkimWords   ™
Scaling an API
APIxx
CLUSTERxx




APIxx   APIxx   APIxx   APIxx   APIxx   APIxx   APIxx   APIxx




                                    ELB
                                    ELB
US-WEST     US-EAST               EU-WEST




CLUSTERxx   CLUSTERxx CLUSTERxx   CLUSTERxx CLUSTERxx
Scaling an API
Scaling an API
Black Friday '10
Pinterest
Scaling an API
Scaling an API
Scaling an API
Scaling an API
JAYxx


Apache Flume
CLUSTERxx

                            Apache Flume
                              Collector




        C   Prefix Tree
                              C      NLP
                                  Filtering


JAYxx   JAYxx   JAYxx     JAYxx   JAYxx    JAYxx   JAYxx   JAYxx




                                      ELB
                                       ELB
US-WEST




            US-EAST               EU-WEST


                                            Apache Flume
                                               Master




CLUSTERxx   CLUSTERxx CLUSTERxx   CLUSTERxx CLUSTERxx
Scaling an API
Cacti
Scaling an API
{
    "dataYear": "2012",
    "dataMonth": "august",
    "dataPeriod": "31 days",
    "dataRequests": {
        "apiRequests": "14,256,654,541",
        "impressions": "4,073,142,857",
        "uniqueUsers": "145,137,679"
    },
    "customers": {
        "websitesPowered": "987,348",
        "customers": "26,379",
    }
}
Scaling an API
From Reboots to Redis, how (not) to do it



           Questions?
{
    "fullName": "Ciarán Rooney",
    "workLife": {
        "company": "Skimlinks",
        "position": "CTO",
        "website": "http://skimlinks.com"
    },
    "homeLife": {
        "email": "ciaran.rooney@gmail.com",
        "twitter": "@CiaranR",
        "website": "http://ciaran.ie"
        "hobby": "http://phplondon.org"
    }
}

Weitere ähnliche Inhalte

Andere mochten auch

Monetising Social Media By Vanessa Tindale
Monetising Social Media By Vanessa TindaleMonetising Social Media By Vanessa Tindale
Monetising Social Media By Vanessa TindaleCiaran Rooney
 
Mafalda Feliz2010
Mafalda Feliz2010Mafalda Feliz2010
Mafalda Feliz2010Chesana
 
Social media and nursing
Social media and nursingSocial media and nursing
Social media and nursingkgraetz
 
Scaling an API: From Reboots to Redis, how (not) to do it
Scaling an API: From Reboots to Redis, how (not) to do itScaling an API: From Reboots to Redis, how (not) to do it
Scaling an API: From Reboots to Redis, how (not) to do itCiaran Rooney
 
Protein X at The Sway By William Rowe
Protein X at The Sway By William RoweProtein X at The Sway By William Rowe
Protein X at The Sway By William RoweCiaran Rooney
 
Alicias Intro Presentation
Alicias Intro PresentationAlicias Intro Presentation
Alicias Intro PresentationCiaran Rooney
 
Narrating PowerPoint Slides
Narrating PowerPoint SlidesNarrating PowerPoint Slides
Narrating PowerPoint Slideskgraetz
 

Andere mochten auch (9)

Monetising Social Media By Vanessa Tindale
Monetising Social Media By Vanessa TindaleMonetising Social Media By Vanessa Tindale
Monetising Social Media By Vanessa Tindale
 
Performance test
Performance testPerformance test
Performance test
 
Mafalda Feliz2010
Mafalda Feliz2010Mafalda Feliz2010
Mafalda Feliz2010
 
Lmt Filecopy
Lmt FilecopyLmt Filecopy
Lmt Filecopy
 
Social media and nursing
Social media and nursingSocial media and nursing
Social media and nursing
 
Scaling an API: From Reboots to Redis, how (not) to do it
Scaling an API: From Reboots to Redis, how (not) to do itScaling an API: From Reboots to Redis, how (not) to do it
Scaling an API: From Reboots to Redis, how (not) to do it
 
Protein X at The Sway By William Rowe
Protein X at The Sway By William RoweProtein X at The Sway By William Rowe
Protein X at The Sway By William Rowe
 
Alicias Intro Presentation
Alicias Intro PresentationAlicias Intro Presentation
Alicias Intro Presentation
 
Narrating PowerPoint Slides
Narrating PowerPoint SlidesNarrating PowerPoint Slides
Narrating PowerPoint Slides
 

Ähnlich wie Scaling an API

Building Serverless Applications Using AWS AppSync and Amazon Neptune (SRV307...
Building Serverless Applications Using AWS AppSync and Amazon Neptune (SRV307...Building Serverless Applications Using AWS AppSync and Amazon Neptune (SRV307...
Building Serverless Applications Using AWS AppSync and Amazon Neptune (SRV307...Amazon Web Services
 
The 7 Deadly Sins of API Design
The 7 Deadly Sins of API DesignThe 7 Deadly Sins of API Design
The 7 Deadly Sins of API Designluisw19
 
Development in the could: How do we do it(Cloud computing. Microservices. Faas)
Development in the could: How do we do it(Cloud computing. Microservices. Faas)Development in the could: How do we do it(Cloud computing. Microservices. Faas)
Development in the could: How do we do it(Cloud computing. Microservices. Faas)Preply.com
 
IoT Building Blocks From Edge Devices to Analytics in the Cloud
IoT Building Blocks From Edge Devices to Analytics in the CloudIoT Building Blocks From Edge Devices to Analytics in the Cloud
IoT Building Blocks From Edge Devices to Analytics in the CloudAmazon Web Services
 
Migrate an existing application RESTful API’s to GraphQL using AWS Amplify an...
Migrate an existing application RESTful API’s to GraphQL using AWS Amplify an...Migrate an existing application RESTful API’s to GraphQL using AWS Amplify an...
Migrate an existing application RESTful API’s to GraphQL using AWS Amplify an...Amazon Web Services
 
Amplify로 Neptune 그래프 DB 기반 모바일 앱 만들기 :: 김현민 - AWS Community Day 2019
Amplify로 Neptune 그래프 DB 기반 모바일 앱 만들기 :: 김현민 - AWS Community Day 2019Amplify로 Neptune 그래프 DB 기반 모바일 앱 만들기 :: 김현민 - AWS Community Day 2019
Amplify로 Neptune 그래프 DB 기반 모바일 앱 만들기 :: 김현민 - AWS Community Day 2019AWSKRUG - AWS한국사용자모임
 
Salesforce integration options
Salesforce integration optionsSalesforce integration options
Salesforce integration optionsTechforce Services
 
Rapid API Development with LoopBack/StrongLoop
Rapid API Development with LoopBack/StrongLoopRapid API Development with LoopBack/StrongLoop
Rapid API Development with LoopBack/StrongLoopRaymond Camden
 
Socket applications
Socket applicationsSocket applications
Socket applicationsJoão Moura
 
The liferay case: lessons learned evolving from RPC to Hypermedia REST APIs
The liferay case: lessons learned evolving from RPC to Hypermedia REST APIsThe liferay case: lessons learned evolving from RPC to Hypermedia REST APIs
The liferay case: lessons learned evolving from RPC to Hypermedia REST APIsJorge Ferrer
 
How to develop Alexa Skill Kit based on Serverless Architecture
How to develop Alexa Skill Kit based on Serverless ArchitectureHow to develop Alexa Skill Kit based on Serverless Architecture
How to develop Alexa Skill Kit based on Serverless ArchitectureHidetaka Okamoto
 
AWS Stripe Meetup - Powering UK Startup Economy
AWS Stripe Meetup - Powering UK Startup EconomyAWS Stripe Meetup - Powering UK Startup Economy
AWS Stripe Meetup - Powering UK Startup EconomyAmazon Web Services
 
Fully Automate Application Delivery with Puppet and F5 - PuppetConf 2014
Fully Automate Application Delivery with Puppet and F5 - PuppetConf 2014Fully Automate Application Delivery with Puppet and F5 - PuppetConf 2014
Fully Automate Application Delivery with Puppet and F5 - PuppetConf 2014Puppet
 
「リモートペアプロでマントルを突き抜けろ!」AWS Cloud9でリモートペアプロ&楽々サーバーレス開発
「リモートペアプロでマントルを突き抜けろ!」AWS Cloud9でリモートペアプロ&楽々サーバーレス開発「リモートペアプロでマントルを突き抜けろ!」AWS Cloud9でリモートペアプロ&楽々サーバーレス開発
「リモートペアプロでマントルを突き抜けろ!」AWS Cloud9でリモートペアプロ&楽々サーバーレス開発Atsushi Fukui
 
Building Alexa Skills with AWS | AWS Floor28
Building Alexa Skills with AWS | AWS Floor28Building Alexa Skills with AWS | AWS Floor28
Building Alexa Skills with AWS | AWS Floor28Amazon Web Services
 
Building Alexa Skills - Floor28
Building Alexa Skills - Floor28Building Alexa Skills - Floor28
Building Alexa Skills - Floor28Boaz Ziniman
 
Mасштабирование микросервисов на Go, Matt Heath (Hailo)
Mасштабирование микросервисов на Go, Matt Heath (Hailo)Mасштабирование микросервисов на Go, Matt Heath (Hailo)
Mасштабирование микросервисов на Go, Matt Heath (Hailo)Ontico
 
KazooCon 2014 - Introduction to Kazoo APIs!
KazooCon 2014 - Introduction to Kazoo APIs!KazooCon 2014 - Introduction to Kazoo APIs!
KazooCon 2014 - Introduction to Kazoo APIs!2600Hz
 
Automate Your Alexa Lambda Function Deployment Workflows Using AWS CodeCommit...
Automate Your Alexa Lambda Function Deployment Workflows Using AWS CodeCommit...Automate Your Alexa Lambda Function Deployment Workflows Using AWS CodeCommit...
Automate Your Alexa Lambda Function Deployment Workflows Using AWS CodeCommit...Amazon Web Services
 
Aws summit devops 云端多环境自动化运维和部署
Aws summit devops   云端多环境自动化运维和部署Aws summit devops   云端多环境自动化运维和部署
Aws summit devops 云端多环境自动化运维和部署Leon Li
 

Ähnlich wie Scaling an API (20)

Building Serverless Applications Using AWS AppSync and Amazon Neptune (SRV307...
Building Serverless Applications Using AWS AppSync and Amazon Neptune (SRV307...Building Serverless Applications Using AWS AppSync and Amazon Neptune (SRV307...
Building Serverless Applications Using AWS AppSync and Amazon Neptune (SRV307...
 
The 7 Deadly Sins of API Design
The 7 Deadly Sins of API DesignThe 7 Deadly Sins of API Design
The 7 Deadly Sins of API Design
 
Development in the could: How do we do it(Cloud computing. Microservices. Faas)
Development in the could: How do we do it(Cloud computing. Microservices. Faas)Development in the could: How do we do it(Cloud computing. Microservices. Faas)
Development in the could: How do we do it(Cloud computing. Microservices. Faas)
 
IoT Building Blocks From Edge Devices to Analytics in the Cloud
IoT Building Blocks From Edge Devices to Analytics in the CloudIoT Building Blocks From Edge Devices to Analytics in the Cloud
IoT Building Blocks From Edge Devices to Analytics in the Cloud
 
Migrate an existing application RESTful API’s to GraphQL using AWS Amplify an...
Migrate an existing application RESTful API’s to GraphQL using AWS Amplify an...Migrate an existing application RESTful API’s to GraphQL using AWS Amplify an...
Migrate an existing application RESTful API’s to GraphQL using AWS Amplify an...
 
Amplify로 Neptune 그래프 DB 기반 모바일 앱 만들기 :: 김현민 - AWS Community Day 2019
Amplify로 Neptune 그래프 DB 기반 모바일 앱 만들기 :: 김현민 - AWS Community Day 2019Amplify로 Neptune 그래프 DB 기반 모바일 앱 만들기 :: 김현민 - AWS Community Day 2019
Amplify로 Neptune 그래프 DB 기반 모바일 앱 만들기 :: 김현민 - AWS Community Day 2019
 
Salesforce integration options
Salesforce integration optionsSalesforce integration options
Salesforce integration options
 
Rapid API Development with LoopBack/StrongLoop
Rapid API Development with LoopBack/StrongLoopRapid API Development with LoopBack/StrongLoop
Rapid API Development with LoopBack/StrongLoop
 
Socket applications
Socket applicationsSocket applications
Socket applications
 
The liferay case: lessons learned evolving from RPC to Hypermedia REST APIs
The liferay case: lessons learned evolving from RPC to Hypermedia REST APIsThe liferay case: lessons learned evolving from RPC to Hypermedia REST APIs
The liferay case: lessons learned evolving from RPC to Hypermedia REST APIs
 
How to develop Alexa Skill Kit based on Serverless Architecture
How to develop Alexa Skill Kit based on Serverless ArchitectureHow to develop Alexa Skill Kit based on Serverless Architecture
How to develop Alexa Skill Kit based on Serverless Architecture
 
AWS Stripe Meetup - Powering UK Startup Economy
AWS Stripe Meetup - Powering UK Startup EconomyAWS Stripe Meetup - Powering UK Startup Economy
AWS Stripe Meetup - Powering UK Startup Economy
 
Fully Automate Application Delivery with Puppet and F5 - PuppetConf 2014
Fully Automate Application Delivery with Puppet and F5 - PuppetConf 2014Fully Automate Application Delivery with Puppet and F5 - PuppetConf 2014
Fully Automate Application Delivery with Puppet and F5 - PuppetConf 2014
 
「リモートペアプロでマントルを突き抜けろ!」AWS Cloud9でリモートペアプロ&楽々サーバーレス開発
「リモートペアプロでマントルを突き抜けろ!」AWS Cloud9でリモートペアプロ&楽々サーバーレス開発「リモートペアプロでマントルを突き抜けろ!」AWS Cloud9でリモートペアプロ&楽々サーバーレス開発
「リモートペアプロでマントルを突き抜けろ!」AWS Cloud9でリモートペアプロ&楽々サーバーレス開発
 
Building Alexa Skills with AWS | AWS Floor28
Building Alexa Skills with AWS | AWS Floor28Building Alexa Skills with AWS | AWS Floor28
Building Alexa Skills with AWS | AWS Floor28
 
Building Alexa Skills - Floor28
Building Alexa Skills - Floor28Building Alexa Skills - Floor28
Building Alexa Skills - Floor28
 
Mасштабирование микросервисов на Go, Matt Heath (Hailo)
Mасштабирование микросервисов на Go, Matt Heath (Hailo)Mасштабирование микросервисов на Go, Matt Heath (Hailo)
Mасштабирование микросервисов на Go, Matt Heath (Hailo)
 
KazooCon 2014 - Introduction to Kazoo APIs!
KazooCon 2014 - Introduction to Kazoo APIs!KazooCon 2014 - Introduction to Kazoo APIs!
KazooCon 2014 - Introduction to Kazoo APIs!
 
Automate Your Alexa Lambda Function Deployment Workflows Using AWS CodeCommit...
Automate Your Alexa Lambda Function Deployment Workflows Using AWS CodeCommit...Automate Your Alexa Lambda Function Deployment Workflows Using AWS CodeCommit...
Automate Your Alexa Lambda Function Deployment Workflows Using AWS CodeCommit...
 
Aws summit devops 云端多环境自动化运维和部署
Aws summit devops   云端多环境自动化运维和部署Aws summit devops   云端多环境自动化运维和部署
Aws summit devops 云端多环境自动化运维和部署
 

Kürzlich hochgeladen

ADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDE
ADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDEADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDE
ADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDELiveplex
 
Basic Building Blocks of Internet of Things.
Basic Building Blocks of Internet of Things.Basic Building Blocks of Internet of Things.
Basic Building Blocks of Internet of Things.YounusS2
 
AI You Can Trust - Ensuring Success with Data Integrity Webinar
AI You Can Trust - Ensuring Success with Data Integrity WebinarAI You Can Trust - Ensuring Success with Data Integrity Webinar
AI You Can Trust - Ensuring Success with Data Integrity WebinarPrecisely
 
Introduction to Matsuo Laboratory (ENG).pptx
Introduction to Matsuo Laboratory (ENG).pptxIntroduction to Matsuo Laboratory (ENG).pptx
Introduction to Matsuo Laboratory (ENG).pptxMatsuo Lab
 
Bird eye's view on Camunda open source ecosystem
Bird eye's view on Camunda open source ecosystemBird eye's view on Camunda open source ecosystem
Bird eye's view on Camunda open source ecosystemAsko Soukka
 
Building AI-Driven Apps Using Semantic Kernel.pptx
Building AI-Driven Apps Using Semantic Kernel.pptxBuilding AI-Driven Apps Using Semantic Kernel.pptx
Building AI-Driven Apps Using Semantic Kernel.pptxUdaiappa Ramachandran
 
UiPath Studio Web workshop series - Day 7
UiPath Studio Web workshop series - Day 7UiPath Studio Web workshop series - Day 7
UiPath Studio Web workshop series - Day 7DianaGray10
 
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCostKubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCostMatt Ray
 
Secure your environment with UiPath and CyberArk technologies - Session 1
Secure your environment with UiPath and CyberArk technologies - Session 1Secure your environment with UiPath and CyberArk technologies - Session 1
Secure your environment with UiPath and CyberArk technologies - Session 1DianaGray10
 
VoIP Service and Marketing using Odoo and Asterisk PBX
VoIP Service and Marketing using Odoo and Asterisk PBXVoIP Service and Marketing using Odoo and Asterisk PBX
VoIP Service and Marketing using Odoo and Asterisk PBXTarek Kalaji
 
Comparing Sidecar-less Service Mesh from Cilium and Istio
Comparing Sidecar-less Service Mesh from Cilium and IstioComparing Sidecar-less Service Mesh from Cilium and Istio
Comparing Sidecar-less Service Mesh from Cilium and IstioChristian Posta
 
AI Fame Rush Review – Virtual Influencer Creation In Just Minutes
AI Fame Rush Review – Virtual Influencer Creation In Just MinutesAI Fame Rush Review – Virtual Influencer Creation In Just Minutes
AI Fame Rush Review – Virtual Influencer Creation In Just MinutesMd Hossain Ali
 
Crea il tuo assistente AI con lo Stregatto (open source python framework)
Crea il tuo assistente AI con lo Stregatto (open source python framework)Crea il tuo assistente AI con lo Stregatto (open source python framework)
Crea il tuo assistente AI con lo Stregatto (open source python framework)Commit University
 
Salesforce Miami User Group Event - 1st Quarter 2024
Salesforce Miami User Group Event - 1st Quarter 2024Salesforce Miami User Group Event - 1st Quarter 2024
Salesforce Miami User Group Event - 1st Quarter 2024SkyPlanner
 
How Accurate are Carbon Emissions Projections?
How Accurate are Carbon Emissions Projections?How Accurate are Carbon Emissions Projections?
How Accurate are Carbon Emissions Projections?IES VE
 
activity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdf
activity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdf
activity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdfJamie (Taka) Wang
 
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve Decarbonization
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve DecarbonizationUsing IESVE for Loads, Sizing and Heat Pump Modeling to Achieve Decarbonization
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve DecarbonizationIES VE
 

Kürzlich hochgeladen (20)

ADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDE
ADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDEADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDE
ADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDE
 
Basic Building Blocks of Internet of Things.
Basic Building Blocks of Internet of Things.Basic Building Blocks of Internet of Things.
Basic Building Blocks of Internet of Things.
 
AI You Can Trust - Ensuring Success with Data Integrity Webinar
AI You Can Trust - Ensuring Success with Data Integrity WebinarAI You Can Trust - Ensuring Success with Data Integrity Webinar
AI You Can Trust - Ensuring Success with Data Integrity Webinar
 
Introduction to Matsuo Laboratory (ENG).pptx
Introduction to Matsuo Laboratory (ENG).pptxIntroduction to Matsuo Laboratory (ENG).pptx
Introduction to Matsuo Laboratory (ENG).pptx
 
Bird eye's view on Camunda open source ecosystem
Bird eye's view on Camunda open source ecosystemBird eye's view on Camunda open source ecosystem
Bird eye's view on Camunda open source ecosystem
 
Building AI-Driven Apps Using Semantic Kernel.pptx
Building AI-Driven Apps Using Semantic Kernel.pptxBuilding AI-Driven Apps Using Semantic Kernel.pptx
Building AI-Driven Apps Using Semantic Kernel.pptx
 
UiPath Studio Web workshop series - Day 7
UiPath Studio Web workshop series - Day 7UiPath Studio Web workshop series - Day 7
UiPath Studio Web workshop series - Day 7
 
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCostKubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
 
20230104 - machine vision
20230104 - machine vision20230104 - machine vision
20230104 - machine vision
 
Secure your environment with UiPath and CyberArk technologies - Session 1
Secure your environment with UiPath and CyberArk technologies - Session 1Secure your environment with UiPath and CyberArk technologies - Session 1
Secure your environment with UiPath and CyberArk technologies - Session 1
 
VoIP Service and Marketing using Odoo and Asterisk PBX
VoIP Service and Marketing using Odoo and Asterisk PBXVoIP Service and Marketing using Odoo and Asterisk PBX
VoIP Service and Marketing using Odoo and Asterisk PBX
 
Comparing Sidecar-less Service Mesh from Cilium and Istio
Comparing Sidecar-less Service Mesh from Cilium and IstioComparing Sidecar-less Service Mesh from Cilium and Istio
Comparing Sidecar-less Service Mesh from Cilium and Istio
 
AI Fame Rush Review – Virtual Influencer Creation In Just Minutes
AI Fame Rush Review – Virtual Influencer Creation In Just MinutesAI Fame Rush Review – Virtual Influencer Creation In Just Minutes
AI Fame Rush Review – Virtual Influencer Creation In Just Minutes
 
20150722 - AGV
20150722 - AGV20150722 - AGV
20150722 - AGV
 
Crea il tuo assistente AI con lo Stregatto (open source python framework)
Crea il tuo assistente AI con lo Stregatto (open source python framework)Crea il tuo assistente AI con lo Stregatto (open source python framework)
Crea il tuo assistente AI con lo Stregatto (open source python framework)
 
201610817 - edge part1
201610817 - edge part1201610817 - edge part1
201610817 - edge part1
 
Salesforce Miami User Group Event - 1st Quarter 2024
Salesforce Miami User Group Event - 1st Quarter 2024Salesforce Miami User Group Event - 1st Quarter 2024
Salesforce Miami User Group Event - 1st Quarter 2024
 
How Accurate are Carbon Emissions Projections?
How Accurate are Carbon Emissions Projections?How Accurate are Carbon Emissions Projections?
How Accurate are Carbon Emissions Projections?
 
activity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdf
activity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdf
activity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdf
 
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve Decarbonization
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve DecarbonizationUsing IESVE for Loads, Sizing and Heat Pump Modeling to Achieve Decarbonization
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve Decarbonization
 

Scaling an API

  • 1. Scaling an API From Reboots to Redis, how (not) to do it
  • 2. { "fullName": "Ciarán Rooney", "workLife": { "company": "Skimlinks", "position": "CTO", "website": "http://skimlinks.com" }, "homeLife": { "email": "ciaran.rooney@gmail.com", "twitter": "@CiaranR", "website": "http://ciaran.ie" "hobby": "http://phplondon.org" } }
  • 3. { "dataYear": "2012", "dataMonth": "august", "dataPeriod": "31 days", "dataRequests": { "apiRequests": "14,256,654,541", "impressions": "4,073,142,857", "uniqueUsers": "145,137,679" }, "customers": { "websitesPowered": "987,348", "customers": "26,379", } }
  • 16. BIND / DNS eMail imageStorage
  • 25. APIs
  • 31. WordPress Pinterest
  • 34. Black Friday '09 Black Friday '10
  • 35. Black Friday '09 Black Friday '11 Black Friday '10
  • 42. SkimWords
  • 44. APIxx
  • 45. CLUSTERxx APIxx APIxx APIxx APIxx APIxx APIxx APIxx APIxx ELB ELB
  • 46. US-WEST US-EAST EU-WEST CLUSTERxx CLUSTERxx CLUSTERxx CLUSTERxx CLUSTERxx
  • 56. CLUSTERxx Apache Flume Collector C Prefix Tree C NLP Filtering JAYxx JAYxx JAYxx JAYxx JAYxx JAYxx JAYxx JAYxx ELB ELB
  • 57. US-WEST US-EAST EU-WEST Apache Flume Master CLUSTERxx CLUSTERxx CLUSTERxx CLUSTERxx CLUSTERxx
  • 59. Cacti
  • 61. { "dataYear": "2012", "dataMonth": "august", "dataPeriod": "31 days", "dataRequests": { "apiRequests": "14,256,654,541", "impressions": "4,073,142,857", "uniqueUsers": "145,137,679" }, "customers": { "websitesPowered": "987,348", "customers": "26,379", } }
  • 62. Scaling an API From Reboots to Redis, how (not) to do it Questions?
  • 63. { "fullName": "Ciarán Rooney", "workLife": { "company": "Skimlinks", "position": "CTO", "website": "http://skimlinks.com" }, "homeLife": { "email": "ciaran.rooney@gmail.com", "twitter": "@CiaranR", "website": "http://ciaran.ie" "hobby": "http://phplondon.org" } }

Hinweis der Redaktion

  1. Good Afternoon big lunch, not put you sleep share my experiences So from reboots, To Redis, still works my journey Not how make Not why should But What happens when BIG
  2. So who I am? Ciaran, CTO Skimlinks. former seedcamp Content monetization company service ordinated architecture Internal and external APIs why me? Numbers!
  3. 14 Billion API requests 4 Billion Page views 145 Million Unique users monetize 1million Websites 26 thousand customers didn’t start like that
  4. Started Skimbit.com Rebranded skimit.com social decision making and bookmarking tool Launched 2007 topic create a page
  5. bookmark items brides maids shoes friends and family comments / rating bit like another site
  6. But we’ll come back to that site in a bit
  7. going ok steady base of users ticking along nicely something BIG happened
  8. We got Techcrunched!! Website didn’t go down So …
  9. Partied! cut short then something else happened
  10. hosted at FlexiScale anyone? No? Small, local, cloud CEO’s number on speed dial Everything we need
  11. App very straight forward It looked like this Few still like this? node.js on solar powered raspberry pi’s Ours & we were happy with it one day this happened ->
  12. three day outage to the hour wasn’t our fault Well really it was
  13. the cloud floated away human error deleted main storage restore via remount Not enough hardware ship from Germany Didn’t have a backup plan
  14. We panicked! CEO’s number no answer
  15. Our app is simple Just deploy Not that easy We thought But…
  16. DNS servers were OUR server Email records on OUR Server Even customer uploaded images on it This is how NOT to do it
  17. DNS was the hardest 24 Hour TTL on NS records
  18. Email right after Already using Google Apps 30 min TTL on MX Once DNS back ..
  19. Finally Back up images to S3
  20. Shortly after this We started October 2008
  21. Well more like this Fist customer live Nov 10 2008
  22. We business means new website Nov 2008
  23. New hosting company The Planet Real dedicated servers No cloud float Backward? Sensible
  24. New App New architecture Half dozen servers
  25. 2 API boxes Mostly read Writes local
  26. Master slave DB MySQL replication Batch writes to master
  27. Finally client app Reports Website Etc Last mention
  28. Simple Unbreakable
  29. Growth over the next few years Interesting events
  30. Signed Wordpress
  31. Pinterest took off!
  32. Zoom in Monetizable events More insight
  33. BF 2009
  34. BF 2010
  35. BF 2011
  36. Remember our site
  37. Hosting = planet Real severs Never float
  38. Archictecure = this Never could break
  39. Stopped recording Important lesson Only record what you need Later we did record
  40. International traffic growing 40% = USA 20% = UK 10% = France 30% =others like germany canada
  41. Started to improve New Website easly 2010
  42. New product Skimwords Analyses content Creates links
  43. Great thing New products go on new servers Eventually all on AWS This where start talking about the right way
  44. Created API boxen Apache PHP Memcached
  45. Scaled out API Added ELB Backed by MYSQL Skimwords = MongoDB
  46. Amazon regions Availabily zones Mysql Master slave Mongo Master Slave Mongo 1.6 Ultra DNS = GLB Failover Short TTL
  47. It worked Latency dropped huge
  48. World Graphs Blue = europe Green = us east Red = us west
  49. We were where BF ‘10
  50. And this was coming Pinterest over 1 year Growing fast
  51. First step New website Early 2011
  52. We bought Atma
  53. And did some research Bake off Apache Worker = Not thread safe PHP Drivers Mongo at the time
  54. Picked Python Why not scala? Many reasons Team skills
  55. New Jay Box Jay = Json Ngix PHP/Apache Tornado / Python Flume for logs+events
  56. Make a Cluster Jays ELB Mysql Memcached Flume C Tree Filter + Redis
  57. Again International Clusters Balance with Ultra DNS Mysql Master Slave Flume -> S3 Hadoop Hbase
  58. Other tools Puppet Jenkins-ci / RMP’s / Hubot Fastly Api Axle
  59. Monitoring Local Cacti Icinga aka nagios Pindom & Padgerduty
  60. NOC Always watching
  61. Remember numbers
  62. Good Afternoon big lunch, not put you sleep share my experiences So from reboots, To Redis, still works my journey Not how make Not why should But What happens when BIG
  63. My Details Hope you got something from this We learned a lot Hope you can apply to your business here for the rest of day Hope you all get a lot out of the rest of the week