SlideShare ist ein Scribd-Unternehmen logo
1 von 16
Downloaden Sie, um offline zu lesen
What’s behind
 facebook
     A Jen
• font-end
  o HipHop
  o BigPipe
• back-end           outline
  o Thrift
  o Scribe
  o Tornado
• data persistence
• other
font-end
Linux & Apache
 PHP + HipHop
    BigPipe
HipHop
•   PHP complier developed by Facebook.
•   Convert PHP code to C ++.
•   Compiled by gcc.
•   CPU usage recudes 50%.
•   Git Hub
BigPipe
• It’s a fundamental redesign of the dynamic web.
  page serving system.
• Decomposing web pages into 「pagelets」.
• Implemented by ONLY PHP + JavaScript!
• Link
back-end
    Thrift
   Scribe
  Tornado
Thrift
• Cross-language services development.
• Code generation engine.
• Sample Code




   definition file   thrift complier   source code
Scribe
• Distributed Loggin System.
• git hub
Tornado
•   Web Server written by python.
•   For real-time web services
•   non-blocking web service.
•   git hub
•   show case
•   link
data persistence
            MySQL
 Hive + Hadoop + Hbase + HDFS
          Memcache
           Haystack
MySQL
• It’s still backbon of database infra.
• How ?
   o Learn more : link
   o facebook patches
Hive + hadoop + Hbase +
         HDFS
             Hive

            hadoop

             Hbase

             HDFS
Memcache
•   Distributed memory object caching system
•   MySQL Cache
•   General purpose cache
•   Link
Haystack
• Efficient storage of billions of photos.
• Reducing Disk I/O.
• Components
   o Haystack Diretory
   o Haystack Store
   o Haystack Cache

• Link
Other
• Chat
  o Epoll Server using Erlang
  o Accessed using thrift

• Message Search
  o Inverted index stored in HBase
Thank you
   A Jen

Weitere ähnliche Inhalte

Was ist angesagt?

Qcon 090408233824-phpapp01
Qcon 090408233824-phpapp01Qcon 090408233824-phpapp01
Qcon 090408233824-phpapp01
jgregory1234
 
Storage Infrastructure Behind Facebook Messages
Storage Infrastructure Behind Facebook MessagesStorage Infrastructure Behind Facebook Messages
Storage Infrastructure Behind Facebook Messages
feng1212
 
How facebook works and function- a complete approach
How facebook works and function- a complete approachHow facebook works and function- a complete approach
How facebook works and function- a complete approach
Prakhar Gethe
 
Yahoo Communities Architecture Unlikely Bedfellows
Yahoo Communities Architecture Unlikely BedfellowsYahoo Communities Architecture Unlikely Bedfellows
Yahoo Communities Architecture Unlikely Bedfellows
ConSanFrancisco123
 
Yahoo Pipes Middleware In The Cloud
Yahoo Pipes Middleware In The CloudYahoo Pipes Middleware In The Cloud
Yahoo Pipes Middleware In The Cloud
ConSanFrancisco123
 
PUBLISHED: Facebook Cloud Computing
PUBLISHED: Facebook Cloud ComputingPUBLISHED: Facebook Cloud Computing
PUBLISHED: Facebook Cloud Computing
RichardBatka
 
Open Ap Is State Of The Market
Open Ap Is State Of The MarketOpen Ap Is State Of The Market
Open Ap Is State Of The Market
ConSanFrancisco123
 

Was ist angesagt? (17)

Hadoop World 2011: Apache HBase Road Map - Jonathan Gray - Facebook
Hadoop World 2011: Apache HBase Road Map - Jonathan Gray - FacebookHadoop World 2011: Apache HBase Road Map - Jonathan Gray - Facebook
Hadoop World 2011: Apache HBase Road Map - Jonathan Gray - Facebook
 
Qcon
QconQcon
Qcon
 
Qcon 090408233824-phpapp01
Qcon 090408233824-phpapp01Qcon 090408233824-phpapp01
Qcon 090408233824-phpapp01
 
High Scalability by Example – How can Web-Architecture scale like Facebook, T...
High Scalability by Example – How can Web-Architecture scale like Facebook, T...High Scalability by Example – How can Web-Architecture scale like Facebook, T...
High Scalability by Example – How can Web-Architecture scale like Facebook, T...
 
RBS in SharePoint
RBS in SharePointRBS in SharePoint
RBS in SharePoint
 
Storage Infrastructure Behind Facebook Messages
Storage Infrastructure Behind Facebook MessagesStorage Infrastructure Behind Facebook Messages
Storage Infrastructure Behind Facebook Messages
 
How facebook works and function- a complete approach
How facebook works and function- a complete approachHow facebook works and function- a complete approach
How facebook works and function- a complete approach
 
Remote Blog Storage (RBS) Best Practices in SharePoint 2010 - EPC Group
Remote Blog Storage (RBS) Best Practices in SharePoint 2010 - EPC GroupRemote Blog Storage (RBS) Best Practices in SharePoint 2010 - EPC Group
Remote Blog Storage (RBS) Best Practices in SharePoint 2010 - EPC Group
 
Yahoo Communities Architecture Unlikely Bedfellows
Yahoo Communities Architecture Unlikely BedfellowsYahoo Communities Architecture Unlikely Bedfellows
Yahoo Communities Architecture Unlikely Bedfellows
 
Yahoo Pipes Middleware In The Cloud
Yahoo Pipes Middleware In The CloudYahoo Pipes Middleware In The Cloud
Yahoo Pipes Middleware In The Cloud
 
BRAINREPUBLIC - Powered by no-SQL
BRAINREPUBLIC - Powered by no-SQLBRAINREPUBLIC - Powered by no-SQL
BRAINREPUBLIC - Powered by no-SQL
 
LAMP
LAMPLAMP
LAMP
 
PUBLISHED: Facebook Cloud Computing
PUBLISHED: Facebook Cloud ComputingPUBLISHED: Facebook Cloud Computing
PUBLISHED: Facebook Cloud Computing
 
Introduction To Web Development & The New Digital Workplace
Introduction To Web Development & The New Digital WorkplaceIntroduction To Web Development & The New Digital Workplace
Introduction To Web Development & The New Digital Workplace
 
Produce & Publish Authoring Environment V 2.0 (english version)
Produce & Publish Authoring Environment V 2.0 (english version)Produce & Publish Authoring Environment V 2.0 (english version)
Produce & Publish Authoring Environment V 2.0 (english version)
 
NOSQL, CouchDB, and the Cloud
NOSQL, CouchDB, and the CloudNOSQL, CouchDB, and the Cloud
NOSQL, CouchDB, and the Cloud
 
Open Ap Is State Of The Market
Open Ap Is State Of The MarketOpen Ap Is State Of The Market
Open Ap Is State Of The Market
 

Andere mochten auch

Big Data in Real-Time at Twitter
Big Data in Real-Time at TwitterBig Data in Real-Time at Twitter
Big Data in Real-Time at Twitter
nkallen
 
Social and-psychological-impact-of-social-networking-sites
Social and-psychological-impact-of-social-networking-sitesSocial and-psychological-impact-of-social-networking-sites
Social and-psychological-impact-of-social-networking-sites
Sumit Malhotra
 
Network Security July 1
Network Security July 1Network Security July 1
Network Security July 1
Jd Mercado
 
Yii Framework Security
Yii Framework SecurityYii Framework Security
Yii Framework Security
Ilko Kacharov
 

Andere mochten auch (20)

Facebook architecture
Facebook architectureFacebook architecture
Facebook architecture
 
Facebook architecture presentation: scalability challenge
Facebook architecture presentation: scalability challengeFacebook architecture presentation: scalability challenge
Facebook architecture presentation: scalability challenge
 
thrift-20070401
thrift-20070401thrift-20070401
thrift-20070401
 
Python at Facebook
Python at FacebookPython at Facebook
Python at Facebook
 
Big Data in Real-Time at Twitter
Big Data in Real-Time at TwitterBig Data in Real-Time at Twitter
Big Data in Real-Time at Twitter
 
Social and-psychological-impact-of-social-networking-sites
Social and-psychological-impact-of-social-networking-sitesSocial and-psychological-impact-of-social-networking-sites
Social and-psychological-impact-of-social-networking-sites
 
Social network architecture - Part 2. News feed
Social network architecture - Part 2. News feedSocial network architecture - Part 2. News feed
Social network architecture - Part 2. News feed
 
Social network architecture - Part 1. Core user
Social network architecture - Part 1. Core userSocial network architecture - Part 1. Core user
Social network architecture - Part 1. Core user
 
Social network architecture - Part 3. Big data - Machine learning
Social network architecture - Part 3. Big data - Machine learningSocial network architecture - Part 3. Big data - Machine learning
Social network architecture - Part 3. Big data - Machine learning
 
Social Media, Cloud Computing and architecture
Social Media, Cloud Computing and architectureSocial Media, Cloud Computing and architecture
Social Media, Cloud Computing and architecture
 
Researching Social Media – Big Data and Social Media Analysis
Researching Social Media – Big Data and Social Media AnalysisResearching Social Media – Big Data and Social Media Analysis
Researching Social Media – Big Data and Social Media Analysis
 
Network Security July 1
Network Security July 1Network Security July 1
Network Security July 1
 
Collapse of angolan banking system copy
Collapse of angolan banking system copyCollapse of angolan banking system copy
Collapse of angolan banking system copy
 
Open Source Software Concepts
Open Source Software ConceptsOpen Source Software Concepts
Open Source Software Concepts
 
Cyberpunk
CyberpunkCyberpunk
Cyberpunk
 
Yii Framework Security
Yii Framework SecurityYii Framework Security
Yii Framework Security
 
Informatics Practices Chapter 2 Open Source Software Concepts Class 12th
 Informatics Practices Chapter 2  Open Source Software Concepts Class 12th Informatics Practices Chapter 2  Open Source Software Concepts Class 12th
Informatics Practices Chapter 2 Open Source Software Concepts Class 12th
 
CyberPunk
CyberPunkCyberPunk
CyberPunk
 
Social networking
Social networkingSocial networking
Social networking
 
Social Networking: Past, Present & Future
Social Networking: Past, Present & FutureSocial Networking: Past, Present & Future
Social Networking: Past, Present & Future
 

Ähnlich wie What's behind facebook

Big Data in the Microsoft Platform
Big Data in the Microsoft PlatformBig Data in the Microsoft Platform
Big Data in the Microsoft Platform
Jesus Rodriguez
 

Ähnlich wie What's behind facebook (20)

Big Data and Hadoop - History, Technical Deep Dive, and Industry Trends
Big Data and Hadoop - History, Technical Deep Dive, and Industry TrendsBig Data and Hadoop - History, Technical Deep Dive, and Industry Trends
Big Data and Hadoop - History, Technical Deep Dive, and Industry Trends
 
Big Data and Hadoop - History, Technical Deep Dive, and Industry Trends
Big Data and Hadoop - History, Technical Deep Dive, and Industry TrendsBig Data and Hadoop - History, Technical Deep Dive, and Industry Trends
Big Data and Hadoop - History, Technical Deep Dive, and Industry Trends
 
HBase app HUG talk
HBase app HUG talkHBase app HUG talk
HBase app HUG talk
 
CIS13: Big Data Platform Vendor’s Perspective: Insights from the Bleeding Edge
CIS13: Big Data Platform Vendor’s Perspective: Insights from the Bleeding EdgeCIS13: Big Data Platform Vendor’s Perspective: Insights from the Bleeding Edge
CIS13: Big Data Platform Vendor’s Perspective: Insights from the Bleeding Edge
 
Building Google-in-a-box: using Apache SolrCloud and Bigtop to index your big...
Building Google-in-a-box: using Apache SolrCloud and Bigtop to index your big...Building Google-in-a-box: using Apache SolrCloud and Bigtop to index your big...
Building Google-in-a-box: using Apache SolrCloud and Bigtop to index your big...
 
Infrastructure, Hiphop for PHP, deploy @ Hyves
Infrastructure, Hiphop for PHP, deploy @ HyvesInfrastructure, Hiphop for PHP, deploy @ Hyves
Infrastructure, Hiphop for PHP, deploy @ Hyves
 
Big data and hadoop anupama
Big data and hadoop anupamaBig data and hadoop anupama
Big data and hadoop anupama
 
Intro to HBase - Lars George
Intro to HBase - Lars GeorgeIntro to HBase - Lars George
Intro to HBase - Lars George
 
HyperDB, MySQL Performance, & Flavors of MySQL
HyperDB, MySQL Performance, & Flavors of MySQLHyperDB, MySQL Performance, & Flavors of MySQL
HyperDB, MySQL Performance, & Flavors of MySQL
 
IPFS: The Permanent Web
IPFS: The Permanent WebIPFS: The Permanent Web
IPFS: The Permanent Web
 
Big Data in the Microsoft Platform
Big Data in the Microsoft PlatformBig Data in the Microsoft Platform
Big Data in the Microsoft Platform
 
Trend Micro Big Data Platform and Apache Bigtop
Trend Micro Big Data Platform and Apache BigtopTrend Micro Big Data Platform and Apache Bigtop
Trend Micro Big Data Platform and Apache Bigtop
 
High Voltage - Building Static Sites With Wordpress-Managed Content
High Voltage - Building Static Sites With Wordpress-Managed ContentHigh Voltage - Building Static Sites With Wordpress-Managed Content
High Voltage - Building Static Sites With Wordpress-Managed Content
 
Hp hadoop platform
Hp hadoop platformHp hadoop platform
Hp hadoop platform
 
Tcloud Computing Hadoop Family and Ecosystem Service 2013.Q2
Tcloud Computing Hadoop Family and Ecosystem Service 2013.Q2Tcloud Computing Hadoop Family and Ecosystem Service 2013.Q2
Tcloud Computing Hadoop Family and Ecosystem Service 2013.Q2
 
List of Engineering Colleges in Uttarakhand
List of Engineering Colleges in UttarakhandList of Engineering Colleges in Uttarakhand
List of Engineering Colleges in Uttarakhand
 
Hadoop.pptx
Hadoop.pptxHadoop.pptx
Hadoop.pptx
 
Hadoop.pptx
Hadoop.pptxHadoop.pptx
Hadoop.pptx
 
Apache Giraph: start analyzing graph relationships in your bigdata in 45 minu...
Apache Giraph: start analyzing graph relationships in your bigdata in 45 minu...Apache Giraph: start analyzing graph relationships in your bigdata in 45 minu...
Apache Giraph: start analyzing graph relationships in your bigdata in 45 minu...
 
Concepts on Hadoop
Concepts on HadoopConcepts on Hadoop
Concepts on Hadoop
 

Mehr von Ajen 陳

Mobile Web vs Native App
Mobile Web vs Native AppMobile Web vs Native App
Mobile Web vs Native App
Ajen 陳
 
敏捷開發 - Scrum
敏捷開發 - Scrum敏捷開發 - Scrum
敏捷開發 - Scrum
Ajen 陳
 
第一次聊天就麻吉
第一次聊天就麻吉第一次聊天就麻吉
第一次聊天就麻吉
Ajen 陳
 
20120430小組分享-來一場自助旅行吧
20120430小組分享-來一場自助旅行吧20120430小組分享-來一場自助旅行吧
20120430小組分享-來一場自助旅行吧
Ajen 陳
 
20120325 小組分享
20120325 小組分享20120325 小組分享
20120325 小組分享
Ajen 陳
 

Mehr von Ajen 陳 (6)

Mobile Web vs Native App
Mobile Web vs Native AppMobile Web vs Native App
Mobile Web vs Native App
 
敏捷開發 - Scrum
敏捷開發 - Scrum敏捷開發 - Scrum
敏捷開發 - Scrum
 
第一次聊天就麻吉
第一次聊天就麻吉第一次聊天就麻吉
第一次聊天就麻吉
 
What is MDM?
What is MDM?What is MDM?
What is MDM?
 
20120430小組分享-來一場自助旅行吧
20120430小組分享-來一場自助旅行吧20120430小組分享-來一場自助旅行吧
20120430小組分享-來一場自助旅行吧
 
20120325 小組分享
20120325 小組分享20120325 小組分享
20120325 小組分享
 

Kürzlich hochgeladen

CNv6 Instructor Chapter 6 Quality of Service
CNv6 Instructor Chapter 6 Quality of ServiceCNv6 Instructor Chapter 6 Quality of Service
CNv6 Instructor Chapter 6 Quality of Service
giselly40
 
Histor y of HAM Radio presentation slide
Histor y of HAM Radio presentation slideHistor y of HAM Radio presentation slide
Histor y of HAM Radio presentation slide
vu2urc
 

Kürzlich hochgeladen (20)

Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...
Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...
Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...
 
08448380779 Call Girls In Civil Lines Women Seeking Men
08448380779 Call Girls In Civil Lines Women Seeking Men08448380779 Call Girls In Civil Lines Women Seeking Men
08448380779 Call Girls In Civil Lines Women Seeking Men
 
A Domino Admins Adventures (Engage 2024)
A Domino Admins Adventures (Engage 2024)A Domino Admins Adventures (Engage 2024)
A Domino Admins Adventures (Engage 2024)
 
04-2024-HHUG-Sales-and-Marketing-Alignment.pptx
04-2024-HHUG-Sales-and-Marketing-Alignment.pptx04-2024-HHUG-Sales-and-Marketing-Alignment.pptx
04-2024-HHUG-Sales-and-Marketing-Alignment.pptx
 
🐬 The future of MySQL is Postgres 🐘
🐬  The future of MySQL is Postgres   🐘🐬  The future of MySQL is Postgres   🐘
🐬 The future of MySQL is Postgres 🐘
 
Factors to Consider When Choosing Accounts Payable Services Providers.pptx
Factors to Consider When Choosing Accounts Payable Services Providers.pptxFactors to Consider When Choosing Accounts Payable Services Providers.pptx
Factors to Consider When Choosing Accounts Payable Services Providers.pptx
 
Driving Behavioral Change for Information Management through Data-Driven Gree...
Driving Behavioral Change for Information Management through Data-Driven Gree...Driving Behavioral Change for Information Management through Data-Driven Gree...
Driving Behavioral Change for Information Management through Data-Driven Gree...
 
Data Cloud, More than a CDP by Matt Robison
Data Cloud, More than a CDP by Matt RobisonData Cloud, More than a CDP by Matt Robison
Data Cloud, More than a CDP by Matt Robison
 
Breaking the Kubernetes Kill Chain: Host Path Mount
Breaking the Kubernetes Kill Chain: Host Path MountBreaking the Kubernetes Kill Chain: Host Path Mount
Breaking the Kubernetes Kill Chain: Host Path Mount
 
Presentation on how to chat with PDF using ChatGPT code interpreter
Presentation on how to chat with PDF using ChatGPT code interpreterPresentation on how to chat with PDF using ChatGPT code interpreter
Presentation on how to chat with PDF using ChatGPT code interpreter
 
2024: Domino Containers - The Next Step. News from the Domino Container commu...
2024: Domino Containers - The Next Step. News from the Domino Container commu...2024: Domino Containers - The Next Step. News from the Domino Container commu...
2024: Domino Containers - The Next Step. News from the Domino Container commu...
 
Axa Assurance Maroc - Insurer Innovation Award 2024
Axa Assurance Maroc - Insurer Innovation Award 2024Axa Assurance Maroc - Insurer Innovation Award 2024
Axa Assurance Maroc - Insurer Innovation Award 2024
 
CNv6 Instructor Chapter 6 Quality of Service
CNv6 Instructor Chapter 6 Quality of ServiceCNv6 Instructor Chapter 6 Quality of Service
CNv6 Instructor Chapter 6 Quality of Service
 
Advantages of Hiring UIUX Design Service Providers for Your Business
Advantages of Hiring UIUX Design Service Providers for Your BusinessAdvantages of Hiring UIUX Design Service Providers for Your Business
Advantages of Hiring UIUX Design Service Providers for Your Business
 
Histor y of HAM Radio presentation slide
Histor y of HAM Radio presentation slideHistor y of HAM Radio presentation slide
Histor y of HAM Radio presentation slide
 
Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...
Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...
Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...
 
Slack Application Development 101 Slides
Slack Application Development 101 SlidesSlack Application Development 101 Slides
Slack Application Development 101 Slides
 
[2024]Digital Global Overview Report 2024 Meltwater.pdf
[2024]Digital Global Overview Report 2024 Meltwater.pdf[2024]Digital Global Overview Report 2024 Meltwater.pdf
[2024]Digital Global Overview Report 2024 Meltwater.pdf
 
Scaling API-first – The story of a global engineering organization
Scaling API-first – The story of a global engineering organizationScaling API-first – The story of a global engineering organization
Scaling API-first – The story of a global engineering organization
 
Boost Fertility New Invention Ups Success Rates.pdf
Boost Fertility New Invention Ups Success Rates.pdfBoost Fertility New Invention Ups Success Rates.pdf
Boost Fertility New Invention Ups Success Rates.pdf
 

What's behind facebook

  • 2. • font-end o HipHop o BigPipe • back-end outline o Thrift o Scribe o Tornado • data persistence • other
  • 3. font-end Linux & Apache PHP + HipHop BigPipe
  • 4. HipHop • PHP complier developed by Facebook. • Convert PHP code to C ++. • Compiled by gcc. • CPU usage recudes 50%. • Git Hub
  • 5. BigPipe • It’s a fundamental redesign of the dynamic web. page serving system. • Decomposing web pages into 「pagelets」. • Implemented by ONLY PHP + JavaScript! • Link
  • 6. back-end Thrift Scribe Tornado
  • 7. Thrift • Cross-language services development. • Code generation engine. • Sample Code definition file thrift complier source code
  • 8. Scribe • Distributed Loggin System. • git hub
  • 9. Tornado • Web Server written by python. • For real-time web services • non-blocking web service. • git hub • show case • link
  • 10. data persistence MySQL Hive + Hadoop + Hbase + HDFS Memcache Haystack
  • 11. MySQL • It’s still backbon of database infra. • How ? o Learn more : link o facebook patches
  • 12. Hive + hadoop + Hbase + HDFS Hive hadoop Hbase HDFS
  • 13. Memcache • Distributed memory object caching system • MySQL Cache • General purpose cache • Link
  • 14. Haystack • Efficient storage of billions of photos. • Reducing Disk I/O. • Components o Haystack Diretory o Haystack Store o Haystack Cache • Link
  • 15. Other • Chat o Epoll Server using Erlang o Accessed using thrift • Message Search o Inverted index stored in HBase
  • 16. Thank you A Jen