SlideShare ist ein Scribd-Unternehmen logo
1 von 35
Don’t block the event loop!
    EventMachine: Scalable non-blocking i/o in ruby
Jinpu Hu (胡金埔)

Blog: http://hujinpu.com/
Github: https://github.com/hujinpu
Twitter: https://twitter.com/#!/hujinpu
Sina weibo: http://weibo.com/hujinpu
Email: hujinpu@gmail.com
Blocking IO sucks
Evented IO rules
CXXXK problem
CXXXK problem


RPS(QPS/TPS)

TPR
Concurrency Level
Efficient Processing


 Focus on maximizing machine utilization
 Google tries for ~80% utilization
Different


 non blocking i/o
 asynchronous programming
• Blocking IO a UNIVERSAL problem
• All programming languages have trouble with it
• The Reactor pattern resolves it
• Most languages have the Reactor pattern!
      JavaScript          Node.js

       Python             Twisted

        Ruby           EventMachine

         Java             MINA

         Lua             LuaNode
Reactor Pattern


What’s reactor?
What’s design patterns?
So, what’s reactor pattern?
reactor


 reactor is simply a single thread, endless loop
 It reacts to incoming events
design patterns



http://en.wikipedia.org/wiki/Pattern
reactor pattern

 resources (description/file handler)
 synchronous event demultiplexer (select/epoll/kqueue)
 dispatcher
 request handler
What’s eventmachine?


Ruby implementation of the Reactor pattern
Single threaded by default
Allows us to interleave multiple IO ops and a single
CPU op simultaneously
Let’s play with EM



 gem install eventmachine --pre
EM.run


takes a block to execute once the reactor has started
EM.run is blocking
the loop takes over the ruby process, so code after
EM.run will not run
all your code(relative to em) must in EM.run
EM.next_tick




queue a proc to be executed on the next iteration of
the reactor loop
EM.tick_loop


                  80k+/s
               on my mac air
EM::Iterator




 That’s cool for writing crawler
EM.defer

on first invocation, spawns up EM.threadpool_size
threads in a pool
all the threads read procs from a queue and execute
them in parallel to the reactor thread
optional second proc is invoked with the result of the
first proc, but back inside the reactor thread
EM::Timer
EM::Queue




async queue
if queue is empty, it is invoked when someone calls
#push
EM::Channel
EM.system
Network Servers and Clients


EM.start_server / EM.stop_server
EM.connect / EM.bind_connect
EM.open_datagram_socket
TCP is stream
And more



read eventmachine source code!
Callback Hell




var   useFile = function(filename,callback){
      posix.stat(filename).addCallback(function (stats) {
          posix.open(filename, process.O_RDONLY, 0666).addCallback(function (fd) {
              posix.read(fd, stats.size, 0).addCallback(function(contents){
                  callback(contents);
              });
          });
      });
};
Fiber


 What’s fiber?
 How to use fiber?
what’s fiber

 light weight cooperative concurrency
 the scheduling must be done by the
 programmer and not the VM
 a small 4KB stack
use fiber




 http://ruby-doc.org/core-1.9.3/Fiber.html
Solotion to callback hell


 Fiber into EM = em-synchrony
 thanks to Ilya Grigorik(CTO of PostRank)
The End

Weitere ähnliche Inhalte

Was ist angesagt?

Rails Hardware (no conclusions!)
Rails Hardware (no conclusions!)Rails Hardware (no conclusions!)
Rails Hardware (no conclusions!)
yarry
 
Eduardo Silva - monkey http-server everywhere
Eduardo Silva - monkey http-server everywhereEduardo Silva - monkey http-server everywhere
Eduardo Silva - monkey http-server everywhere
StarTech Conference
 

Was ist angesagt? (20)

Rails Hardware (no conclusions!)
Rails Hardware (no conclusions!)Rails Hardware (no conclusions!)
Rails Hardware (no conclusions!)
 
RuntimeUnitTestToolkit for Unity(English)
RuntimeUnitTestToolkit for Unity(English)RuntimeUnitTestToolkit for Unity(English)
RuntimeUnitTestToolkit for Unity(English)
 
Node day 2014
Node day 2014Node day 2014
Node day 2014
 
XebiCon'16 : Server-Side Swift. Par Simone Civetta, Développeur iOS chez Xebia
XebiCon'16 : Server-Side Swift. Par Simone Civetta, Développeur iOS chez XebiaXebiCon'16 : Server-Side Swift. Par Simone Civetta, Développeur iOS chez Xebia
XebiCon'16 : Server-Side Swift. Par Simone Civetta, Développeur iOS chez Xebia
 
Puppet Camp Düsseldorf 2014: Continuously Deliver Your Puppet Code with Jenki...
Puppet Camp Düsseldorf 2014: Continuously Deliver Your Puppet Code with Jenki...Puppet Camp Düsseldorf 2014: Continuously Deliver Your Puppet Code with Jenki...
Puppet Camp Düsseldorf 2014: Continuously Deliver Your Puppet Code with Jenki...
 
Concurrency in Python
Concurrency in PythonConcurrency in Python
Concurrency in Python
 
nouka inventry manager
nouka inventry managernouka inventry manager
nouka inventry manager
 
objection - runtime mobile exploration
objection - runtime mobile explorationobjection - runtime mobile exploration
objection - runtime mobile exploration
 
Nine Circles of Inferno or Explaining the PostgreSQL Vacuum
Nine Circles of Inferno or Explaining the PostgreSQL VacuumNine Circles of Inferno or Explaining the PostgreSQL Vacuum
Nine Circles of Inferno or Explaining the PostgreSQL Vacuum
 
Puppet and Openshift
Puppet and OpenshiftPuppet and Openshift
Puppet and Openshift
 
DevOps(2) : Vagrant - (MOSG)
DevOps(2) : Vagrant  -  (MOSG)DevOps(2) : Vagrant  -  (MOSG)
DevOps(2) : Vagrant - (MOSG)
 
T.Pollak y C.Yaconi - Prey
T.Pollak y C.Yaconi - PreyT.Pollak y C.Yaconi - Prey
T.Pollak y C.Yaconi - Prey
 
Ищем уязвимости нулевого дня в ядре Linux
Ищем уязвимости нулевого дня в ядре LinuxИщем уязвимости нулевого дня в ядре Linux
Ищем уязвимости нулевого дня в ядре Linux
 
Puppet User Group Presentation - 15 March 2012
Puppet User Group Presentation - 15 March 2012Puppet User Group Presentation - 15 March 2012
Puppet User Group Presentation - 15 March 2012
 
Масштабируемая конфигурация Nginx, Игорь Сысоев (Nginx)
Масштабируемая конфигурация Nginx, Игорь Сысоев (Nginx)Масштабируемая конфигурация Nginx, Игорь Сысоев (Nginx)
Масштабируемая конфигурация Nginx, Игорь Сысоев (Nginx)
 
How to make a large C++-code base manageable
How to make a large C++-code base manageableHow to make a large C++-code base manageable
How to make a large C++-code base manageable
 
Profiling and optimizing go programs
Profiling and optimizing go programsProfiling and optimizing go programs
Profiling and optimizing go programs
 
Eduardo Silva - monkey http-server everywhere
Eduardo Silva - monkey http-server everywhereEduardo Silva - monkey http-server everywhere
Eduardo Silva - monkey http-server everywhere
 
Spark Day 2017- Spark 의 과거, 현재, 미래
Spark Day 2017- Spark 의 과거, 현재, 미래Spark Day 2017- Spark 의 과거, 현재, 미래
Spark Day 2017- Spark 의 과거, 현재, 미래
 
Coding in the context era
Coding in the context eraCoding in the context era
Coding in the context era
 

Ähnlich wie Don’t block the event loop!

Node js presentation
Node js presentationNode js presentation
Node js presentation
martincabrera
 
Node.js for Rubists
Node.js for RubistsNode.js for Rubists
Node.js for Rubists
Sagiv Ofek
 

Ähnlich wie Don’t block the event loop! (20)

Concurrency in ruby
Concurrency in rubyConcurrency in ruby
Concurrency in ruby
 
Node js presentation
Node js presentationNode js presentation
Node js presentation
 
Node.js for Rubists
Node.js for RubistsNode.js for Rubists
Node.js for Rubists
 
No Callbacks, No Threads - RailsConf 2010
No Callbacks, No Threads - RailsConf 2010No Callbacks, No Threads - RailsConf 2010
No Callbacks, No Threads - RailsConf 2010
 
No callbacks, No Threads - Cooperative web servers in Ruby 1.9
No callbacks, No Threads - Cooperative web servers in Ruby 1.9No callbacks, No Threads - Cooperative web servers in Ruby 1.9
No callbacks, No Threads - Cooperative web servers in Ruby 1.9
 
32 OpenMP Traps For C++ Developers
32 OpenMP Traps For C++ Developers32 OpenMP Traps For C++ Developers
32 OpenMP Traps For C++ Developers
 
Servers with Event Machine - David Troy - RailsConf 2011
Servers with Event Machine - David Troy - RailsConf 2011Servers with Event Machine - David Troy - RailsConf 2011
Servers with Event Machine - David Troy - RailsConf 2011
 
Asynchronous programming - .NET Way
Asynchronous programming - .NET WayAsynchronous programming - .NET Way
Asynchronous programming - .NET Way
 
Node.js
Node.jsNode.js
Node.js
 
SFO15-202: Towards Multi-Threaded Tiny Code Generator (TCG) in QEMU
SFO15-202: Towards Multi-Threaded Tiny Code Generator (TCG) in QEMUSFO15-202: Towards Multi-Threaded Tiny Code Generator (TCG) in QEMU
SFO15-202: Towards Multi-Threaded Tiny Code Generator (TCG) in QEMU
 
Node.js: CAMTA Presentation
Node.js: CAMTA PresentationNode.js: CAMTA Presentation
Node.js: CAMTA Presentation
 
Achieving mass scale with Quasar Fibers
Achieving mass scale with Quasar FibersAchieving mass scale with Quasar Fibers
Achieving mass scale with Quasar Fibers
 
Async/Await: TPL & Message Pumps
Async/Await: TPL & Message Pumps Async/Await: TPL & Message Pumps
Async/Await: TPL & Message Pumps
 
Introduction to OpenMP
Introduction to OpenMPIntroduction to OpenMP
Introduction to OpenMP
 
Gabriele Santomaggio - Inside Elixir/Erlang - Codemotion Milan 2018
Gabriele Santomaggio - Inside Elixir/Erlang - Codemotion Milan 2018Gabriele Santomaggio - Inside Elixir/Erlang - Codemotion Milan 2018
Gabriele Santomaggio - Inside Elixir/Erlang - Codemotion Milan 2018
 
Beyond the RTOS: A Better Way to Design Real-Time Embedded Software
Beyond the RTOS: A Better Way to Design Real-Time Embedded SoftwareBeyond the RTOS: A Better Way to Design Real-Time Embedded Software
Beyond the RTOS: A Better Way to Design Real-Time Embedded Software
 
From Generator to Fiber the Road to Coroutine in PHP
From Generator to Fiber the Road to Coroutine in PHPFrom Generator to Fiber the Road to Coroutine in PHP
From Generator to Fiber the Road to Coroutine in PHP
 
Bugs from Outer Space | while42 SF #6
Bugs from Outer Space | while42 SF #6Bugs from Outer Space | while42 SF #6
Bugs from Outer Space | while42 SF #6
 
Java multi thread programming on cmp system
Java multi thread programming on cmp systemJava multi thread programming on cmp system
Java multi thread programming on cmp system
 
Here comes the Loom - Ya!vaConf.pdf
Here comes the Loom - Ya!vaConf.pdfHere comes the Loom - Ya!vaConf.pdf
Here comes the Loom - Ya!vaConf.pdf
 

Kürzlich hochgeladen

Finding Java's Hidden Performance Traps @ DevoxxUK 2024
Finding Java's Hidden Performance Traps @ DevoxxUK 2024Finding Java's Hidden Performance Traps @ DevoxxUK 2024
Finding Java's Hidden Performance Traps @ DevoxxUK 2024
Victor Rentea
 
+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...
+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...
+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...
?#DUbAI#??##{{(☎️+971_581248768%)**%*]'#abortion pills for sale in dubai@
 
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers:  A Deep Dive into Serverless Spatial Data and FMECloud Frontiers:  A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Safe Software
 
Architecting Cloud Native Applications
Architecting Cloud Native ApplicationsArchitecting Cloud Native Applications
Architecting Cloud Native Applications
WSO2
 

Kürzlich hochgeladen (20)

Understanding the FAA Part 107 License ..
Understanding the FAA Part 107 License ..Understanding the FAA Part 107 License ..
Understanding the FAA Part 107 License ..
 
DBX First Quarter 2024 Investor Presentation
DBX First Quarter 2024 Investor PresentationDBX First Quarter 2024 Investor Presentation
DBX First Quarter 2024 Investor Presentation
 
EMPOWERMENT TECHNOLOGY GRADE 11 QUARTER 2 REVIEWER
EMPOWERMENT TECHNOLOGY GRADE 11 QUARTER 2 REVIEWEREMPOWERMENT TECHNOLOGY GRADE 11 QUARTER 2 REVIEWER
EMPOWERMENT TECHNOLOGY GRADE 11 QUARTER 2 REVIEWER
 
Elevate Developer Efficiency & build GenAI Application with Amazon Q​
Elevate Developer Efficiency & build GenAI Application with Amazon Q​Elevate Developer Efficiency & build GenAI Application with Amazon Q​
Elevate Developer Efficiency & build GenAI Application with Amazon Q​
 
Corporate and higher education May webinar.pptx
Corporate and higher education May webinar.pptxCorporate and higher education May webinar.pptx
Corporate and higher education May webinar.pptx
 
Finding Java's Hidden Performance Traps @ DevoxxUK 2024
Finding Java's Hidden Performance Traps @ DevoxxUK 2024Finding Java's Hidden Performance Traps @ DevoxxUK 2024
Finding Java's Hidden Performance Traps @ DevoxxUK 2024
 
[BuildWithAI] Introduction to Gemini.pdf
[BuildWithAI] Introduction to Gemini.pdf[BuildWithAI] Introduction to Gemini.pdf
[BuildWithAI] Introduction to Gemini.pdf
 
+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...
+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...
+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...
 
AWS Community Day CPH - Three problems of Terraform
AWS Community Day CPH - Three problems of TerraformAWS Community Day CPH - Three problems of Terraform
AWS Community Day CPH - Three problems of Terraform
 
FWD Group - Insurer Innovation Award 2024
FWD Group - Insurer Innovation Award 2024FWD Group - Insurer Innovation Award 2024
FWD Group - Insurer Innovation Award 2024
 
Introduction to Multilingual Retrieval Augmented Generation (RAG)
Introduction to Multilingual Retrieval Augmented Generation (RAG)Introduction to Multilingual Retrieval Augmented Generation (RAG)
Introduction to Multilingual Retrieval Augmented Generation (RAG)
 
presentation ICT roal in 21st century education
presentation ICT roal in 21st century educationpresentation ICT roal in 21st century education
presentation ICT roal in 21st century education
 
Navigating the Deluge_ Dubai Floods and the Resilience of Dubai International...
Navigating the Deluge_ Dubai Floods and the Resilience of Dubai International...Navigating the Deluge_ Dubai Floods and the Resilience of Dubai International...
Navigating the Deluge_ Dubai Floods and the Resilience of Dubai International...
 
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers:  A Deep Dive into Serverless Spatial Data and FMECloud Frontiers:  A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
 
ICT role in 21st century education and its challenges
ICT role in 21st century education and its challengesICT role in 21st century education and its challenges
ICT role in 21st century education and its challenges
 
Artificial Intelligence Chap.5 : Uncertainty
Artificial Intelligence Chap.5 : UncertaintyArtificial Intelligence Chap.5 : Uncertainty
Artificial Intelligence Chap.5 : Uncertainty
 
Architecting Cloud Native Applications
Architecting Cloud Native ApplicationsArchitecting Cloud Native Applications
Architecting Cloud Native Applications
 
Strategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff
Strategize a Smooth Tenant-to-tenant Migration and Copilot TakeoffStrategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff
Strategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff
 
Apidays New York 2024 - APIs in 2030: The Risk of Technological Sleepwalk by ...
Apidays New York 2024 - APIs in 2030: The Risk of Technological Sleepwalk by ...Apidays New York 2024 - APIs in 2030: The Risk of Technological Sleepwalk by ...
Apidays New York 2024 - APIs in 2030: The Risk of Technological Sleepwalk by ...
 
Connector Corner: Accelerate revenue generation using UiPath API-centric busi...
Connector Corner: Accelerate revenue generation using UiPath API-centric busi...Connector Corner: Accelerate revenue generation using UiPath API-centric busi...
Connector Corner: Accelerate revenue generation using UiPath API-centric busi...
 

Don’t block the event loop!

Hinweis der Redaktion

  1. \n
  2. \n
  3. \n
  4. \n
  5. \n
  6. \n
  7. \n
  8. \n
  9. \n
  10. \n
  11. \n
  12. \n
  13. \n
  14. \n
  15. \n
  16. \n
  17. \n
  18. \n
  19. \n
  20. \n
  21. \n
  22. \n
  23. \n
  24. \n
  25. \n
  26. \n
  27. \n
  28. \n
  29. \n
  30. \n
  31. \n
  32. \n