SlideShare ist ein Scribd-Unternehmen logo
1 von 35
Don’t block the event loop!
    EventMachine: Scalable non-blocking i/o in ruby
Jinpu Hu (胡金埔)

Blog: http://hujinpu.com/
Github: https://github.com/hujinpu
Twitter: https://twitter.com/#!/hujinpu
Sina weibo: http://weibo.com/hujinpu
Email: hujinpu@gmail.com
Blocking IO sucks
Evented IO rules
CXXXK problem
CXXXK problem


RPS(QPS/TPS)

TPR
Concurrency Level
Efficient Processing


 Focus on maximizing machine utilization
 Google tries for ~80% utilization
Different


 non blocking i/o
 asynchronous programming
• Blocking IO a UNIVERSAL problem
• All programming languages have trouble with it
• The Reactor pattern resolves it
• Most languages have the Reactor pattern!
      JavaScript          Node.js

       Python             Twisted

        Ruby           EventMachine

         Java             MINA

         Lua             LuaNode
Reactor Pattern


What’s reactor?
What’s design patterns?
So, what’s reactor pattern?
reactor


 reactor is simply a single thread, endless loop
 It reacts to incoming events
design patterns



http://en.wikipedia.org/wiki/Pattern
reactor pattern

 resources (description/file handler)
 synchronous event demultiplexer (select/epoll/kqueue)
 dispatcher
 request handler
What’s eventmachine?


Ruby implementation of the Reactor pattern
Single threaded by default
Allows us to interleave multiple IO ops and a single
CPU op simultaneously
Let’s play with EM



 gem install eventmachine --pre
EM.run


takes a block to execute once the reactor has started
EM.run is blocking
the loop takes over the ruby process, so code after
EM.run will not run
all your code(relative to em) must in EM.run
EM.next_tick




queue a proc to be executed on the next iteration of
the reactor loop
EM.tick_loop


                  80k+/s
               on my mac air
EM::Iterator




 That’s cool for writing crawler
EM.defer

on first invocation, spawns up EM.threadpool_size
threads in a pool
all the threads read procs from a queue and execute
them in parallel to the reactor thread
optional second proc is invoked with the result of the
first proc, but back inside the reactor thread
EM::Timer
EM::Queue




async queue
if queue is empty, it is invoked when someone calls
#push
EM::Channel
EM.system
Network Servers and Clients


EM.start_server / EM.stop_server
EM.connect / EM.bind_connect
EM.open_datagram_socket
TCP is stream
And more



read eventmachine source code!
Callback Hell




var   useFile = function(filename,callback){
      posix.stat(filename).addCallback(function (stats) {
          posix.open(filename, process.O_RDONLY, 0666).addCallback(function (fd) {
              posix.read(fd, stats.size, 0).addCallback(function(contents){
                  callback(contents);
              });
          });
      });
};
Fiber


 What’s fiber?
 How to use fiber?
what’s fiber

 light weight cooperative concurrency
 the scheduling must be done by the
 programmer and not the VM
 a small 4KB stack
use fiber




 http://ruby-doc.org/core-1.9.3/Fiber.html
Solotion to callback hell


 Fiber into EM = em-synchrony
 thanks to Ilya Grigorik(CTO of PostRank)
The End

Weitere ähnliche Inhalte

Was ist angesagt?

Rails Hardware (no conclusions!)
Rails Hardware (no conclusions!)Rails Hardware (no conclusions!)
Rails Hardware (no conclusions!)
yarry
 
Eduardo Silva - monkey http-server everywhere
Eduardo Silva - monkey http-server everywhereEduardo Silva - monkey http-server everywhere
Eduardo Silva - monkey http-server everywhere
StarTech Conference
 

Was ist angesagt? (20)

Rails Hardware (no conclusions!)
Rails Hardware (no conclusions!)Rails Hardware (no conclusions!)
Rails Hardware (no conclusions!)
 
RuntimeUnitTestToolkit for Unity(English)
RuntimeUnitTestToolkit for Unity(English)RuntimeUnitTestToolkit for Unity(English)
RuntimeUnitTestToolkit for Unity(English)
 
Node day 2014
Node day 2014Node day 2014
Node day 2014
 
XebiCon'16 : Server-Side Swift. Par Simone Civetta, Développeur iOS chez Xebia
XebiCon'16 : Server-Side Swift. Par Simone Civetta, Développeur iOS chez XebiaXebiCon'16 : Server-Side Swift. Par Simone Civetta, Développeur iOS chez Xebia
XebiCon'16 : Server-Side Swift. Par Simone Civetta, Développeur iOS chez Xebia
 
Puppet Camp Düsseldorf 2014: Continuously Deliver Your Puppet Code with Jenki...
Puppet Camp Düsseldorf 2014: Continuously Deliver Your Puppet Code with Jenki...Puppet Camp Düsseldorf 2014: Continuously Deliver Your Puppet Code with Jenki...
Puppet Camp Düsseldorf 2014: Continuously Deliver Your Puppet Code with Jenki...
 
Concurrency in Python
Concurrency in PythonConcurrency in Python
Concurrency in Python
 
nouka inventry manager
nouka inventry managernouka inventry manager
nouka inventry manager
 
objection - runtime mobile exploration
objection - runtime mobile explorationobjection - runtime mobile exploration
objection - runtime mobile exploration
 
Nine Circles of Inferno or Explaining the PostgreSQL Vacuum
Nine Circles of Inferno or Explaining the PostgreSQL VacuumNine Circles of Inferno or Explaining the PostgreSQL Vacuum
Nine Circles of Inferno or Explaining the PostgreSQL Vacuum
 
Puppet and Openshift
Puppet and OpenshiftPuppet and Openshift
Puppet and Openshift
 
DevOps(2) : Vagrant - (MOSG)
DevOps(2) : Vagrant  -  (MOSG)DevOps(2) : Vagrant  -  (MOSG)
DevOps(2) : Vagrant - (MOSG)
 
T.Pollak y C.Yaconi - Prey
T.Pollak y C.Yaconi - PreyT.Pollak y C.Yaconi - Prey
T.Pollak y C.Yaconi - Prey
 
Ищем уязвимости нулевого дня в ядре Linux
Ищем уязвимости нулевого дня в ядре LinuxИщем уязвимости нулевого дня в ядре Linux
Ищем уязвимости нулевого дня в ядре Linux
 
Puppet User Group Presentation - 15 March 2012
Puppet User Group Presentation - 15 March 2012Puppet User Group Presentation - 15 March 2012
Puppet User Group Presentation - 15 March 2012
 
Масштабируемая конфигурация Nginx, Игорь Сысоев (Nginx)
Масштабируемая конфигурация Nginx, Игорь Сысоев (Nginx)Масштабируемая конфигурация Nginx, Игорь Сысоев (Nginx)
Масштабируемая конфигурация Nginx, Игорь Сысоев (Nginx)
 
How to make a large C++-code base manageable
How to make a large C++-code base manageableHow to make a large C++-code base manageable
How to make a large C++-code base manageable
 
Profiling and optimizing go programs
Profiling and optimizing go programsProfiling and optimizing go programs
Profiling and optimizing go programs
 
Eduardo Silva - monkey http-server everywhere
Eduardo Silva - monkey http-server everywhereEduardo Silva - monkey http-server everywhere
Eduardo Silva - monkey http-server everywhere
 
Spark Day 2017- Spark 의 과거, 현재, 미래
Spark Day 2017- Spark 의 과거, 현재, 미래Spark Day 2017- Spark 의 과거, 현재, 미래
Spark Day 2017- Spark 의 과거, 현재, 미래
 
Coding in the context era
Coding in the context eraCoding in the context era
Coding in the context era
 

Ähnlich wie Don’t block the event loop!

Node js presentation
Node js presentationNode js presentation
Node js presentation
martincabrera
 
Node.js for Rubists
Node.js for RubistsNode.js for Rubists
Node.js for Rubists
Sagiv Ofek
 

Ähnlich wie Don’t block the event loop! (20)

Concurrency in ruby
Concurrency in rubyConcurrency in ruby
Concurrency in ruby
 
Node js presentation
Node js presentationNode js presentation
Node js presentation
 
Node.js for Rubists
Node.js for RubistsNode.js for Rubists
Node.js for Rubists
 
No Callbacks, No Threads - RailsConf 2010
No Callbacks, No Threads - RailsConf 2010No Callbacks, No Threads - RailsConf 2010
No Callbacks, No Threads - RailsConf 2010
 
No callbacks, No Threads - Cooperative web servers in Ruby 1.9
No callbacks, No Threads - Cooperative web servers in Ruby 1.9No callbacks, No Threads - Cooperative web servers in Ruby 1.9
No callbacks, No Threads - Cooperative web servers in Ruby 1.9
 
32 OpenMP Traps For C++ Developers
32 OpenMP Traps For C++ Developers32 OpenMP Traps For C++ Developers
32 OpenMP Traps For C++ Developers
 
Servers with Event Machine - David Troy - RailsConf 2011
Servers with Event Machine - David Troy - RailsConf 2011Servers with Event Machine - David Troy - RailsConf 2011
Servers with Event Machine - David Troy - RailsConf 2011
 
Asynchronous programming - .NET Way
Asynchronous programming - .NET WayAsynchronous programming - .NET Way
Asynchronous programming - .NET Way
 
Node.js
Node.jsNode.js
Node.js
 
SFO15-202: Towards Multi-Threaded Tiny Code Generator (TCG) in QEMU
SFO15-202: Towards Multi-Threaded Tiny Code Generator (TCG) in QEMUSFO15-202: Towards Multi-Threaded Tiny Code Generator (TCG) in QEMU
SFO15-202: Towards Multi-Threaded Tiny Code Generator (TCG) in QEMU
 
Node.js: CAMTA Presentation
Node.js: CAMTA PresentationNode.js: CAMTA Presentation
Node.js: CAMTA Presentation
 
Achieving mass scale with Quasar Fibers
Achieving mass scale with Quasar FibersAchieving mass scale with Quasar Fibers
Achieving mass scale with Quasar Fibers
 
Async/Await: TPL & Message Pumps
Async/Await: TPL & Message Pumps Async/Await: TPL & Message Pumps
Async/Await: TPL & Message Pumps
 
Introduction to OpenMP
Introduction to OpenMPIntroduction to OpenMP
Introduction to OpenMP
 
Gabriele Santomaggio - Inside Elixir/Erlang - Codemotion Milan 2018
Gabriele Santomaggio - Inside Elixir/Erlang - Codemotion Milan 2018Gabriele Santomaggio - Inside Elixir/Erlang - Codemotion Milan 2018
Gabriele Santomaggio - Inside Elixir/Erlang - Codemotion Milan 2018
 
Beyond the RTOS: A Better Way to Design Real-Time Embedded Software
Beyond the RTOS: A Better Way to Design Real-Time Embedded SoftwareBeyond the RTOS: A Better Way to Design Real-Time Embedded Software
Beyond the RTOS: A Better Way to Design Real-Time Embedded Software
 
From Generator to Fiber the Road to Coroutine in PHP
From Generator to Fiber the Road to Coroutine in PHPFrom Generator to Fiber the Road to Coroutine in PHP
From Generator to Fiber the Road to Coroutine in PHP
 
Bugs from Outer Space | while42 SF #6
Bugs from Outer Space | while42 SF #6Bugs from Outer Space | while42 SF #6
Bugs from Outer Space | while42 SF #6
 
Java multi thread programming on cmp system
Java multi thread programming on cmp systemJava multi thread programming on cmp system
Java multi thread programming on cmp system
 
Here comes the Loom - Ya!vaConf.pdf
Here comes the Loom - Ya!vaConf.pdfHere comes the Loom - Ya!vaConf.pdf
Here comes the Loom - Ya!vaConf.pdf
 

Kürzlich hochgeladen

Artificial Intelligence: Facts and Myths
Artificial Intelligence: Facts and MythsArtificial Intelligence: Facts and Myths
Artificial Intelligence: Facts and Myths
Joaquim Jorge
 

Kürzlich hochgeladen (20)

04-2024-HHUG-Sales-and-Marketing-Alignment.pptx
04-2024-HHUG-Sales-and-Marketing-Alignment.pptx04-2024-HHUG-Sales-and-Marketing-Alignment.pptx
04-2024-HHUG-Sales-and-Marketing-Alignment.pptx
 
GenCyber Cyber Security Day Presentation
GenCyber Cyber Security Day PresentationGenCyber Cyber Security Day Presentation
GenCyber Cyber Security Day Presentation
 
A Domino Admins Adventures (Engage 2024)
A Domino Admins Adventures (Engage 2024)A Domino Admins Adventures (Engage 2024)
A Domino Admins Adventures (Engage 2024)
 
Driving Behavioral Change for Information Management through Data-Driven Gree...
Driving Behavioral Change for Information Management through Data-Driven Gree...Driving Behavioral Change for Information Management through Data-Driven Gree...
Driving Behavioral Change for Information Management through Data-Driven Gree...
 
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
 
A Year of the Servo Reboot: Where Are We Now?
A Year of the Servo Reboot: Where Are We Now?A Year of the Servo Reboot: Where Are We Now?
A Year of the Servo Reboot: Where Are We Now?
 
How to Troubleshoot Apps for the Modern Connected Worker
How to Troubleshoot Apps for the Modern Connected WorkerHow to Troubleshoot Apps for the Modern Connected Worker
How to Troubleshoot Apps for the Modern Connected Worker
 
Finology Group – Insurtech Innovation Award 2024
Finology Group – Insurtech Innovation Award 2024Finology Group – Insurtech Innovation Award 2024
Finology Group – Insurtech Innovation Award 2024
 
ProductAnonymous-April2024-WinProductDiscovery-MelissaKlemke
ProductAnonymous-April2024-WinProductDiscovery-MelissaKlemkeProductAnonymous-April2024-WinProductDiscovery-MelissaKlemke
ProductAnonymous-April2024-WinProductDiscovery-MelissaKlemke
 
Axa Assurance Maroc - Insurer Innovation Award 2024
Axa Assurance Maroc - Insurer Innovation Award 2024Axa Assurance Maroc - Insurer Innovation Award 2024
Axa Assurance Maroc - Insurer Innovation Award 2024
 
Advantages of Hiring UIUX Design Service Providers for Your Business
Advantages of Hiring UIUX Design Service Providers for Your BusinessAdvantages of Hiring UIUX Design Service Providers for Your Business
Advantages of Hiring UIUX Design Service Providers for Your Business
 
GenAI Risks & Security Meetup 01052024.pdf
GenAI Risks & Security Meetup 01052024.pdfGenAI Risks & Security Meetup 01052024.pdf
GenAI Risks & Security Meetup 01052024.pdf
 
Scaling API-first – The story of a global engineering organization
Scaling API-first – The story of a global engineering organizationScaling API-first – The story of a global engineering organization
Scaling API-first – The story of a global engineering organization
 
HTML Injection Attacks: Impact and Mitigation Strategies
HTML Injection Attacks: Impact and Mitigation StrategiesHTML Injection Attacks: Impact and Mitigation Strategies
HTML Injection Attacks: Impact and Mitigation Strategies
 
What Are The Drone Anti-jamming Systems Technology?
What Are The Drone Anti-jamming Systems Technology?What Are The Drone Anti-jamming Systems Technology?
What Are The Drone Anti-jamming Systems Technology?
 
2024: Domino Containers - The Next Step. News from the Domino Container commu...
2024: Domino Containers - The Next Step. News from the Domino Container commu...2024: Domino Containers - The Next Step. News from the Domino Container commu...
2024: Domino Containers - The Next Step. News from the Domino Container commu...
 
Automating Google Workspace (GWS) & more with Apps Script
Automating Google Workspace (GWS) & more with Apps ScriptAutomating Google Workspace (GWS) & more with Apps Script
Automating Google Workspace (GWS) & more with Apps Script
 
TrustArc Webinar - Unlock the Power of AI-Driven Data Discovery
TrustArc Webinar - Unlock the Power of AI-Driven Data DiscoveryTrustArc Webinar - Unlock the Power of AI-Driven Data Discovery
TrustArc Webinar - Unlock the Power of AI-Driven Data Discovery
 
Artificial Intelligence: Facts and Myths
Artificial Intelligence: Facts and MythsArtificial Intelligence: Facts and Myths
Artificial Intelligence: Facts and Myths
 
Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...
Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...
Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...
 

Don’t block the event loop!

Hinweis der Redaktion

  1. \n
  2. \n
  3. \n
  4. \n
  5. \n
  6. \n
  7. \n
  8. \n
  9. \n
  10. \n
  11. \n
  12. \n
  13. \n
  14. \n
  15. \n
  16. \n
  17. \n
  18. \n
  19. \n
  20. \n
  21. \n
  22. \n
  23. \n
  24. \n
  25. \n
  26. \n
  27. \n
  28. \n
  29. \n
  30. \n
  31. \n
  32. \n