SlideShare ist ein Scribd-Unternehmen logo
1 von 61
Implementing distributed
     applications with


...and some other bad guys...
         Crippa Francesco
Scalability vs Complexity
Scalability vs Complexity

     what we want




                                      Complexity
1 Thread    2 Threads   2 Nodes   4 Nodes
Scalability vs Complexity

      reality




                                          Complexity
1 Thread        2 Threads   2 Nodes   4 Nodes
The Q in ØMQ
The Ø in ØMQ
The Ø in ØMQ

Zero Broker
The Ø in ØMQ

Zero Broker

Zero Latency (as close as possible...)
The Ø in ØMQ

Zero Broker

Zero Latency (as close as possible...)

Zero administration
The Ø in ØMQ

Zero Broker

Zero Latency (as close as possible...)

Zero administration

Zero cost
The Ø in ØMQ

Zero Broker

Zero Latency (as close as possible...)

Zero administration

Zero cost

Zero waste
Sockets
Sockets
Unicast transports (inproc, ipc, tcp)
Sockets
Unicast transports (inproc, ipc, tcp)

Multicast transports (pgm or epgm)
Sockets
Unicast transports (inproc, ipc, tcp)

Multicast transports (pgm or epgm)

connect() and bind() are independent
Sockets
Unicast transports (inproc, ipc, tcp)

Multicast transports (pgm or epgm)

connect() and bind() are independent

They are asynchronous (with queues)
Sockets
Unicast transports (inproc, ipc, tcp)

Multicast transports (pgm or epgm)

connect() and bind() are independent

They are asynchronous (with queues)

They express a certain "messaging pattern"
Sockets
Unicast transports (inproc, ipc, tcp)

Multicast transports (pgm or epgm)

connect() and bind() are independent

They are asynchronous (with queues)

They express a certain "messaging pattern"

They are not necessarily one-to-one
...and, of course...

Cross Platform (Linux, Windows, Mac, etc...)

Multiple Languages (c, c++, python, java,
ruby, erlang, php, perl, ada, c#, lua, scala,
objective-c, go, haskell, racket, cl, basic...)

OpenSource
if you have a laptop...



http://www.zeromq.org/

http://zguide.zeromq.org/
Basic Message Patterns
Server
Client
Demo
Basic Message Patterns
Server
Client
Demo
Publisher
                Subscriber
The PUB-SUB socket pair is asynchronous

when you use a SUB socket you must set a
subscription using zmq_setsockopt and
SUBSCRIBE

"slow joiner" symptom
Basic Message Patterns
Ventilator
Worker
Sink
Demo
Workers

Always synchronize the start of the batch

The ventilator's PUSH socket distributes
tasks to workers (load balancing)

The sink's PULL socket collects results from
workers evenly (fair-queuing)
Fair-Queuing
Basic Message Patterns
Send
messages
  back
Demo
Allowed Patterns

PUB and SUB         DEALER and DEALER

REQ and REP         ROUTER and ROUTER

REQ and ROUTER      PUSH and PULL

DEALER and REP      PAIR and PAIR

DEALER and ROUTER
Scalability
Scalability
Scalability
Scalability
A Publish-Subscribe Proxy
Built-in Devices


QUEUE (request-reply broker.)

FORWARDER (pub-sub proxy server)

STREAMER (like FORWARDER but for
pipeline flows)
Dynamic Scalability...
Dynamic Scalability...
              SCA   LOW
                  LAB
                       ILI
                  !!!!     TY
Dynamic Scalability...
ØMQ : Sockets = Python : C++

Weitere ähnliche Inhalte

Was ist angesagt?

Was ist angesagt? (20)

ZeroMQ with NodeJS
ZeroMQ with NodeJSZeroMQ with NodeJS
ZeroMQ with NodeJS
 
gRPC Design and Implementation
gRPC Design and ImplementationgRPC Design and Implementation
gRPC Design and Implementation
 
Service-mesh options with Linkerd, Consul, Istio and AWS AppMesh
Service-mesh options with Linkerd, Consul, Istio and AWS AppMeshService-mesh options with Linkerd, Consul, Istio and AWS AppMesh
Service-mesh options with Linkerd, Consul, Istio and AWS AppMesh
 
Do flink on web with flow - Dongwon Kim & Haemee park, SK Telecom)
Do flink on web with flow - Dongwon Kim & Haemee park, SK Telecom)Do flink on web with flow - Dongwon Kim & Haemee park, SK Telecom)
Do flink on web with flow - Dongwon Kim & Haemee park, SK Telecom)
 
Akka Actor presentation
Akka Actor presentationAkka Actor presentation
Akka Actor presentation
 
DevFest 2022 Taipei 使用 Standalone Component 來寫 Angular 吧!
DevFest 2022 Taipei 使用 Standalone Component 來寫 Angular 吧!DevFest 2022 Taipei 使用 Standalone Component 來寫 Angular 吧!
DevFest 2022 Taipei 使用 Standalone Component 來寫 Angular 吧!
 
Istio
Istio Istio
Istio
 
Introduction to Actor Model and Akka
Introduction to Actor Model and AkkaIntroduction to Actor Model and Akka
Introduction to Actor Model and Akka
 
Reactive Card Magic: Understanding Spring WebFlux and Project Reactor
Reactive Card Magic: Understanding Spring WebFlux and Project ReactorReactive Card Magic: Understanding Spring WebFlux and Project Reactor
Reactive Card Magic: Understanding Spring WebFlux and Project Reactor
 
Microservices with Spring 5 Webflux - jProfessionals
Microservices  with Spring 5 Webflux - jProfessionalsMicroservices  with Spring 5 Webflux - jProfessionals
Microservices with Spring 5 Webflux - jProfessionals
 
[KubeCon EU 2022] Running containerd and k3s on macOS
[KubeCon EU 2022] Running containerd and k3s on macOS[KubeCon EU 2022] Running containerd and k3s on macOS
[KubeCon EU 2022] Running containerd and k3s on macOS
 
zeromq
zeromqzeromq
zeromq
 
Docker Swarm for Beginner
Docker Swarm for BeginnerDocker Swarm for Beginner
Docker Swarm for Beginner
 
Introduction to gRPC
Introduction to gRPCIntroduction to gRPC
Introduction to gRPC
 
Reactive Applications with Apache Pulsar and Spring Boot
Reactive Applications with Apache Pulsar and Spring BootReactive Applications with Apache Pulsar and Spring Boot
Reactive Applications with Apache Pulsar and Spring Boot
 
Cypress-vs-Playwright: Let the Code Speak
Cypress-vs-Playwright: Let the Code SpeakCypress-vs-Playwright: Let the Code Speak
Cypress-vs-Playwright: Let the Code Speak
 
Docker 101 - Nov 2016
Docker 101 - Nov 2016Docker 101 - Nov 2016
Docker 101 - Nov 2016
 
Dead Letter Queues for Kafka Consumers in Robinhood, Sreeram Ramji and Wenlon...
Dead Letter Queues for Kafka Consumers in Robinhood, Sreeram Ramji and Wenlon...Dead Letter Queues for Kafka Consumers in Robinhood, Sreeram Ramji and Wenlon...
Dead Letter Queues for Kafka Consumers in Robinhood, Sreeram Ramji and Wenlon...
 
Dual write strategies for microservices
Dual write strategies for microservicesDual write strategies for microservices
Dual write strategies for microservices
 
Introduction MQTT in English
Introduction MQTT in EnglishIntroduction MQTT in English
Introduction MQTT in English
 

Ähnlich wie Europycon2011: Implementing distributed application using ZeroMQ

Lindsay distributed geventzmq
Lindsay distributed geventzmqLindsay distributed geventzmq
Lindsay distributed geventzmq
Robin Xiao
 
RabbitMQ with python and ruby RuPy 2009
RabbitMQ with python and ruby RuPy 2009RabbitMQ with python and ruby RuPy 2009
RabbitMQ with python and ruby RuPy 2009
Paolo Negri
 
[Apostila] programação arduíno brian w. evans
[Apostila] programação arduíno   brian w. evans[Apostila] programação arduíno   brian w. evans
[Apostila] programação arduíno brian w. evans
Web-Desegner
 

Ähnlich wie Europycon2011: Implementing distributed application using ZeroMQ (20)

Lindsay distributed geventzmq
Lindsay distributed geventzmqLindsay distributed geventzmq
Lindsay distributed geventzmq
 
High performance network programming on the jvm oscon 2012
High performance network programming on the jvm   oscon 2012 High performance network programming on the jvm   oscon 2012
High performance network programming on the jvm oscon 2012
 
St Petersburg R user group meetup 2, Parallel R
St Petersburg R user group meetup 2, Parallel RSt Petersburg R user group meetup 2, Parallel R
St Petersburg R user group meetup 2, Parallel R
 
Kamailio with Docker and Kubernetes
Kamailio with Docker and KubernetesKamailio with Docker and Kubernetes
Kamailio with Docker and Kubernetes
 
ZeroMQ - Sockets on steroids!
ZeroMQ - Sockets on steroids!ZeroMQ - Sockets on steroids!
ZeroMQ - Sockets on steroids!
 
Intro to OpenMP
Intro to OpenMPIntro to OpenMP
Intro to OpenMP
 
RabbitMQ with python and ruby RuPy 2009
RabbitMQ with python and ruby RuPy 2009RabbitMQ with python and ruby RuPy 2009
RabbitMQ with python and ruby RuPy 2009
 
Peyton jones-2011-parallel haskell-the_future
Peyton jones-2011-parallel haskell-the_futurePeyton jones-2011-parallel haskell-the_future
Peyton jones-2011-parallel haskell-the_future
 
Simon Peyton Jones: Managing parallelism
Simon Peyton Jones: Managing parallelismSimon Peyton Jones: Managing parallelism
Simon Peyton Jones: Managing parallelism
 
Python Ireland 2012 - Message brokers and Python by Fernando Ciciliati
Python Ireland 2012 - Message brokers and Python by Fernando Ciciliati Python Ireland 2012 - Message brokers and Python by Fernando Ciciliati
Python Ireland 2012 - Message brokers and Python by Fernando Ciciliati
 
The Message Passing Interface (MPI) in Layman's Terms
The Message Passing Interface (MPI) in Layman's TermsThe Message Passing Interface (MPI) in Layman's Terms
The Message Passing Interface (MPI) in Layman's Terms
 
Anton Moldovan "Building an efficient replication system for thousands of ter...
Anton Moldovan "Building an efficient replication system for thousands of ter...Anton Moldovan "Building an efficient replication system for thousands of ter...
Anton Moldovan "Building an efficient replication system for thousands of ter...
 
[Apostila] programação arduíno brian w. evans
[Apostila] programação arduíno   brian w. evans[Apostila] programação arduíno   brian w. evans
[Apostila] programação arduíno brian w. evans
 
TCP/IP For Engineers
TCP/IP For EngineersTCP/IP For Engineers
TCP/IP For Engineers
 
Building & Operating High-Fidelity Data Streams - QCon Plus 2021
Building & Operating High-Fidelity Data Streams - QCon Plus 2021Building & Operating High-Fidelity Data Streams - QCon Plus 2021
Building & Operating High-Fidelity Data Streams - QCon Plus 2021
 
YOW! Data Keynote (2021)
YOW! Data Keynote (2021)YOW! Data Keynote (2021)
YOW! Data Keynote (2021)
 
Hadoop Summit Europe 2014: Apache Storm Architecture
Hadoop Summit Europe 2014: Apache Storm ArchitectureHadoop Summit Europe 2014: Apache Storm Architecture
Hadoop Summit Europe 2014: Apache Storm Architecture
 
Open Source Debugging v1.3.2
Open Source Debugging v1.3.2Open Source Debugging v1.3.2
Open Source Debugging v1.3.2
 
Porting a Streaming Pipeline from Scala to Rust
Porting a Streaming Pipeline from Scala to RustPorting a Streaming Pipeline from Scala to Rust
Porting a Streaming Pipeline from Scala to Rust
 
%w(map reduce).first - A Tale About Rabbits, Latency, and Slim Crontabs
%w(map reduce).first - A Tale About Rabbits, Latency, and Slim Crontabs%w(map reduce).first - A Tale About Rabbits, Latency, and Slim Crontabs
%w(map reduce).first - A Tale About Rabbits, Latency, and Slim Crontabs
 

Kürzlich hochgeladen

Why Teams call analytics are critical to your entire business
Why Teams call analytics are critical to your entire businessWhy Teams call analytics are critical to your entire business
Why Teams call analytics are critical to your entire business
panagenda
 
+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...
+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...
+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...
?#DUbAI#??##{{(☎️+971_581248768%)**%*]'#abortion pills for sale in dubai@
 
Finding Java's Hidden Performance Traps @ DevoxxUK 2024
Finding Java's Hidden Performance Traps @ DevoxxUK 2024Finding Java's Hidden Performance Traps @ DevoxxUK 2024
Finding Java's Hidden Performance Traps @ DevoxxUK 2024
Victor Rentea
 

Kürzlich hochgeladen (20)

Artificial Intelligence Chap.5 : Uncertainty
Artificial Intelligence Chap.5 : UncertaintyArtificial Intelligence Chap.5 : Uncertainty
Artificial Intelligence Chap.5 : Uncertainty
 
CNIC Information System with Pakdata Cf In Pakistan
CNIC Information System with Pakdata Cf In PakistanCNIC Information System with Pakdata Cf In Pakistan
CNIC Information System with Pakdata Cf In Pakistan
 
EMPOWERMENT TECHNOLOGY GRADE 11 QUARTER 2 REVIEWER
EMPOWERMENT TECHNOLOGY GRADE 11 QUARTER 2 REVIEWEREMPOWERMENT TECHNOLOGY GRADE 11 QUARTER 2 REVIEWER
EMPOWERMENT TECHNOLOGY GRADE 11 QUARTER 2 REVIEWER
 
Connector Corner: Accelerate revenue generation using UiPath API-centric busi...
Connector Corner: Accelerate revenue generation using UiPath API-centric busi...Connector Corner: Accelerate revenue generation using UiPath API-centric busi...
Connector Corner: Accelerate revenue generation using UiPath API-centric busi...
 
presentation ICT roal in 21st century education
presentation ICT roal in 21st century educationpresentation ICT roal in 21st century education
presentation ICT roal in 21st century education
 
Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...
Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...
Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...
 
Six Myths about Ontologies: The Basics of Formal Ontology
Six Myths about Ontologies: The Basics of Formal OntologySix Myths about Ontologies: The Basics of Formal Ontology
Six Myths about Ontologies: The Basics of Formal Ontology
 
Why Teams call analytics are critical to your entire business
Why Teams call analytics are critical to your entire businessWhy Teams call analytics are critical to your entire business
Why Teams call analytics are critical to your entire business
 
TrustArc Webinar - Unlock the Power of AI-Driven Data Discovery
TrustArc Webinar - Unlock the Power of AI-Driven Data DiscoveryTrustArc Webinar - Unlock the Power of AI-Driven Data Discovery
TrustArc Webinar - Unlock the Power of AI-Driven Data Discovery
 
+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...
+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...
+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...
 
Elevate Developer Efficiency & build GenAI Application with Amazon Q​
Elevate Developer Efficiency & build GenAI Application with Amazon Q​Elevate Developer Efficiency & build GenAI Application with Amazon Q​
Elevate Developer Efficiency & build GenAI Application with Amazon Q​
 
Finding Java's Hidden Performance Traps @ DevoxxUK 2024
Finding Java's Hidden Performance Traps @ DevoxxUK 2024Finding Java's Hidden Performance Traps @ DevoxxUK 2024
Finding Java's Hidden Performance Traps @ DevoxxUK 2024
 
Boost Fertility New Invention Ups Success Rates.pdf
Boost Fertility New Invention Ups Success Rates.pdfBoost Fertility New Invention Ups Success Rates.pdf
Boost Fertility New Invention Ups Success Rates.pdf
 
ProductAnonymous-April2024-WinProductDiscovery-MelissaKlemke
ProductAnonymous-April2024-WinProductDiscovery-MelissaKlemkeProductAnonymous-April2024-WinProductDiscovery-MelissaKlemke
ProductAnonymous-April2024-WinProductDiscovery-MelissaKlemke
 
Apidays New York 2024 - The value of a flexible API Management solution for O...
Apidays New York 2024 - The value of a flexible API Management solution for O...Apidays New York 2024 - The value of a flexible API Management solution for O...
Apidays New York 2024 - The value of a flexible API Management solution for O...
 
Mcleodganj Call Girls 🥰 8617370543 Service Offer VIP Hot Model
Mcleodganj Call Girls 🥰 8617370543 Service Offer VIP Hot ModelMcleodganj Call Girls 🥰 8617370543 Service Offer VIP Hot Model
Mcleodganj Call Girls 🥰 8617370543 Service Offer VIP Hot Model
 
Rising Above_ Dubai Floods and the Fortitude of Dubai International Airport.pdf
Rising Above_ Dubai Floods and the Fortitude of Dubai International Airport.pdfRising Above_ Dubai Floods and the Fortitude of Dubai International Airport.pdf
Rising Above_ Dubai Floods and the Fortitude of Dubai International Airport.pdf
 
Navigating the Deluge_ Dubai Floods and the Resilience of Dubai International...
Navigating the Deluge_ Dubai Floods and the Resilience of Dubai International...Navigating the Deluge_ Dubai Floods and the Resilience of Dubai International...
Navigating the Deluge_ Dubai Floods and the Resilience of Dubai International...
 
Corporate and higher education May webinar.pptx
Corporate and higher education May webinar.pptxCorporate and higher education May webinar.pptx
Corporate and higher education May webinar.pptx
 
DEV meet-up UiPath Document Understanding May 7 2024 Amsterdam
DEV meet-up UiPath Document Understanding May 7 2024 AmsterdamDEV meet-up UiPath Document Understanding May 7 2024 Amsterdam
DEV meet-up UiPath Document Understanding May 7 2024 Amsterdam
 

Europycon2011: Implementing distributed application using ZeroMQ

Hinweis der Redaktion

  1. \n
  2. \n
  3. \n
  4. \n
  5. \n
  6. \n
  7. \n
  8. \n
  9. \n
  10. \n
  11. \n
  12. \n
  13. \n
  14. \n
  15. \n
  16. \n
  17. \n
  18. \n
  19. \n
  20. \n
  21. \n
  22. \n
  23. \n
  24. \n
  25. \n
  26. \n
  27. \n
  28. \n
  29. \n
  30. \n
  31. \n
  32. \n
  33. \n
  34. \n
  35. \n
  36. \n
  37. \n
  38. \n
  39. \n
  40. \n
  41. \n
  42. \n
  43. \n
  44. \n
  45. \n
  46. \n
  47. \n
  48. \n
  49. \n
  50. \n
  51. \n
  52. \n
  53. \n
  54. \n
  55. \n
  56. \n
  57. \n
  58. \n
  59. \n
  60. \n
  61. \n
  62. \n
  63. \n