SlideShare ist ein Scribd-Unternehmen logo
1 von 19
Downloaden Sie, um offline zu lesen
ZeroMQ
Anatomy & JeroMQ
ZeroMQ
MQ 가 아니무니다..
TCP on Steroids
ZMQ Socket
ROUTER DEALER XPUB XSUB PUSH
PULLREP REQ PUB SUB
ZeroMQ Anatomy
R w
io_thread_t
ZMQ Socket
Mailbox
Signaler
ypipe_t
<cmd_t>
Session
Socket
Session
Socket
poller
r
fd
fd
ypipe_t <msg_t> inpipe
ypipe_t <msg_t> outpipe
ZMQ
Socket
ZMQ
Socket
ZMQ
Socket
ZMQ
Socket
SessionSessionSession
Context
Mailbox
r
Msg
union {
...
struct {
unsigned char data [29];
unsigned char size;
unsigned char type;
unsigned char flags;
} vsm;
struct {
content_t *content;
unsigned char unused
[29 + 1 - sizeof (content_t*)];
unsigned char type;
unsigned char flags;
} lmsg;
DATA S T F
T F
S DATA
vsm
lmsg
writer thread
reader thread
YQueue
yqueue_t <msg_t, 256>
yqueue_t <command_t, 16>
struct chunk_t
{
T values [N];
chunk_t *prev;
chunk_t *next;
};
atomic_ptr_t <chunk_t>
spare_chunk
...
begin
back
• Only single thread can read from begin_chunk
• Only single thread can write to yqueue_t <msg_t> to back chunk
• Only single thread can write to yqueue_t <command_t> at a time
YPipe
atomic_ptr_t <T> c
Reader
Writer
T* r
T* w
T* f
bool flush () {
if ( c.cas ( w, f ) != w ) {
c.set ( f ) ;
w = f ;
return false ; // reader is sleeping
}
w = f ;
return true ;
}
bool check_read () {
if ( queue.front () != r ) return true ;
r = c.cas ( queue.front () , NULL ) ;
if ( queue.front () == r )
return false ; // nothing to read
return true;
}
read() =>
queue<T>.pop
write() =>
queue<T>.push
r w f
Signaler
socketpair ( AF_UNIX, SOCK_STREAM, 0, sv[2] )
int w = sv[0]; // writer fd
int r = sv[1]; // reader fd, Used at IOThread.poll
Reader
recv () {
:: recv ( r , dummy[1] );
}
wait ( timeout ) {
poll ( r , timeout );
}
Writer
send () {
:: send ( w, dummy[1] );
}
Mailbox
ypipe_t <command_t, 16> cpipe
signaler_t signaler
Reader
Writer
recv (timeout) {
signaler.wait (timeout)
return cpipe.read ()
}
send (cmd) {
lock ()
cpipe.write (cmd)
cpipe.flush ()
unlock ()
signaler.send ()
}
ZMQ Socket
ZMQ Socket
mailbox_t
signaler_
typipe_t
<cmd_t>
Session
Socket
Session
Socket
r
fd
fd
ypipe_t <msg_t> inpipe
ypipe_t <msg_t> outpipe
• Session is create when connect / accept
•Worker thread owns ZMQ socket
• IOThread owns Sessions
• recv msg from socket and en-queue inpipe
• recv msg from outpipe and send to socket
• zsocket.send en-queue to outpipe
• zsocket.recv de-queue from inpipe if any
main/worker
IOThread
io_thread
ZMQ Socket
mailbox
Session
Socket
poller
ZMQ
Socket
ZMQ
Socket
ZMQ
Socket
ZMQ
Socket
Session
mailbox
zsocket.write(msg)
outpipe.write(msg)
activate_read
session.read_activated
engine.activate_out
engine.out_event
SessionSessionSession
Stream Engine
engine.in_event
outpipe.write(msg)
activate_read
zsocket.read_activated
msg = inpipe.read
in_event () { // read from socket
size = 8192 // ***
buffer = decoder . get_buffer (size)
::recv (buffer, size)
decoder . process_buffer(buffer)
session . flush()
}
out_event () { // write to socket
size = 8192 // ***
buffer = encoder . get_data (size)
::send (buffer, size)
}
msg = parse (buffer)
session.write (msg)
Stream Engine
msg = session.read ()
buffer.fill (msg)
Summary
inline
lock-free
well-defined threading model
less system calls
• malloc / free
• bulk socket recv / send
• balance copy overhead
JeroMQ - jeromq.org
Hand-made
Compatible with ZeroMQ-3
tcp:// inproc://
ipc:// pgm://
same dev experience
95%~ performance
JeroMQ - diff
Java NIO
SelectableChannel
(Direct) ByteBuffer
Signaler
Pipe.SourceChannel / Pipe.SinkChannel
Mixed with counter
YQueue - circular linked list
Plain Socket Proxy
Persistence - Kafka alike
MappedByteBuffer
ZeroCopy
Future
Zero Persistence Queue
http://github.com/miniway/zper
Faster than Kafka
50% Less code
Configurable topology
Java 7 - AsynchronousChannel
Conclusion
Why reinvent the wheel
“The only limitation is your imagination and
sobriety”
Q/A

Weitere ähnliche Inhalte

Was ist angesagt?

エスイーが要件定義でやるべきたったひとつのこと
エスイーが要件定義でやるべきたったひとつのことエスイーが要件定義でやるべきたったひとつのこと
エスイーが要件定義でやるべきたったひとつのこと
Yoshitaka Kawashima
 
スマートフォン向けサービスにおけるサーバサイド設計入門
スマートフォン向けサービスにおけるサーバサイド設計入門スマートフォン向けサービスにおけるサーバサイド設計入門
スマートフォン向けサービスにおけるサーバサイド設計入門
Hisashi HATAKEYAMA
 
闇魔術を触ってみた
闇魔術を触ってみた闇魔術を触ってみた
闇魔術を触ってみた
Satoshi Sato
 

Was ist angesagt? (20)

Datomic&datalog紹介
Datomic&datalog紹介Datomic&datalog紹介
Datomic&datalog紹介
 
エスイーが要件定義でやるべきたったひとつのこと
エスイーが要件定義でやるべきたったひとつのことエスイーが要件定義でやるべきたったひとつのこと
エスイーが要件定義でやるべきたったひとつのこと
 
[CSStudy] 코딩인터뷰 완전 분석 #7.pdf
[CSStudy] 코딩인터뷰 완전 분석 #7.pdf[CSStudy] 코딩인터뷰 완전 분석 #7.pdf
[CSStudy] 코딩인터뷰 완전 분석 #7.pdf
 
HTTP/2 入門
HTTP/2 入門HTTP/2 入門
HTTP/2 入門
 
ZeroMQ with NodeJS
ZeroMQ with NodeJSZeroMQ with NodeJS
ZeroMQ with NodeJS
 
IIJmio meeting 29 総務省 モバイル市場の現状と政策動向
IIJmio meeting 29 総務省 モバイル市場の現状と政策動向IIJmio meeting 29 総務省 モバイル市場の現状と政策動向
IIJmio meeting 29 総務省 モバイル市場の現状と政策動向
 
Best Network Performance Monitoring Tool
Best Network Performance Monitoring ToolBest Network Performance Monitoring Tool
Best Network Performance Monitoring Tool
 
Seminar On Trojan Horse
Seminar On Trojan HorseSeminar On Trojan Horse
Seminar On Trojan Horse
 
MSBuild + Git + Jenkins
MSBuild + Git + JenkinsMSBuild + Git + Jenkins
MSBuild + Git + Jenkins
 
[2B7]시즌2 멀티쓰레드프로그래밍이 왜 이리 힘드나요
[2B7]시즌2 멀티쓰레드프로그래밍이 왜 이리 힘드나요[2B7]시즌2 멀티쓰레드프로그래밍이 왜 이리 힘드나요
[2B7]시즌2 멀티쓰레드프로그래밍이 왜 이리 힘드나요
 
Domain Modeling Made Functional (DevTernity 2022)
Domain Modeling Made Functional (DevTernity 2022)Domain Modeling Made Functional (DevTernity 2022)
Domain Modeling Made Functional (DevTernity 2022)
 
MPLS VPN
MPLS VPNMPLS VPN
MPLS VPN
 
スマートフォン向けサービスにおけるサーバサイド設計入門
スマートフォン向けサービスにおけるサーバサイド設計入門スマートフォン向けサービスにおけるサーバサイド設計入門
スマートフォン向けサービスにおけるサーバサイド設計入門
 
闇魔術を触ってみた
闇魔術を触ってみた闇魔術を触ってみた
闇魔術を触ってみた
 
IIJ Technical DAY 2019 ~ IIJのサーバインフラはここまでやっています
IIJ Technical DAY 2019 ~ IIJのサーバインフラはここまでやっていますIIJ Technical DAY 2019 ~ IIJのサーバインフラはここまでやっています
IIJ Technical DAY 2019 ~ IIJのサーバインフラはここまでやっています
 
Functional Design Patterns (DevTernity 2018)
Functional Design Patterns (DevTernity 2018)Functional Design Patterns (DevTernity 2018)
Functional Design Patterns (DevTernity 2018)
 
How BGP Works
How BGP WorksHow BGP Works
How BGP Works
 
Goji とレイヤ化アーキテクチャ
Goji とレイヤ化アーキテクチャGoji とレイヤ化アーキテクチャ
Goji とレイヤ化アーキテクチャ
 
Secured Internet Gateway for ISP with pfsense & FRR
Secured Internet Gateway for ISP with pfsense & FRRSecured Internet Gateway for ISP with pfsense & FRR
Secured Internet Gateway for ISP with pfsense & FRR
 
コードで学ぶドメイン駆動設計入門
コードで学ぶドメイン駆動設計入門コードで学ぶドメイン駆動設計入門
コードで学ぶドメイン駆動設計入門
 

Ähnlich wie Zeromq anatomy & jeromq

Степан Кольцов — Rust — лучше, чем C++
Степан Кольцов — Rust — лучше, чем C++Степан Кольцов — Rust — лучше, чем C++
Степан Кольцов — Rust — лучше, чем C++
Yandex
 
Shell to be modified#include stdlib.h #include unistd.h .pdf
Shell to be modified#include stdlib.h #include unistd.h .pdfShell to be modified#include stdlib.h #include unistd.h .pdf
Shell to be modified#include stdlib.h #include unistd.h .pdf
clarityvision
 

Ähnlich wie Zeromq anatomy & jeromq (20)

Distributed Computing Patterns in R
Distributed Computing Patterns in RDistributed Computing Patterns in R
Distributed Computing Patterns in R
 
Zmq in context of openstack
Zmq in context of openstackZmq in context of openstack
Zmq in context of openstack
 
20120705 - LUA_MODULE
20120705 - LUA_MODULE20120705 - LUA_MODULE
20120705 - LUA_MODULE
 
RAZORPOINT TCP/UDP PORTS LIST
RAZORPOINT TCP/UDP PORTS LISTRAZORPOINT TCP/UDP PORTS LIST
RAZORPOINT TCP/UDP PORTS LIST
 
Debugging Ruby Systems
Debugging Ruby SystemsDebugging Ruby Systems
Debugging Ruby Systems
 
OpenSMTPD: we deliver !!
OpenSMTPD: we deliver !!OpenSMTPD: we deliver !!
OpenSMTPD: we deliver !!
 
The New Systems Performance
The New Systems PerformanceThe New Systems Performance
The New Systems Performance
 
Npc08
Npc08Npc08
Npc08
 
Степан Кольцов — Rust — лучше, чем C++
Степан Кольцов — Rust — лучше, чем C++Степан Кольцов — Rust — лучше, чем C++
Степан Кольцов — Rust — лучше, чем C++
 
Pf: the OpenBSD packet filter
Pf: the OpenBSD packet filterPf: the OpenBSD packet filter
Pf: the OpenBSD packet filter
 
Services
ServicesServices
Services
 
Tc pdump mod
Tc pdump modTc pdump mod
Tc pdump mod
 
Shell to be modified#include stdlib.h #include unistd.h .pdf
Shell to be modified#include stdlib.h #include unistd.h .pdfShell to be modified#include stdlib.h #include unistd.h .pdf
Shell to be modified#include stdlib.h #include unistd.h .pdf
 
20120521 - zeroMQ
20120521 - zeroMQ20120521 - zeroMQ
20120521 - zeroMQ
 
WMQ Toolbox: 20 Scripts, One-liners, & Utilities for UNIX & Windows
WMQ Toolbox: 20 Scripts, One-liners, & Utilities for UNIX & Windows WMQ Toolbox: 20 Scripts, One-liners, & Utilities for UNIX & Windows
WMQ Toolbox: 20 Scripts, One-liners, & Utilities for UNIX & Windows
 
اسلاید دوم جلسه یازدهم کلاس پایتون برای هکر های قانونی
اسلاید دوم جلسه یازدهم کلاس پایتون برای هکر های قانونیاسلاید دوم جلسه یازدهم کلاس پایتون برای هکر های قانونی
اسلاید دوم جلسه یازدهم کلاس پایتون برای هکر های قانونی
 
A22 Introduction to DTrace by Kyle Hailey
A22 Introduction to DTrace by Kyle HaileyA22 Introduction to DTrace by Kyle Hailey
A22 Introduction to DTrace by Kyle Hailey
 
[CB20]-U25 Automated Hunting for Cross-Server Xrefs in Microsoft RPC and COM ...
[CB20]-U25 Automated Hunting for Cross-Server Xrefs in Microsoft RPC and COM ...[CB20]-U25 Automated Hunting for Cross-Server Xrefs in Microsoft RPC and COM ...
[CB20]-U25 Automated Hunting for Cross-Server Xrefs in Microsoft RPC and COM ...
 
Debugging Ruby
Debugging RubyDebugging Ruby
Debugging Ruby
 
Pycon - Python for ethical hackers
Pycon - Python for ethical hackers Pycon - Python for ethical hackers
Pycon - Python for ethical hackers
 

Kürzlich hochgeladen

%+27788225528 love spells in Atlanta Psychic Readings, Attraction spells,Brin...
%+27788225528 love spells in Atlanta Psychic Readings, Attraction spells,Brin...%+27788225528 love spells in Atlanta Psychic Readings, Attraction spells,Brin...
%+27788225528 love spells in Atlanta Psychic Readings, Attraction spells,Brin...
masabamasaba
 
TECUNIQUE: Success Stories: IT Service provider
TECUNIQUE: Success Stories: IT Service providerTECUNIQUE: Success Stories: IT Service provider
TECUNIQUE: Success Stories: IT Service provider
mohitmore19
 
AI Mastery 201: Elevating Your Workflow with Advanced LLM Techniques
AI Mastery 201: Elevating Your Workflow with Advanced LLM TechniquesAI Mastery 201: Elevating Your Workflow with Advanced LLM Techniques
AI Mastery 201: Elevating Your Workflow with Advanced LLM Techniques
VictorSzoltysek
 

Kürzlich hochgeladen (20)

%+27788225528 love spells in Atlanta Psychic Readings, Attraction spells,Brin...
%+27788225528 love spells in Atlanta Psychic Readings, Attraction spells,Brin...%+27788225528 love spells in Atlanta Psychic Readings, Attraction spells,Brin...
%+27788225528 love spells in Atlanta Psychic Readings, Attraction spells,Brin...
 
TECUNIQUE: Success Stories: IT Service provider
TECUNIQUE: Success Stories: IT Service providerTECUNIQUE: Success Stories: IT Service provider
TECUNIQUE: Success Stories: IT Service provider
 
%in Lydenburg+277-882-255-28 abortion pills for sale in Lydenburg
%in Lydenburg+277-882-255-28 abortion pills for sale in Lydenburg%in Lydenburg+277-882-255-28 abortion pills for sale in Lydenburg
%in Lydenburg+277-882-255-28 abortion pills for sale in Lydenburg
 
%in kempton park+277-882-255-28 abortion pills for sale in kempton park
%in kempton park+277-882-255-28 abortion pills for sale in kempton park %in kempton park+277-882-255-28 abortion pills for sale in kempton park
%in kempton park+277-882-255-28 abortion pills for sale in kempton park
 
AI & Machine Learning Presentation Template
AI & Machine Learning Presentation TemplateAI & Machine Learning Presentation Template
AI & Machine Learning Presentation Template
 
AI Mastery 201: Elevating Your Workflow with Advanced LLM Techniques
AI Mastery 201: Elevating Your Workflow with Advanced LLM TechniquesAI Mastery 201: Elevating Your Workflow with Advanced LLM Techniques
AI Mastery 201: Elevating Your Workflow with Advanced LLM Techniques
 
SHRMPro HRMS Software Solutions Presentation
SHRMPro HRMS Software Solutions PresentationSHRMPro HRMS Software Solutions Presentation
SHRMPro HRMS Software Solutions Presentation
 
Right Money Management App For Your Financial Goals
Right Money Management App For Your Financial GoalsRight Money Management App For Your Financial Goals
Right Money Management App For Your Financial Goals
 
%in Bahrain+277-882-255-28 abortion pills for sale in Bahrain
%in Bahrain+277-882-255-28 abortion pills for sale in Bahrain%in Bahrain+277-882-255-28 abortion pills for sale in Bahrain
%in Bahrain+277-882-255-28 abortion pills for sale in Bahrain
 
Announcing Codolex 2.0 from GDK Software
Announcing Codolex 2.0 from GDK SoftwareAnnouncing Codolex 2.0 from GDK Software
Announcing Codolex 2.0 from GDK Software
 
The Top App Development Trends Shaping the Industry in 2024-25 .pdf
The Top App Development Trends Shaping the Industry in 2024-25 .pdfThe Top App Development Trends Shaping the Industry in 2024-25 .pdf
The Top App Development Trends Shaping the Industry in 2024-25 .pdf
 
Chinsurah Escorts ☎️8617697112 Starting From 5K to 15K High Profile Escorts ...
Chinsurah Escorts ☎️8617697112  Starting From 5K to 15K High Profile Escorts ...Chinsurah Escorts ☎️8617697112  Starting From 5K to 15K High Profile Escorts ...
Chinsurah Escorts ☎️8617697112 Starting From 5K to 15K High Profile Escorts ...
 
call girls in Vaishali (Ghaziabad) 🔝 >༒8448380779 🔝 genuine Escort Service 🔝✔️✔️
call girls in Vaishali (Ghaziabad) 🔝 >༒8448380779 🔝 genuine Escort Service 🔝✔️✔️call girls in Vaishali (Ghaziabad) 🔝 >༒8448380779 🔝 genuine Escort Service 🔝✔️✔️
call girls in Vaishali (Ghaziabad) 🔝 >༒8448380779 🔝 genuine Escort Service 🔝✔️✔️
 
Microsoft AI Transformation Partner Playbook.pdf
Microsoft AI Transformation Partner Playbook.pdfMicrosoft AI Transformation Partner Playbook.pdf
Microsoft AI Transformation Partner Playbook.pdf
 
%in Durban+277-882-255-28 abortion pills for sale in Durban
%in Durban+277-882-255-28 abortion pills for sale in Durban%in Durban+277-882-255-28 abortion pills for sale in Durban
%in Durban+277-882-255-28 abortion pills for sale in Durban
 
10 Trends Likely to Shape Enterprise Technology in 2024
10 Trends Likely to Shape Enterprise Technology in 202410 Trends Likely to Shape Enterprise Technology in 2024
10 Trends Likely to Shape Enterprise Technology in 2024
 
Define the academic and professional writing..pdf
Define the academic and professional writing..pdfDefine the academic and professional writing..pdf
Define the academic and professional writing..pdf
 
The Ultimate Test Automation Guide_ Best Practices and Tips.pdf
The Ultimate Test Automation Guide_ Best Practices and Tips.pdfThe Ultimate Test Automation Guide_ Best Practices and Tips.pdf
The Ultimate Test Automation Guide_ Best Practices and Tips.pdf
 
%in kaalfontein+277-882-255-28 abortion pills for sale in kaalfontein
%in kaalfontein+277-882-255-28 abortion pills for sale in kaalfontein%in kaalfontein+277-882-255-28 abortion pills for sale in kaalfontein
%in kaalfontein+277-882-255-28 abortion pills for sale in kaalfontein
 
Reassessing the Bedrock of Clinical Function Models: An Examination of Large ...
Reassessing the Bedrock of Clinical Function Models: An Examination of Large ...Reassessing the Bedrock of Clinical Function Models: An Examination of Large ...
Reassessing the Bedrock of Clinical Function Models: An Examination of Large ...
 

Zeromq anatomy & jeromq