he recent explosion in products supporting multiple devices require exposing data through lightweight services and data driven APIs. We're shifting data across flaky networks, massaging between different storage backends, encode / decode by protocol and frequently drive
this from Cloud platforms with very variable Quality of Service and availability guarantees (if any). Yet most engineering teams aren't up to speed with POSIX and POSIX.1b(Realtime-extensions), the API's exposed by most modern production systems for I/O.
In this talk I'll look at a few very often misunderstood concepts, as well as challenge others.
* Understanding buffered I/O and how to sustain throughput for fast and slow clients alike
* Blocking VS Non-blocking I/O
* POSIX Lies - compliance isn't always as it should be, in some cases *worse* than CSS and other specifications "supported" by modern browsers
* Asynchronous I/O myths and why I/O concurrency is often an illusion
* Why and how Windows I/O Completion Ports knock the socks off most UNIX systems
* I/O multiplexing gotchas
* The Cloud: managing latency with variable network and storage characteristics
* High throughput system calls
Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...
TX/RX 101: Transfer data efficiently
1. Tx/Rx 101: Transfer data efficiently
SAPO Codebits 2011
Lisbon - http://codebits.eu
Lourens Naudé
Wildfire Interactive, Inc. | 1600 Seaport Boulevard, Suite 500, Redwood City, CA 94063 | (888) 274-0929
Thursday, November 10, 11
2. Bio
Operations
@ WildfireApp.com
Wildfire Interactive, Inc. | 1600 Seaport Boulevard, Suite 500, Redwood City, CA 94063 | (888) 274-0929
Thursday, November 10, 11
A few months ago at WildfireApp.com I made the transition from a developer background and role to the operations team.
With an Engineering mind it’s easy to get distracted by lower level details and loose sight of the bigger picture. During this
time we’ve released several products and this changed my perspective on operational complexity, moving parts and the
importance of getting and keeping stable software out in front in customers. Less is more. There will be some notes on this
during today’s talk as well.
3. Communication
All modern software should
represent a connected
world.
Wildfire Interactive, Inc. | 1600 Seaport Boulevard, Suite 500, Redwood City, CA 94063 | (888) 274-0929
Thursday, November 10, 11
Lots of trending products are actually platforms, spread across multiple devices. API and integration driven, like Twitter. Even
this event exposes various APIs for projects to consume in the coming days. Software is social.
4. Communication
Most social problems are
communication problems.
Systems aren’t any different
Wildfire Interactive, Inc. | 1600 Seaport Boulevard, Suite 500, Redwood City, CA 94063 | (888) 274-0929
Thursday, November 10, 11
Two outstanding individuals can engage in a relationship, but it’s bound to fail if there’s little or no communication.
5. All systems GO !
Wildfire Interactive, Inc. | 1600 Seaport Boulevard, Suite 500, Redwood City, CA 94063 | (888) 274-0929
Thursday, November 10, 11
http://www.flickr.com/photos/23968709@N03/5452362745/
6. Efficiency
Performant individual requests
Scalability
Operational complexity
Minimal / no developer on boarding
Cost of ownership (today and forward
looking)
Wildfire Interactive, Inc. | 1600 Seaport Boulevard, Suite 500, Redwood City, CA 94063 | (888) 274-0929
Thursday, November 10, 11
Different things to different people. It’s not just about performant individual requests, but also scaling those out to multiple
clients. Operational complexity and moving parts should be kept to a minimum. Often overlooked, especially in startup
environments, is on boarding costs of new developers, especially in the presence of lower level / funky protocols etc. that
require a bootstrap period. Rule of thumb: your cowboy should get it. All of this directly affects the bottom line.
7. Efficient Scalability
To maintain acceptable
response times for more clients,
with the same or less
infrastructure
Wildfire Interactive, Inc. | 1600 Seaport Boulevard, Suite 500, Redwood City, CA 94063 | (888) 274-0929
Thursday, November 10, 11
This is representative of most public facing production deployments. Remember, not about the speed of a single
request, but being able to easily handle increased throughput.
8. Agenda
System calls and file descriptors
Blocking, nonblocking and async I/O
Multiplexed I/O
There’s a better way ...
Optimizations
Wildfire Interactive, Inc. | 1600 Seaport Boulevard, Suite 500, Redwood City, CA 94063 | (888) 274-0929
Thursday, November 10, 11
We’ll allocate time towards system calls, file descriptors - the canonical reference used by user space apps for I/O and how all
of that fits into common I/O models: blocking, nonblocking and async. The next challenge is applying this knowledge at scale
using multiplexed I/O, however apparently the world is coming to an end in 2012 and there’s much better alternatives. We’ll
then phase out with a few common optimizations.
9. Head count ?
Wildfire Interactive, Inc. | 1600 Seaport Boulevard, Suite 500, Redwood City, CA 94063 | (888) 274-0929
Thursday, November 10, 11
Anyone currently consuming or exposing APIs ? Messing around with or using any of these technologies in
production ?
10. The kernel
Mediates access to system
resources :
I/O, memory and CPU
Wildfire Interactive, Inc. | 1600 Seaport Boulevard, Suite 500, Redwood City, CA 94063 | (888) 274-0929
Thursday, November 10, 11
Since the 60s, nothings really changed much with how we access system resources. First line of communication:
between the Kernel and User space is a negotiation for resources.
11. User mode
Sandboxed execution for
security and stability
requirements
Wildfire Interactive, Inc. | 1600 Seaport Boulevard, Suite 500, Redwood City, CA 94063 | (888) 274-0929
Thursday, November 10, 11
The vast majority of processes, especially scripting languages, will spend most of their on CPU time in user mode.
How does user processes access system resources ?
12. System calls
Ruby
process
Kernel disk
Ruby
process
read(5, &buf, 4000)
Wildfire Interactive, Inc. | 1600 Seaport Boulevard, Suite 500, Redwood City, CA 94063 | (888) 274-0929
Thursday, November 10, 11
Ask the Kernel to do work on behalf of a user process.
13. Syscall definition
A protected function call
from user to kernel space
with the intent to interact
with a system resource
Wildfire Interactive, Inc. | 1600 Seaport Boulevard, Suite 500, Redwood City, CA 94063 | (888) 274-0929
Thursday, November 10, 11
14. Characteristics
Uniform API on POSIX systems
Ditto for behavior. LIES !
Slow emulations
ssize_t read(int d, void *buf,
size_t nbytes)
ssize_t write(int d, const void
*buf, size_t nbytes)
Wildfire Interactive, Inc. | 1600 Seaport Boulevard, Suite 500, Redwood City, CA 94063 | (888) 274-0929
Thursday, November 10, 11
It's also guaranteed to have a uniform API and behavior on POSIX compliant (read: most UNIX) systems. Just like
HTML and CSS compliance in modern browsers, most operating system implementations do not conform 100% to
spec. Configure checks at compile time may confirm the presence of a performant system call when in fact it’s just
an emulation.
15. Syscall performance
MUCH slower than function calls
20x and more
Definite context switch
Calls can block - slow upstream peers
Wildfire Interactive, Inc. | 1600 Seaport Boulevard, Suite 500, Redwood City, CA 94063 | (888) 274-0929
Thursday, November 10, 11
An important performance characteristic of system calls is that they're an order of magnitude more expensive
than eg. a libc function
call in the same process. In other words, there's a context switch that affects several subsystems eg. swapping out
CPU registers and memory references.
They may also block indefinitely or until a condition's met eg. a full buffer on a read syscall.
16. Who doesn't know what file
descriptors or file handles
are ?
Wildfire Interactive, Inc. | 1600 Seaport Boulevard, Suite 500, Redwood City, CA 94063 | (888) 274-0929
Thursday, November 10, 11
17. open syscall
fd = open(“/path/file”, O_READ)
read(fd, &buf, 4000)
Wildfire Interactive, Inc. | 1600 Seaport Boulevard, Suite 500, Redwood City, CA 94063 | (888) 274-0929
Thursday, November 10, 11
OPEN system call to open a file. Give it a path and it returns a resource, which is used in subsequent READ and
WRITE calls for I/O.
18. Examples
local file
socket
directory
pipe
Consistent API, semantics differ
Wildfire Interactive, Inc. | 1600 Seaport Boulevard, Suite 500, Redwood City, CA 94063 | (888) 274-0929
Thursday, November 10, 11
A file descriptor may represent a local file, a socket, a directory or a pipe etc. Semantics for these resources differ
immensely, even
though the canonical reference is a numeric handle and the basic read and write system call APIs are equivalent.
Only ever a device to the kernel.
19. Definition
Numeric handle
References a kernel allocated resource
Kernel and user space buffer
/path/serv.sock
( fd 5 )
User Kernel
buffer buffer
Wildfire Interactive, Inc. | 1600 Seaport Boulevard, Suite 500, Redwood City, CA 94063 | (888) 274-0929
Thursday, November 10, 11
A file descriptor (or handle) is a numeric handle representing an I/O resource allocated by the Kernel, with
metadata in the Kernel and buffers in both Kernel and User space. SIMPLE and EFFICIENT contract.
20. Blocking I/O
A request to read a chunk of
data from a descriptor may
block depending on
buffer state
Wildfire Interactive, Inc. | 1600 Seaport Boulevard, Suite 500, Redwood City, CA 94063 | (888) 274-0929
Thursday, November 10, 11
File descriptors are blocking by DEFAULT. Easiest I/O model, but also unpredictable with the worst performance
guarantees.
21. Blocked buffer state
read(5, &b, 4000)
User Kernel
buffer buffer
2000
1000
Wildfire Interactive, Inc. | 1600 Seaport Boulevard, Suite 500, Redwood City, CA 94063 | (888) 274-0929
Thursday, November 10, 11
Kernel puts the caller process in a waiting state. The application ( execution thread ) sleeps until the I/O operation
has completed (or has generated an error) at which point it is scheduled to run again. Simple to use, but many
downsides. Domino effect on connected peers.
22. Nonblocking I/O
fd = socket (PF_INET, SOCK_STREAM, 0);
fcntl (fd, F_SETFL, O_NONBLOCK)
Wildfire Interactive, Inc. | 1600 Seaport Boulevard, Suite 500, Redwood City, CA 94063 | (888) 274-0929
Thursday, November 10, 11
The POSIX spec allows for changing this behavior by setting the O_NONBLOCK flag on a descriptor.
23. Nonblocking I/O
A read request would only
be initiated if the system
call won’t block
Wildfire Interactive, Inc. | 1600 Seaport Boulevard, Suite 500, Redwood City, CA 94063 | (888) 274-0929
Thursday, November 10, 11
24. read(5, &b, 4000)
User Kernel
buffer EAGAIN buffer
2000
1000
Wildfire Interactive, Inc. | 1600 Seaport Boulevard, Suite 500, Redwood City, CA 94063 | (888) 274-0929
Thursday, November 10, 11
In this scenario, we’ll partial read the 1000 bytes, but errno would be set to EAGAIN, indicating that we need to
issue the read request again.
Lots of system call overhead – constantly making system calls to see if I/O is ready. Can be high latency if I/O
arrives and a system call is not made for a while. Efficiency is very poor due to the constant polling with system
calls from user space
25. Don’t drink the kool aid
Guideline only
Not free - invokes a syscall
Not supported by all devices
Wildfire Interactive, Inc. | 1600 Seaport Boulevard, Suite 500, Redwood City, CA 94063 | (888) 274-0929
Thursday, November 10, 11
Do note that this behavior is thus just a guideline for the caller. We’re told to retry, just not when. It's not free
either - there's the context switch as well as system call arguments validation in the Kernel as well.
There's a considerable grey area, as per POSIX spec, that ignores O_NONBLOCK semantics on any file handle that
references a file on a
block device. This has an important performance impact on systems that access block devices with variable
performance characteristics, like EBS @ Amazon AWS. All system calls don’t respect non-blocking I/O either
( libeio / libuv )
26. Async I/O myth
Nonblocking I/O IS NOT
asynchronous I/O
Wildfire Interactive, Inc. | 1600 Seaport Boulevard, Suite 500, Redwood City, CA 94063 | (888) 274-0929
Thursday, November 10, 11
File descriptors change state ( read / write ) asynchronously at the Kernel layer, we handle those events in User
space. A NIC receives data which is buffered in the kernel. We still need to ask for and copy it. The takeaway, and
what you MUST remember here is that this IS NOT asynchronous I/O.
An event occurs asynchronously, but is handled synchronously.
27. Async I/O
Callbacks and notifications
Issues I/O in parallel
Direct I/O (no double buffering)
Popular with database systems
Tell the Kernel everything it needs to
complete an I/O job
Wildfire Interactive, Inc. | 1600 Seaport Boulevard, Suite 500, Redwood City, CA 94063 | (888) 274-0929
Thursday, November 10, 11
Helps maximize I/O throughput by allowing lots of I/O to issued in parallel. Network IO is frequently not
supported.
28. fd : 10
op : read
notify : signal
buffer : 0x100431
submit job
size : 4
callback() data, notify
"data"
Application Kernel
Wildfire Interactive, Inc. | 1600 Seaport Boulevard, Suite 500, Redwood City, CA 94063 | (888) 274-0929
Thursday, November 10, 11
29. POSIX Realtime Extension
aio_read, aio_write etc.
Supports file I/O only
Wildfire Interactive, Inc. | 1600 Seaport Boulevard, Suite 500, Redwood City, CA 94063 | (888) 274-0929
Thursday, November 10, 11
30. OS X Lion - AIO support
case SIGEV_NONE:
! ! break;
case SIGEV_THREAD:
! /* Unsupported [RTS] */
default:
! return (EINVAL);
Wildfire Interactive, Inc. | 1600 Seaport Boulevard, Suite 500, Redwood City, CA 94063 | (888) 274-0929
Thursday, November 10, 11
Preferred notification mechanism is through signal ( SIGIO ), but doesn’t support passing any metadata either.
Thus if there’s 10 jobs submitted, we still need to figure out which of those jobs are in a completed state.
31. Windows I/O Completion Ports
Supports all descriptor types
Very mature and stable technology
Wildfire Interactive, Inc. | 1600 Seaport Boulevard, Suite 500, Redwood City, CA 94063 | (888) 274-0929
Thursday, November 10, 11
Redmond has given us malware, but has done some things right as well.
32. Recap
O_NONBLOCK is a guideline only
Invoked in user space
Data transfer in user space
Blocking and nonblocking I/O terms
Wildfire Interactive, Inc. | 1600 Seaport Boulevard, Suite 500, Redwood City, CA 94063 | (888) 274-0929
Thursday, November 10, 11
Moving forward, we shall prefer the terms blocking and non-blocking I/O as the work done transferring data from
Kernel to User space
buffers is still done in user space. Even if we do change descriptor behavior with the O_NONBLOCK flag, it's merely
a guideline for the
caller to choose a strategy as the intent and additional context is known there.
33. Challenges of scale
Large number of file descriptors
Respond to descriptor state changes
Execute domain logic
Wildfire Interactive, Inc. | 1600 Seaport Boulevard, Suite 500, Redwood City, CA 94063 | (888) 274-0929
Thursday, November 10, 11
SINGLE FDs until now. Remember, the primary uses case is increased throughput. The main challenge here is
handling a large number of file descriptors (representing client connections) efficiently in a single process.
Previous examples represented single descriptors only.
34. Multiplexed I/O
Who's familiar with select,
poll, epoll or kqueue ?
Wildfire Interactive, Inc. | 1600 Seaport Boulevard, Suite 500, Redwood City, CA 94063 | (888) 274-0929
Thursday, November 10, 11
35. Multiplexed I/O
Nonblocking I/O is inefficient
Retry on EAGAIN is polling
Multiplexed I/O: concurrent blocking
Notified of state changes on any
registered fd
Wildfire Interactive, Inc. | 1600 Seaport Boulevard, Suite 500, Redwood City, CA 94063 | (888) 274-0929
Thursday, November 10, 11
Nonblocking I/O is inefficient - we continuously need to retry on EAGAIN error conditions and as such waste
resources polling. Multiplexed I/O allows an application to concurrently block on a set of file descriptors
(thousands with epoll and kqueue) and receive notification when any of them are in a readable or writable state.
We aim to block on a set of descriptors, not just 1.
36. Multiplexed I/O
fd 5
fd 6
fd 7
fd 8
I/O bound
App Multiplexer
fd 9
fd 10
fd 11
fd 12
User space Kernel space
Wildfire Interactive, Inc. | 1600 Seaport Boulevard, Suite 500, Redwood City, CA 94063 | (888) 274-0929
Thursday, November 10, 11
At any given time a small subset of the descriptors registered with the multiplexer would be in a readable or
writable state.
37. Multiplexed I/O
1. Tell me when fds 1..200
are ready for a read or a
write
Wildfire Interactive, Inc. | 1600 Seaport Boulevard, Suite 500, Redwood City, CA 94063 | (888) 274-0929
Thursday, November 10, 11
38. Multiplexed I/O
2. Do other work until one
of them's ready
Wildfire Interactive, Inc. | 1600 Seaport Boulevard, Suite 500, Redwood City, CA 94063 | (888) 274-0929
Thursday, November 10, 11
39. Multiplexed I/O
3. Get woken up by the
multiplexer on state
changes
Wildfire Interactive, Inc. | 1600 Seaport Boulevard, Suite 500, Redwood City, CA 94063 | (888) 274-0929
Thursday, November 10, 11
40. Multiplexed I/O
4. Which of fd set 1..200
are in a ready state ?
Wildfire Interactive, Inc. | 1600 Seaport Boulevard, Suite 500, Redwood City, CA 94063 | (888) 274-0929
Thursday, November 10, 11
41. Multiplexed I/O
5. Handle all fds ready for
I/O
Wildfire Interactive, Inc. | 1600 Seaport Boulevard, Suite 500, Redwood City, CA 94063 | (888) 274-0929
Thursday, November 10, 11
Repeat. There will almost always just be a small subset of fds ready to drive work from. The more we have, the
smaller this total : ready ratio generally becomes.
42. Checkpoint.
Questions ?
Wildfire Interactive, Inc. | 1600 Seaport Boulevard, Suite 500, Redwood City, CA 94063 | (888) 274-0929
Thursday, November 10, 11
43. ETOOCOMPLEX
Wildfire Interactive, Inc. | 1600 Seaport Boulevard, Suite 500, Redwood City, CA 94063 | (888) 274-0929
Thursday, November 10, 11
You don’t need a UNIX beard in your department. And you should be building apps.
http://www.flickr.com/photos/taniwha/2237514055/
44. There’s a better way
http://zeromq.org
Wildfire Interactive, Inc. | 1600 Seaport Boulevard, Suite 500, Redwood City, CA 94063 | (888) 274-0929
Thursday, November 10, 11
Recently I’ve spent some free time on a new set of Ruby bindings for ZeroMQ - a communications and
concurrency framework. This framework is going to change the way we do I/O moving forward - plans for Linux
Kernel integration.
45. Super Sockets
Scripting for sockets
Wildfire Interactive, Inc. | 1600 Seaport Boulevard, Suite 500, Redwood City, CA 94063 | (888) 274-0929
Thursday, November 10, 11
We spend copious amounts of time implementing business logic, refactoring and often don’t think of how
individual components will communicate. Requirements also change over time - sockets should be “scriptable” as
well. Not an out the box technology - provides the building blocks for solutions. Like legos. Especially important
in service integration - connect with Facebook, Twitter etc. Social software. Crappy communication patterns with
third party integrations is much more likely to affect your performance profiles than inefficient code paths
deployed from within your organization.
46. BSD sockets + Messaging patterns
REQ PUB PULL PULL
REP SUB SUB PUSH
Wildfire Interactive, Inc. | 1600 Seaport Boulevard, Suite 500, Redwood City, CA 94063 | (888) 274-0929
Thursday, November 10, 11
Maps to messaging patterns
Request-Reply, Publish-Subscribe, Pipeline
Prefers mailboxes to buffers and is asynchronous by design ( Actor Pattern )
Build out topologies just in time.
Allows us to focus on interfaces and patterns first
47. Transport agnostic
inproc (Threads)
IPC
TCP/IP
Multicast
Uniform API
Wildfire Interactive, Inc. | 1600 Seaport Boulevard, Suite 500, Redwood City, CA 94063 | (888) 274-0929
Thursday, November 10, 11
Scales to X cores: threads in a process, processes on a box, boxes on a network
Same API regardless of transport
Operations can deploy a single process on first service iteration, using threads to utilize cores. If the engineering
team’s weak with multithreaded programming ( or the codebase is too complex ), can move to IPC transport, which
requires managing more processes but allows developers to code for a single thread of execution. If and when
capacity is required, a TCP transport allows for scaling to multiple boxes.
48. Brokerless
Wildfire Interactive, Inc. | 1600 Seaport Boulevard, Suite 500, Redwood City, CA 94063 | (888) 274-0929
Thursday, November 10, 11
Broker is always a SPOF
Less admin overhead to not rely on message broker.
Any intermediate is slow - most brokers touch disk extensively for durability
Sometimes platforms are built around the capabilities of a particular broker implementation
http://www.flickr.com/photos/epsos/5591761716/
49. FAST
methodmissing:rbczmq lourens$ ruby perf/local_thr.rb tcp://
127.0.0.1:5020 10 1000000
message size: 10 [B]
message count: 1000000
mean throughput: 410276 [msg/s]
mean throughput: 32.822 [Mb/s]
methodmissing:rbczmq lourens$ ruby perf/local_thr.rb tcp://
127.0.0.1:5020 1024 1000000
message size: 1024 [B]
message count: 1000000
mean throughput: 176342 [msg/s]
mean throughput: 1444.595 [Mb/s]
methodmissing:rbczmq lourens$ ruby -v
ruby 1.9.2p290 (2011-07-09 revision 32553) [x86_64-
darwin11.0.1]
Wildfire Interactive, Inc. | 1600 Seaport Boulevard, Suite 500, Redwood City, CA 94063 | (888) 274-0929
Thursday, November 10, 11
Minimal latency - higher than TCP (which is normal), BUT higher throuhgput than TCP ( batching )
Communications and concurrency framework - can load balance between threads and processes as well
Socket options allow for tuning individual sockets - not system wide as per raw sockets
50. Resilient
Auto-reconnects
Durable sockets
Socket options: backlog, HWM, SWAP
Clients can come up before servers
Atomic message delivery
Wildfire Interactive, Inc. | 1600 Seaport Boulevard, Suite 500, Redwood City, CA 94063 | (888) 274-0929
Thursday, November 10, 11
Manage a socket’s ability to handle variable throughput and not drop messages with the backlog, HWM and SWAP
socket options.
Easy for upgrades if we don’t need to always coerce things in sequence.
Fault tolerant message passing - atomic - you either get the message (blob), or you don’t
51. Interjection principle
REQ REQ REQ
ROUTER
DEALER
REP REP REP
Wildfire Interactive, Inc. | 1600 Seaport Boulevard, Suite 500, Redwood City, CA 94063 | (888) 274-0929
Thursday, November 10, 11
Interjection principle states that inserting an intermediary node into the topology should not change the
behavior at the endpoints.
DEALER / ROUTER pair allows for non-blocking request/response. Great for slow / fast clients alike.
52. X Platform
Multiple OS’s
Multiple languages - clients ecosystem
No message contracts - BLOBS
Wildfire Interactive, Inc. | 1600 Seaport Boulevard, Suite 500, Redwood City, CA 94063 | (888) 274-0929
Thursday, November 10, 11
A good clients ecosystem is so often overlooked. There’s a Syslog module for easily tapping into event streams as
needed as well.
53. ZMQ efficiency
Brokerless (or roll your own)
Easy to re-architect
Swap transport to scale to more cores
Resiliency
Specialize as needed with socket
options
Wildfire Interactive, Inc. | 1600 Seaport Boulevard, Suite 500, Redwood City, CA 94063 | (888) 274-0929
Thursday, November 10, 11
54. Working with multiple buffers
Write data from multiple
buffers to a single stream ...
b1
b2
b3
Wildfire Interactive, Inc. | 1600 Seaport Boulevard, Suite 500, Redwood City, CA 94063 | (888) 274-0929
Thursday, November 10, 11
55. ... or read data from a single
stream to multiple buffers.
b1
b2
b3
Wildfire Interactive, Inc. | 1600 Seaport Boulevard, Suite 500, Redwood City, CA 94063 | (888) 274-0929
Thursday, November 10, 11
56. Vectored I/O
ssize_t writev(int fildes,
const struct iovec *iov,
int iovcnt);
ssize_t readv(int fildes,
const struct iovec *iov,
int iovcnt);
Wildfire Interactive, Inc. | 1600 Seaport Boulevard, Suite 500, Redwood City, CA 94063 | (888) 274-0929
Thursday, November 10, 11
57. Benefits
Atomic - reads / writes all or nothing
Reduces the amount of syscalls
Fixed sized headers
Database records
Well supported
Wildfire Interactive, Inc. | 1600 Seaport Boulevard, Suite 500, Redwood City, CA 94063 | (888) 274-0929
Thursday, November 10, 11
Reduces the amount of system calls required to read / write data from non-contiguous buffers
58. Write example
b1
b2
Wildfire Interactive, Inc. | 1600 Seaport Boulevard, Suite 500, Redwood City, CA 94063 | (888) 274-0929
Thursday, November 10, 11
59. Call write for each buffer
write
b1
write
b2
Wildfire Interactive, Inc. | 1600 Seaport Boulevard, Suite 500, Redwood City, CA 94063 | (888) 274-0929
Thursday, November 10, 11
60. Allocate buffer large enough for both
buffers, copy data and write once
copy
b1 write
b3
copy
b2
Wildfire Interactive, Inc. | 1600 Seaport Boulevard, Suite 500, Redwood City, CA 94063 | (888) 274-0929
Thursday, November 10, 11
61. Call writev once to output both buffers
b1 writev
b2
Wildfire Interactive, Inc. | 1600 Seaport Boulevard, Suite 500, Redwood City, CA 94063 | (888) 274-0929
Thursday, November 10, 11
62. ZeroMQ message batching
Higher latency than TCP,
but also higher throughput.
Wildfire Interactive, Inc. | 1600 Seaport Boulevard, Suite 500, Redwood City, CA 94063 | (888) 274-0929
Thursday, November 10, 11
63. Message batching
Sending two or more
messages down the
networking stack is faster
than individual messages.
Wildfire Interactive, Inc. | 1600 Seaport Boulevard, Suite 500, Redwood City, CA 94063 | (888) 274-0929
Thursday, November 10, 11
64. Characteristics
Opportunistic batching
NIC asks 0mq for messages
Sends all available at any given time
Low latency for single VS multiple
messages use case
Wildfire Interactive, Inc. | 1600 Seaport Boulevard, Suite 500, Redwood City, CA 94063 | (888) 274-0929
Thursday, November 10, 11
65. Zero-Copy sends
Saves on memcpy()
Delegate ownership of a buffer to omq
Scripting language strings
Callback function on send
Tricky with Garbage Collection
Wildfire Interactive, Inc. | 1600 Seaport Boulevard, Suite 500, Redwood City, CA 94063 | (888) 274-0929
Thursday, November 10, 11
66. void free_msg(void *data, void *s);
zmq_msg_init_data(&m, RSTRING_PTR(s),
RSTRING_LEN(s), free_msg, (void*)s);
rb_gc_register_address(&s);
zmq_send(s, &m);
void free_msg(void *data, void *s){
rb_gc_unregister_address(&s);
}
Wildfire Interactive, Inc. | 1600 Seaport Boulevard, Suite 500, Redwood City, CA 94063 | (888) 274-0929
Thursday, November 10, 11
67. "message"
"message" "message"
callback() "message"
"message"
Ruby ZeroMQ
Wildfire Interactive, Inc. | 1600 Seaport Boulevard, Suite 500, Redwood City, CA 94063 | (888) 274-0929
Thursday, November 10, 11
Message in Ruby VM VS message in ZeroMQ
68. Passing thoughts
System calls
Blocking, nonblocking and async I/O
Messaging patterns ftw!
Efficiency is context dependent
Read http://zguide.zeromq.org
Reread http://zguide.zeromq.org
Wildfire Interactive, Inc. | 1600 Seaport Boulevard, Suite 500, Redwood City, CA 94063 | (888) 274-0929
Thursday, November 10, 11
69. Questions ?
Wildfire Interactive, Inc. | 1600 Seaport Boulevard, Suite 500, Redwood City, CA 94063 | (888) 274-0929
Thursday, November 10, 11
70. Wildfire Interactive, Inc. is hiring
Wildfire Interactive, Inc. | 1600 Seaport Boulevard, Suite 500, Redwood City, CA 94063 | (888) 274-0929
Thursday, November 10, 11
71. Thanks !
http://www.jobscore.com/jobs/
wildfireapp
follow @methodmissing
fork github.com/methodmissing
Wildfire Interactive, Inc. | 1600 Seaport Boulevard, Suite 500, Redwood City, CA 94063 | (888) 274-0929
Thursday, November 10, 11