Daniel Stenberg's talk at FSCONS 2010 about client-side scalable application layer transfers

1. Scalable application layer transfers

2. Daniel Stenberg
●
Free Software
●
Network hacker, code and IETF
●
Embedded developer
●
Consultant
Email: daniel@haxx.se
Twitter: @bagder
Web: daniel.haxx.se
Blog: daniel.haxx.se/blog

3. Agenda
●
Traditional network client
●
… applied to libcurl
●
C10K
●
… applied to libcurl

4. Protocol stack
Contents
Application
Transport
Internet
Link

5. libcurl
●
curl.haxx.se
●
Very portable C library
●
Application protocol transfers: DICT, FILE, FTP, FTPS,
GOPHER, HTTP, HTTPS, IMAP, IMAPS, LDAP, LDAPS, POP3,
POP3S, RTMP, RTSP, SCP, SFTP, SMTP, SMTPS, TELNET and
TFTP
●
Roots from 1997
●
MIT licensed
●
Daniel is lead developer

6. A typical client
●
Create socket
●
Connect socket
●
Select() for data to read or write
●
Read or write data
●
Loop

7. pseudo C code
WARNING
the following source code
examples are not complete and
they resemble C code

8. int main(int argc, char *argv[])
{
int sock; /* Socket descriptor */
if ((sock = socket(PF_INET, SOCK_STREAM, IPPROTO_TCP)) < 0)
DieWithError("socket() failed");
/* it would resolve the host name here */
if (connect(sock to server))
DieWithError("connect() failed");
do {
select(maxfd+1, sock, …); /* wait for socket to become readable
or writable */
if(check sock if writable)
send(sock, string, string_len, 0);
if(check sock if readable)
recv(sock, string, string_len, 0);
} while(everything is fine);
close(sock);
exit(0);
}
School book example
1
2
1
3
4
5

9. Simple case works fine
●
Easy to read
●
Fast to write
●
Easy to learn and teach

10. int main(int argc, char *argv[])
{
int sock[NUM_CONNS]; /* Socket descriptors */
for(i=0; i< NUM_CONNS; i++) {
if ((sock[i] = socket(PF_INET, SOCK_STREAM, IPPROTO_TCP)) < 0)
DieWithError("socket() failed");
if (connect(sock[i] to server[i]))
DieWithError("connect() failed");
do {
/* set up the bitmasks for select() */
select(maxfd+1, sock, …); /* wait for any socket to become readable
or writable */
for(i=0; i< NUM_CONNS; i++) {
if(check sock[i] if writable)
send(sock, string, string_len, 0);
if(check sock[i] if readable)
recv(sock, string, string_len);
}
} while(everything is fine);
close(sock);
exit(0);
}
More connections
1
2
1
3
4
5

11. int main(int argc, char *argv[])
{
int sock[NUM_CONNS]; /* Socket descriptors */
for(i=0; i< NUM_CONNS; i++) {
if ((sock[i] = socket(PF_INET, SOCK_STREAM, IPPROTO_TCP)) < 0)
DieWithError("socket() failed");
if (connect(sock[i] to server[i]))
DieWithError("connect() failed");
do {
/* set up the bitmasks for select() */
select(maxfd+1, sock, …); /* wait for any socket to become readable
or writable */
for(i=0; i< NUM_CONNS; i++) {
if(check sock[i] if writable)
send(sock, string, string_len, 0);
if(check sock[i] if readable)
recv(sock, string, string_len);
}
} while(everything is fine);
close(sock);
exit(0);
}
More connections

12. Many connections
●
Quickly degrades
●
select() and poll() suck
●
select() slightly more due to
FD_SETSIZE
●
Threads help to some degree,
but...

13. what about libcurl
●
examples in C
●
has bindings for 40 languages

14. libcurl simple app
int main(void)
{
CURL *curl;
CURLcode res;
curl = curl_easy_init();
if(curl) {
curl_easy_setopt(curl, CURLOPT_URL, "http://example.com");
res = curl_easy_perform(curl);
/* always cleanup */
curl_easy_cleanup(curl);
}
return 0;
}

15. easy interface
●
easy to write
●
synchronous
●
supports many protocols in one
go
●
made for single transfers

16. libcurl multi app
int main(int argc, char **argv)
{
handle1 = curl_easy_init();
handle2 = curl_easy_init();
curl_easy_setopt(handle1, CURLOPT_URL, "http://www.example.com/");
curl_easy_setopt(handle2, CURLOPT_URL, "http://fscons.org/");
multi_handle = curl_multi_init();
curl_multi_add_handle(multi_handle, handle1);
curl_multi_add_handle(multi_handle, handle2);
curl_multi_perform(multi_handle, &still_running);
while(still_running) {
curl_multi_timeout(multi_handle, &curl_timeo);
curl_multi_fdset(multi_handle, &fdread, &fdwrite, &fdexcep, &maxfd);
rc = select(maxfd+1, &fdread, &fdwrite, &fdexcep, &timeout);
if(rc says timeout or readable/writable sockets)
curl_multi_perform(multi_handle, &still_running);
}
curl_multi_cleanup(multi_handle);
curl_easy_cleanup(http_handle);
curl_easy_cleanup(http_handle2);
return 0;
}
1
2
5
4
6
3
7
8

17. libcurl multi app
int main(int argc, char **argv)
{
handle1 = curl_easy_init();
handle2 = curl_easy_init();
curl_easy_setopt(handle1, CURLOPT_URL, "http://www.example.com/");
curl_easy_setopt(handle2, CURLOPT_URL, "http://fscons.org/");
multi_handle = curl_multi_init();
curl_multi_add_handle(multi_handle, handle1);
curl_multi_add_handle(multi_handle, handle2);
curl_multi_perform(multi_handle, &still_running);
while(still_running) {
curl_multi_timeout(multi_handle, &curl_timeo);
curl_multi_fdset(multi_handle, &fdread, &fdwrite, &fdexcep, &maxfd);
rc = select(maxfd+1, &fdread, &fdwrite, &fdexcep, &timeout);
if(rc says timeout or readable/writable sockets)
curl_multi_perform(multi_handle, &still_running);
}
curl_multi_cleanup(multi_handle);
curl_easy_cleanup(http_handle);
curl_easy_cleanup(http_handle2);
return 0;
}

18. multi interface
●
several “easy” handles dealt with
combined
●
made for simultaneous transfers
●
still easy to write and read
●
but: relies on select()

19. select with many

20. Scale to c10K
●
10000 simultaneous connections
●
not as easy as it may sound

21. A better way
●
event­based
●
react only on the sockets that
have changed actions
●
no “check for action”
●
no loops

22. really scalable

23. not that portable
●
There's no POSIX for event­based
●
The portable libs don't support all
platforms
●
epoll (Linux), kqueue (FreeBSD, NetBSD,
OpenBSD, Darwin), /dev/poll (Solaris, HPUX),
pollset (AIX), Event Completion (Solaris 10), I/O
Completion Ports (Microsoft Windows)

24. libev and libevent
●
potent event libraries we base the
examples on in this presentation
●
they support a subset of platforms
●
the examples are not complete now
either

25. event­based basics
●
detail what sockets we work with
and what do we wait for on the
sockets
●
timeouts
●
wait until something happens

26. simple event client
int function(int socketfd)
{
ev = ev_default_loop();
ev_timer_init(..., timer_callback, …);
ev_io_init(ev, event_callback, socketfd, WHAT);
ev_loop(ev); /* sit here until done */
}
int timer_callback(...)
{
/* timer expired! */
}
int event_callback(..., int socketfd, ...)
{
/* something happened on my socket, now we
recv or send etc */
}

27. event based compared
●
different code­flow
●
might require a different state
machine for the protocol
●
again: not widely portable

28. Other ways to avoid
wrongs
●
one thread per connection with
blocking sockets
●
threads with non­blocking i/o to
handle N connections each

29. using libcurl scalable
●
“multi_socket” API added 2006
●
tells the app what sockets to wait for
what on
●
app then speaks with event library
●
avoids select()
●
event system agnostic

30. libcurl multi_socket
bindings
●
not always supported
●
a matter of writing the glue code
correct – you can help!

31. multi_socket example
int main()
{
CURLM *multi = curl_multi_init();
CURL *easy;
curl_multi_setopt(..., CURLMOPT_SOCKETFUNCTION, socket_cb);
curl_multi_setopt(..., CURLMOPT_TIMERFUNCTION, timer_cb);
easy = curl_easy_init();
curl_easy_setopt(..., CURLOPT_URL, …);
curl_multi_add_handle(..., easy);
event_dispatch(); /* start the libevent dispatch
loop */
}
int event_cb(...)
{
/* event library found event on libcurl's socket, act */
curl_multi_socket_action(..., action, ...);
}
int socket_cb(...)
{
/* called when libcurl adds, removes or changes a socket that
we must deal with */
event_set(..., event_cb); /* tell event lib about setup */
}

32. Scales?
●
done 70,000+ connections
●
port numbers are 16 bit
●
callbacks take time thus limit
throughput

33. Which protocols?
●
(almost) all network based ones
libcurl support! (DICT, FTP, FTPS, GOPHER,
HTTP, HTTPS, IMAP, IMAPS, LDAP, LDAPS, POP3,
POP3S, RTMP, RTSP, SCP, SFTP, SMTP, SMTPS and
TFTP)

34. Caveats?
●
yeah sure, but...
●
c­ares for name resolving
●
not everything is non­blocking

35. More factors
●
Network stack inefficiency
●
Stack size?
●
Memory / mallocs
●
“Other” code may slow down things
●
Copy copy copy copy ...

36. For you who fell asleep
●
old style socket applications
don't scale
●
use event based concepts
●
libcurl can too

37. libcurl is...
●
entirely open source
●
in need of your help!
●
at http://curl.haxx.se/libcurl/