How to build a High Performance PSGI/Plack Server PSGI/Plack・Monocerosで学ぶ ハイパフォーマンス Webアプリケーションサーバの作り方

1. How to build a High Performance PSGI/Plack Server
PSGI/Plack・Monocerosで学ぶ
ハイパフォーマンス
Webアプリケーションサーバの作り方
YAPC::Asia 2013 Tokyo
Masahiro Nagano / @kazeburo

2. Me
• 長野雅広 Masahiro Nagano
• @kazeburo
• PAUSE:KAZEBURO
• Operations Engineer, Site Reliability
• LINE Corp.
Development support
LINE Family, livedoor

3. livedoorBlog
One of the largest Blog Hosting Service in Japan

4. livedoorBlog uses

5. Perl (5.16and 5.8)

6. Carton

7. Plack/PSGI and mod_perl

8. $ curl -I http://blog.livedoor.jp/staff/| grep Server
Server: Plack::Handler::Starlet

9. Starlet handles
1 Billion(10億) reqs/day

10. To get over
this burst traffic,
We need to improve
Performance
across all layers
この負荷を乗り切るために様々なレイヤーで最適化をしています

11. Layers
Hardware / Network
OS
App Server
Routing
Cache Logic
SQL
Template Engine
RDBMS
Cached
Web Server

12. Hardware / Network
OS
Routing
Cache Logic
SQL
Template Engine
RDBMS
Cached
Web Server
Today’s Topic

13. Hardware / Network
OS
Routing
Cache Logic
SQL
Template Engine
RDBMS
Cached
Web Server
Today’s Topic
App Server

14. By the way..

15. “Open & Share”
is our driver
for excellence.
And LOVE CPAN/OSS
Open & Share は私たちの目指すところです。
CPAN/OSSを多く使い、また貢献もしています

16. Improving
Performance of livedoorBlog
directly linked
Performance of Plack/Starlet
on CPAN
livedoorBlogのパフォーマンス改善で行った事は
CPAN上のPlack/Starletにも当然影響してきます

17. 0
3500
7000
10500
14000 13083
6241
“Hello World”Reqs/Sec
Plack 1.0016 1.0029
Starlet 0.16 0.20
2013/02 2013/09
mod_perl era plack era

18. Monoceros

19. Monoceros is
a yet another
Plack/PSGI Server
for Performance

20. the Goal

21. Reduce
TCP 3way hand shake
between
Proxy and PSGI Server

22. Reverse
Proxy
App
Server
GET / HTTP/1.1
Host: example.com
SYN
ACK
SYN+ACK
HTTP/1.1 200 OK
Content-Type: text/html
FIN
ACK
GET /favicon.ico HTTP/1.1
Host: example.com
SYN
ACK
SYN+ACK
HTTP/1.1 404 NOT FOUND
Content-Type: text/html
FIN
ACK

23. HTTP/1.0-1.1 have
KeepAlive

24. Reverse
Proxy
App
Server
GET / HTTP/1.0
Host: example.com
Connection: keep-alive
SYN
ACK
SYN+ACK
HTTP/1.0 200 OK
Content-Type: text/html
Connection: keep-alive
Content-Length: 941
GET /favicon.ico HTTP/1.1
Host: example.com
HTTP/1.1 200 OK
Content-Type: image/vnd.microsoft.icon
Transfer-Encoding: chunked
GET /site.css HTTP/1.1
Host: example.com
HTTP/1.1 200 OK
Content-Type: text/css
Content-Length: 1013

25. C10K problem

26. nginx
C10K
Ready
Reverse
Proxy
nginx
nginx
Starlet
Starman
App
Server
KeepAlive Req
KeepAlive Req
KeepAlive Req

27. Starman, Starlet’s
Preforking model requires
1 connection per 1 process
By default
Starman: 5 procs
Starlet: 10 procs

28. Monoceros adopts
Preforking model,
But C10K ready

29. Worker
Process
Worker
Process
Worker
Process
Worker
Process
Manager
Process
SOCK
Client
GET / HTTP/1.1
Host: example.com
200 OK
Content-Type: text/html

30. Worker
Process
Worker
Process
Worker
Process
Worker
Process
Manager
Process
SOCK
Client
GET / HTTP/1.1
Host: example.com
200 OK
Content-Type: text/html

31. Worker
Process
Worker
Process
Worker
Process
Worker
Process
Manager
Process
SOCK
Client
GET / HTTP/1.1
Host: example.com
200 OK
Content-Type: text/html
GET / HTTP/1.1
Host: example.com
Event Driven

32. Worker
Process
Worker
Process
Worker
Process
Worker
Process
Manager
Process
SOCK
Client
GET / HTTP/1.1
Host: example.com
200 OK
Content-Type: text/html
GET / HTTP/1.1
Host: example.com
200 OK
Content-Type: text/html
Event Driven

33. Monoceros Workers
that inherits“Starlet”
not C10K ready

34. Event Driven
Manager Process
C10K ready
built with AnyEvent

35. KeepAlive Benchmark
like a Browser
1) connect
2) do requests certain number
3) leave alone a socket
4) timeout and close

36. 250 conn / 200 reqs
Starlet Monoceros
Total time (sec) 54.51 8.74
Failed reqs 971 0

37. Plack/PSGI
Basics

38. PSGI = specification
Plack = implementation

39. PSGI Interface
my $app = sub {
my $env = shift;
...
return [200,
[‘Content-Type’ => ‘text/html’],
[‘Hello World’]
];
};

40. PSGI environment hash
* CGI keys
REQUEST_METHOD,SCRIPT_NAME,
PATH_INFO,REQUEST_URI,
QUERY_STRING,SERVER_PROTOCOL,HTTP_*
* PSGI-specific keys
psgi.version, psgi.url_scheme,
psgi.input, psgi.errors,
psgi.multiprocess, psgi.streaming
psgi.nonblocking
$env->{...}

41. PSGI Response (1)
ArrayRef
[200, #status code
[
‘Content-Type’ => ‘text/html’,
‘Content-Length => 10
],
[
‘Hello’,
‘World’
]
];

42. PSGI Response (1’)
arrayref+IO::Handle
open my $fh, ‘<’, ‘/path/icon.jpg’;
[200,
[
‘Content-Type’ => ‘image/jpeg’,
‘Content-Length => 123456789
],
$fh
];

43. PSGI Response (2)
Delayed and Streaming
sub {
my $env = shift;
return sub {
my $responder = shift;
...
$responder->([ 200,
$headers, [$body] ]);
}
};

44. PSGI Response (2’)
Delayed and Streaming
return sub {
my $responder = shift;
my $writer = $responder->([200, $headers]);
wait_for_events(sub {
my $new_event = shift;
if ($new_event) {
$writer->write($new_event->as_json . "n");
} else {
$writer->close;
}
});
};

45. Role of“PSGI Server”

46. PSGI Server
“A PSGI Server is a Perl program
providing an environment for a PSGI
application to run in”

47. PSGI
Server
App
$env
$res
Apache
Nginx
Apache
Proxy
Browser
CGI
mod_perl
FCGI
HTTP
Perl direct

48. PSGI Server
is called
“Plack Handler”

49. Plack Handler
Adaptor interface Plack and PSGI Server.
Make PSGI Server to run
with“plackup”

50. e.g. Starman
Starman::Server
= PSGI Server
Plack::Handler::Starman
= Plack Handler

51. Make
PSGI/Plack
Server
a High Performance

52. Tiny Standalone
PSGI Web Server

53. my $null_io = do { open my $io, "<", ""; $io };
my $app = sub {
my $env = shift
return [200,['Content-Type'=>'text/html'],['Hello','World',"n"]];
};
my $listen = IO::Socket::INET->new(
Listen => 5,
LocalAddr => 'localhost',
LocalPort => 5000,
ReuseAddr => 1,
);
while ( my $conn = $listen->accept ) {
my $env = {
SERVER_PORT => '5000',
SERVER_NAME => 'localhost',
SCRIPT_NAME => '',
REMOTE_ADDR => $conn->peerhost,
'psgi.version' => [ 1, 1 ],
'psgi.errors' => *STDERR,
'psgi.url_scheme' => 'http',
'psgi.run_once' => Plack::Util::FALSE,
'psgi.multithread' => Plack::Util::FALSE,
'psgi.multiprocess' => Plack::Util::FALSE,
'psgi.streaming' => Plack::Util::FALSE,
'psgi.nonblocking' => Plack::Util::FALSE,
'psgi.input' => $null_io,
};
$conn->sysread( my $buf, 4096);
my $reqlen = Plack::HTTPParser::parse_http_request($buf, $env);
my $res = Plack::Util::run_app $app, $env;
my @lines = ("HTTP/1.1 $res->[0] @{[ status_message($res->[0]) ]}015012");
for (my $i = 0; $i < @{$res->[1]}; $i += 2) {
next if $res->[1][$i] eq 'Connection';
push @lines, "$res->[1][$i]: $res->[1][$i + 1]015012";
}
push @lines, "Connection: close0151201512";
$conn->syswrite(join "",@lines);
Plack::Util::foreach($res->[2], sub {
$conn->syswrite(shift);
});
$conn->close;
}
60 lines

54. Listen and Accept

55. my $listen = IO::Socket::INET->new(
Listen => 5,
LocalAddr => 'localhost',
LocalPort => 5000,
ReuseAddr => 1,
);
while ( my $conn = $listen->accept ) {
...
}

56. Read a request

57. use Plack::HTTPParser qw/parse_http_request/;
my $null_io = do { open my $io, "<", ""; $io };
while ( my $conn = $listen->accept ) {
my $env = {
SERVER_PORT => '5000',
SERVER_NAME => 'localhost',
SCRIPT_NAME => '',
REMOTE_ADDR => $conn->peerhost,
'psgi.version' => [ 1, 1 ],
'psgi.errors' => *STDERR,
'psgi.url_scheme' => 'http',
'psgi.multiprocess' => Plack::Util::FALSE,
'psgi.streaming' => Plack::Util::FALSE,
'psgi.nonblocking' => Plack::Util::FALSE,
'psgi.input' => $null_io,
};
$conn->sysread(my $buf, 4096);
my $reqlen = parse_http_request($buf, $env);

58. Run App

59. my $res = $app->($env);
or
my $res = Plack::Util::run_app $app, $env;

60. Write a response

61. use HTTP::Status qw/status_message/;
my $res = ..
my @lines = ("HTTP/1.1 $res->[0]
@{[ status_message($res->[0]) ]}015012");
for (my $i = 0; $i < @{$res->[1]}; $i += 2) {
next if $res->[1][$i] eq 'Connection';
push @lines, "$res->[1][$i]: $res->[1][$i + 1]015012";
}
push @lines, "Connection: close0151201512";
$conn->syswrite(join "",@lines);
foreach my $buf ( @{$res->[2]} ) {
$conn->syswrite($buf);
});
$conn->close;

62. This PSGI Server
has some problem
* handle only one at once
* no timeout
* may not fast

63. Increase concurrency

64. Multi Process
IO Multiplexing
or
Both

65. Preforking model
Simple, Scaling

66. Manager

67. Manager
bind
listen

68. Worker
accept
Worker
accept
Worker
accept
Worker
accept
Manager
bind
listen
fork fork fork fork

69. Worker
accept
Worker
accept
Worker
accept
Worker
accept
Manager
bind
listen
fork fork fork fork
Client Client ClientClient

70. use Parallel::Prefork;
my $listen = IO::Socket::INET->new(
Listen => 5,
LocalAddr => 'localhost',
LocalPort => 5000,
ReuseAddr => 1,
);
my $pm = Parallel::Prefork->new({
max_workers => 5,
trap_signals => {
TERM => 'TERM',
HUP => 'TERM',
}
});
while ( $pm->signal_received ne 'TERM') {
$pm->start(sub{
while ( my $conn = $listen->accept ) {
my $env = {..}

71. NO Accept
Serialization

72. os/kernel
Worker
accept
Worker
accept
Worker
accept
Worker
accept
Manager
bind
listen
Zzz.. Zzz.. Zzz.. Zzz..

73. os/kernel
Worker
accept
Worker
accept
Worker
accept
Worker
accept
Client
Manager
bind
listen
Zzz.. Zzz.. Zzz.. Zzz..

74. os/kernel
Worker
accept
Worker
accept
Worker
accept
Worker
accept
Client
Manager
bind
listen
Zzz.. Zzz.. Zzz.. Zzz..

75. os/kernel
Worker
accept
Worker
accept
Worker
accept
Worker
accept
Client
Manager
bind
listen
Thundering Herd突然の負荷
WakeUp WakeUp WakeUp WakeUP

76. os/kernel
Worker
accept
Worker
accept
Worker
accept
Worker
accept
Client
Manager
bind
listen
Thundering Herd突然の負荷
WakeUp WakeUp WakeUp WakeUP

77. Thundering Herd
is an old story

78. Worker
accept
Worker
accept
Worker Worker
Manager
bind
listen accept accept
Client
Zzz.. Zzz.. Zzz..Zzz..

79. Worker
accept
Worker
accept
Worker Worker
Manager
bind
listen accept accept
Client
Zzz.. Zzz.. Zzz..Zzz..

80. Worker
accept
Worker
accept
Worker Worker
Manager
bind
listen accept accept
Client
modern os/kernel
Zzz.. Zzz.. Zzz..Zzz..

81. Worker
accept
Worker
accept
Worker Worker
Manager
bind
listen accept accept
Client
modern os/kernel
Zzz.. Zzz.. Zzz..Zzz..

82. Worker
accept
Worker
accept
Worker Worker
Manager
bind
listen accept accept
Client
modern os/kernel
Zzz.. Zzz..Zzz..
WakeUp

83. NO Accept Serialization
(except for multiple interface)

84. TCP_DEFER_ACCEPT

85. Wake up a process
when DATA arrived
not established
コネクションが完了したタイミングではなく、
データが到着した段階でプロセスを起こします

86. client
A
client
B
GET / HTTP/1.0
Host: example.com
Connection: keep-alive
SYN
ACK
SYN+ACK
SYN
ACK
SYN+ACK
GET / HTTP/1.0
Host: example.com
Connection: keep-alive
default defer_accept
Accept
RunApp
block
to
read

87. RunApp
client
A
client
B
GET / HTTP/1.0
Host: example.com
Connection: keep-alive
SYN
ACK
SYN+ACK
SYN
ACK
SYN+ACK
GET / HTTP/1.0
Host: example.com
Connection: keep-alive
default defer_accept
Accept
RunApp
Accept
block
to
read

88. RunApp
client
A
client
B
GET / HTTP/1.0
Host: example.com
Connection: keep-alive
SYN
ACK
SYN+ACK
SYN
ACK
SYN+ACK
GET / HTTP/1.0
Host: example.com
Connection: keep-alive
default defer_accept
Accept
RunApp
Accept
block
to
read
idle

89. RunApp
client
A
client
B
GET / HTTP/1.0
Host: example.com
Connection: keep-alive
SYN
ACK
SYN+ACK
SYN
ACK
SYN+ACK
GET / HTTP/1.0
Host: example.com
Connection: keep-alive
default defer_accept
Accept
RunApp
Accept
RunApp
Accept
RunApp
Accept
block
to
read
idle

90. use Socket qw(IPPROTO_TCP);
my $listen = IO::Socket::INET->new(
Listen => 5,
LocalAddr => 'localhost',
LocalPort => 5000,
ReuseAddr => 1,
);
if ($^O eq 'linux') {
setsockopt($listen, IPPROTO_TCP, 9, 1);
}

91. timeout to
read header

92. alarm

93. my $READ_TIMEOUT = 5;
eval {
local $SIG{ALRM} = sub { die "Timed outn"; };
alarm( $READ_TIMEOUT );
$conn->sysread(my $buf, 4096);
};
alarm(0);
next if ( $@ && $@ =~ /Timed out/ );
my $reqlen = parse_http_request($buf, $env);

94. nonblocking + select

95. use IO::Select;
my $READ_TIMEOUT = 5;
while( my $conn = $listen->accept ) {
$conn->blocking(0);
my $select = IO::Select->new($conn);
my @ready = $select->can_read($READ_TIMEOUT);
next unless @ready;
$conn->sysread($buf, 4096);
my $reqlen = parse_http_request($buf, $env);

96. alarm
vs.
nonblocking + select

97. Fewer syscalls is good

98. Hardwares
User Application
OS/Kernel
system calls
listen,fork, accept, read, write, select, alarm
Worker Worker Worker Worker Worker

99. alarm

100. rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
rt_sigaction(SIGALRM, {0x47e5b0, [], SA_RESTORER,
0x7ff7d6e0cba0}, {SIG_DFL, [], SA_RESTORER,
0x7ff7d6e0cba0}, 8) = 0
rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
alarm(5) = 0
read(7, "GET / HTTP/1.1rnUser-Agent: curl"...,
65536) = 155
rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER,
0x7ff7d6e0cba0}, {0x47e5b0, [], SA_RESTORER,
0x7ff7d6e0cba0}, 8) = 0
rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
alarm(0) = 5 9 syscalls

101. non-blocking + select

102. fcntl(5, F_GETFL) = 0x2 (flags O_RDWR)
fcntl(5, F_SETFL, O_RDWR|O_NONBLOCK) = 0
select(8, [5], NULL, [5], {300, 0}) = 1
(in [5], left {299, 999897})
read(5, "GET / HTTP/1.1rnUser-Agent:
curl"..., 131072) = 155
4 syscalls

103. Parse a request with“C”

104. $ cpanm HTTP::Parser::XS
Plack::HTTPParser uses H::P::XS if installed

105. TCP_NODELAY

106. When data was written
TCP packets does not
immediately send
“TCP uses Nagle's algorithm to
collect small packets
for send all at once by default”

107. write(“foo”)
write(“bar”)
os/kernel
network
interface
Application
buffering
“foobar”

108. write(“foo”)
write(“bar”)
os/kernel
network
interface
Application
“foo”
TCP_NODELAY
“bar”

109. Take care of excessive
fragmentation of TCP
packets

110. Write in once
join content in Server

111. my @lines = ("HTTP/1.1 $res->[0]
@{[ status_message($res->[0]) ]}015012");
for (my $i = 0; $i < @{$res->[1]}; $i += 2) {
next if $res->[1][$i] eq 'Connection';
push @lines,
"$res->[1][$i]: $res->[1][$i + 1]015012";
}
push @lines, "Connection: close0151201512";
$conn->syswrite(join "",@lines, @{$res->[2]});

112. accept4, writev

113. Choose
PSGI/Plack
Server

114. CPAN has many
PSGI Server &
Plack::Hanlder:**

115. Standalone
(HTTP::Server::PSGI)
Default server for plackup
Single process Web Server
For development

116. Starman
Preforking Web Server
HTTP/1.1, HTTPS,
Multiple interfaces,
unix-domain socket,
hot deploy using Server::Starter

117. Starlet
Preforking Web Server
HTTP/1.1(0.20~)
hot deploy using Server::Starter
Simple and Fast

118. Monoceros
C10K Ready Preforking Web Server
HTTP/1.1
hot deploy using Server::Starter

119. Twiggy
based on AnyEvent
nonblocking, streaming
Single Process

120. Twiggy::Prefork
based on Twiggy and
Parallel::Prefork
nonblocking, streaming
Multi Process
hot deploy using Server::Starter

121. Feersum
Web server based on EV/libev
nonblocking, streaming
Single/Multi Process

122. How to choose
PSGI Server

123. Single
Process
Multi
Process
CPU Intensive -
Starlet
Starman
Monoceros
Requires
Event Driven
Twiggy
Feersum
Twiggy::Prefork
Feersum
TypeofWebApplication

124. Finding
Bottlenecks of
Performance

125. use Devel::NYTProf

127. Flame Graph is awesome
Profile nytprof.out.{PID}
for preforking server
$ nytprofhtml -f nytprof.out.1210
$ open nytprof/index.html

128. use strace or dtruss
trace syscalls

129. $ strace -tt -s 200 -p {pid}
2>&1 | tee /tmp/trace.txt

130. Process 30929 attached - interrupt to quit
16:13:46.826828 accept(4, {sa_family=AF_INET, sin_port=htons(43783), sin_addr=inet_addr("127.
16:13:48.916233 ioctl(5, SNDCTL_TMR_TIMEBASE or TCGETS, 0x7fff2fb61730) = -1 EINVAL (Invalid
16:13:48.916392 lseek(5, 0, SEEK_CUR) = -1 ESPIPE (Illegal seek)
16:13:48.916493 ioctl(5, SNDCTL_TMR_TIMEBASE or TCGETS, 0x7fff2fb61730) = -1 EINVAL (Invalid
16:13:48.916573 lseek(5, 0, SEEK_CUR) = -1 ESPIPE (Illegal seek)
16:13:48.916661 fcntl(5, F_SETFD, FD_CLOEXEC) = 0
16:13:48.916873 fcntl(5, F_GETFL) = 0x2 (flags O_RDWR)
16:13:48.916959 fcntl(5, F_SETFL, O_RDWR|O_NONBLOCK) = 0
16:13:48.917095 setsockopt(5, SOL_TCP, TCP_NODELAY, [1], 4) = 0
16:13:48.917362 read(5, "GET / HTTP/1.0rnHost: 127.0.0.1:5005rnUser-Agent: ApacheBench/2.
16:13:48.917613 brk(0x1e8e000) = 0x1e8e000
16:13:48.917746 gettimeofday({1379402028, 917802}, NULL) = 0
16:13:48.917953 write(5, "HTTP/1.1 200 OKrnDate: Tue, 17 Sep 2013 07:13:48 GMTrnServer: P
16:13:48.918187 close(5) = 0
16:13:48.918428 accept(4, {sa_family=AF_INET, sin_port=htons(43793), sin_addr=inet_addr("127.
16:13:48.923736 ioctl(5, SNDCTL_TMR_TIMEBASE or TCGETS, 0x7fff2fb61730) = -1 EINVAL (Invalid
16:13:48.923843 lseek(5, 0, SEEK_CUR) = -1 ESPIPE (Illegal seek)
16:13:48.923924 ioctl(5, SNDCTL_TMR_TIMEBASE or TCGETS, 0x7fff2fb61730) = -1 EINVAL (Invalid
16:13:48.924461 lseek(5, 0, SEEK_CUR) = -1 ESPIPE (Illegal seek)
16:13:48.924600 fcntl(5, F_SETFD, FD_CLOEXEC) = 0
16:13:48.924762 fcntl(5, F_GETFL) = 0x2 (flags O_RDWR)
16:13:48.924853 fcntl(5, F_SETFL, O_RDWR|O_NONBLOCK) = 0
16:13:48.924939 setsockopt(5, SOL_TCP, TCP_NODELAY, [1], 4) = 0
16:13:48.925162 read(5, "GET / HTTP/1.0rnHost: 127.0.0.1:5005rnUser-Agent: ApacheBench/2.
16:13:48.925445 gettimeofday({1379402028, 925494}, NULL) = 0
16:13:48.925629 write(5, "HTTP/1.1 200 OKrnDate: Tue, 17 Sep 2013 07:13:48 GMTrnServer: P
16:13:48.925854 close(5) = 0
16:13:48.926084 accept(4, {sa_family=AF_INET, sin_port=htons(43803), sin_addr=inet_addr("127.
16:13:48.930480 ioctl(5, SNDCTL_TMR_TIMEBASE or TCGETS, 0x7fff2fb61730) = -1 EINVAL (Invalid
16:13:48.930626 lseek(5, 0, SEEK_CUR) = -1 ESPIPE (Illegal seek)
16:13:48.930744 ioctl(5, SNDCTL_TMR_TIMEBASE or TCGETS, 0x7fff2fb61730) = -1 EINVAL (Invalid
16:13:48.930838 lseek(5, 0, SEEK_CUR) = -1 ESPIPE (Illegal seek)
16:13:48.930915 fcntl(5, F_SETFD, FD_CLOEXEC) = 0
16:13:48.931070 fcntl(5, F_GETFL) = 0x2 (flags O_RDWR)

131. in conclusion

132. PSGI Server get Faster.

133. PSGI/Plack Rocks
Stable, Fast
Found problems?
RT, GitHub Issue, PullReqs
IRC #perl @kazeburo

134. #fin. Thank you!