1. HTTP::Parser::XS
writing a fast & secure XS module
Cybozu Labs, Inc.
Kazuho Oku

2. Pros & Cons of XS
Pros
Faster (well, not always, but…)
Cons
Difficult and time-consuming
Higher security risk
Mistakes in C programming leads to remote code
injection
May 28 2010 HTTP::Parser::XS - writing a fast & secure XS module 2

3. The Best Practice, in General
Don’t use XS
Until you hit performance bottleneck
Rewrite the bottleneck (only) using XS
May 28 2010 HTTP::Parser::XS - writing a fast & secure XS module 3

4. HTTP::Parser::XS
May 28 2010 HTTP::Parser::XS - writing a fast & secure XS module 4

5. Summary
An HTTP request parser
Designed and implemented to be simple,
fast & secure
Stateless
PSGI-compatible
Required or recommended by many Plack servers
Starman, Starlet, Twiggy, etc.
May 28 2010 HTTP::Parser::XS - writing a fast & secure XS module 5

6. Two-layered Approach
Picohttpparser
HTTP request / response parser written in C
has its own test suite
does not parse request / response content
request line (response line) and headers only
copy-less
faster, lesser probability of security holes and memory
leaks
HTTP::Parser::XS
glue code to access picohttpparser from perl
May 28 2010 HTTP::Parser::XS - writing a fast & secure XS module 6

7. Speed and Complexity
HTTP::Parser::XS is simple, and fast
most of the time is not spent in picohttpparser,
but in the glue code constructing hashref
picohttpparser/trunk can handle >1Mreqs/sec.
reqs/sec. lines of code
HTTP::HeaderParser::XS 116,000 1,166
HTTP::Parser::XS 140,000 487
Plack::HTTPParser::PP 10,100 104
May 28 2010 HTTP::Parser::XS - writing a fast & secure XS module 7

8. Why is it a Stateless?
Faster and simpler than a stateful parser
lower security risks
most HTTP requests / responses arrive
in a single packet, anyway
if written optimally in C, the cost of re-
parsing is smaller than storing headers
into a perlhashref
May 28 2010 HTTP::Parser::XS - writing a fast & secure XS module 8

9. Why is it Stateless? (cont’d)
Easy to determine the end of an multi-
packet HTTP request
by looking for “rnrn” within the last packet
(and preceding three bytes)
mainly as a countermeasure for Slowloris
May 28 2010 HTTP::Parser::XS - writing a fast & secure XS module 9

10. Consistent Design
Functions take same arguments
 buf – points to current char
 buf_end – points to end of buffer
 *ret – error value
 returns pointer to the next char
 or null on error (the reason will be stored in *ret)
const char* parse_http_version(const char* buf, const char* buf_end,
int* minor_version, int* ret)
{
EXPECT_CHAR('H'); EXPECT_CHAR('T'); EXPECT_CHAR('T');
EXPECT_CHAR('P');EXPECT_CHAR('/'); EXPECT_CHAR('1');
EXPECT_CHAR('.');
return parse_int(buf, buf_end, minor_version, ret);
}
May 28 2010 HTTP::Parser::XS - writing a fast & secure XS module 10

11. Macros
Consistent design is
#define CHECK_EOF()
if (buf == buf_end) {
*ret = -2;
essential to heavy use }
return NULL;
of macros #define EXPECT_CHAR(ch)
CHECK_EOF();
Good abstraction ⇒ safe if (*buf++ != ch) {
*ret = -1;
code }
return NULL;
const char* parse_http_version(const char* buf, const char* buf_end,
int* minor_version, int* ret)
{
EXPECT_CHAR('H'); EXPECT_CHAR('T'); EXPECT_CHAR('T');
EXPECT_CHAR('P');EXPECT_CHAR('/'); EXPECT_CHAR('1');
EXPECT_CHAR('.');
return parse_int(buf, buf_end, minor_version, ret);
}
May 28 2010 HTTP::Parser::XS - writing a fast & secure XS module 11

12. Micro-optimization (only in picohttpparser)
Reduce # of conditional
branches, optimize for pipeline (27%
faster) unlikely(x) __builtin_expect(!!(x), 0)
#define
for (; ; ++buf) {
CHECK_EOF();
if (unlikely((unsigned char)*buf<= 'r')
&& (*buf == 'r' || *buf == 'n'))
gotoEOL_FOUND;
}
Unroll loops (36% faster)
May 28 2010 HTTP::Parser::XS - writing a fast & secure XS module 12

13. Release History of HTTP::Parser::XS
Current version: 0.07
No security hole found since initial
release
Two memory leaks were found and fixed
Please let me know if you find any
security holes (especially the ones that
lead to arbitrary code execution)
May 28 2010 HTTP::Parser::XS - writing a fast & secure XS module 13

14. Conclusion – KISS
Keep it simple, stupid
for fast development
simple design leads to more secure code
use perl whenever possible
simple operations (like tokenization) is worth
converting to XS
complex operations (from handling of strings to
database queries) are not so slow in perl
May 28 2010 HTTP::Parser::XS - writing a fast & secure XS module 14