The HTTP caching talk I gave at RIT++ http://ritconf.ru/

1. High-performance
Robust
HTTP
Front-ends
/ tips, tricks and expectations
Saturday, April 23, 2011

2. Who am I? @postwait on twitter
Author of “Scalable Internet Architectures”
Pearson, ISBN: 067232699X
Contributor to “Web Operations”
O’Reilly, ISBN:
Founder of OmniTI, Message Systems, Fontdeck, & Circonus
I like to tackle problems that are “always on” and “always growing.”
I am an Engineer
A practitioner of academic computing.
IEEE member and Senior ACM member.
On the Editorial Board of ACM’s Queue magazine.
2
Saturday, April 23, 2011

3. Agenda
• Why only HTTP?
• HTTP-like protocols
• Performance
• Availability
Saturday, April 23, 2011

4. HTTP
• Why only HTTP... it’s what we do.
• User-based, immediate, short-lived
transactions occupy my life.
• So, not just HTTP.
• HTTPS
• SPDY (... we’ll get to this)
Saturday, April 23, 2011

5. Performance
• ATS (Apache Traffic Server)
• supports SSL
• battle-hardened codebase
• very multi-code capable
• Varnish
• VCL adds unparalleled flexibility
• no SSL!
• nginx
• I don’t see much of this out on the edge
Saturday, April 23, 2011

6. Performance Expectations
• from a single server, you should be able to:
• support 500k concurrent users
• this is only 40k sockets/core
• push in excess of 100k requests/second
• this is only 9k requests/core*second
• push close to 10 gigabits
• this is why 10G was invented
Saturday, April 23, 2011

7. Performance Achievements
• Good load balancers achieve this performance
• with dual socket Westmere processors,
we’re able to achieve in
software on
general purpose hardware
what was only possible in hardware ASICs.
• ATS and Varnish can do this today.
Saturday, April 23, 2011

8. The Basic Rules: Content
• You must serve content from cache
• Your cache should fit in memory
• If it does not, it should spill to SSD,
not spinning media.
Saturday, April 23, 2011

9. The Basic Rules: CPU
• You must cache SSL sessions
• SSL key negotiation is expensive.
• SSL encryption is not*
• Common cases must not cause state on the firewall.
• It’s hard enough to serve 150k requests/second.
• You will spend too much time in kernel in
iptables, ipf, or pf.
• allow port 80 and port 443.
• enable SYN flood prevention
* crypto obviously costs CPU; symmetric crypto is relatively cheap
Saturday, April 23, 2011

10. The Basic Rules: Network
• You must not run a stateful firewall in front
• too expensive
• too little value
• You must be directly behind capable router(s)
• expect anywhere from
1MM to 20MM packets per second
• we need to run BGP for availability
Saturday, April 23, 2011

11. Availability
• We learned in the performance section:
• 1 machine / 10Gbps uplink performs well enough
• We need redundancy:
• Linux HA?
• VRRP/HSRP?
• CARP?
• No...
Saturday, April 23, 2011

12. Availability: Constraints
• Client TCP sessions are relatively short lived.
• The web is a largely idempotent place.
• Clients are capable of retrying on failure.
• This means:
• forget stateful failover.
• focus on availability for new connections.
Saturday, April 23, 2011

13. Availability: Setup
• You are behind a capable router (it was a rule)
• Use routing protocols (BGP) to maintain availability.
BGP
10.1.0.0/24 10.1.1.0/24
10.1.0.0/23 10.1.0.0/23
Saturday, April 23, 2011

14. Working Stacks
• Linux (OS/TCP stack) • Illumos (OS/TCP stack)
• Varnish (HTTP) • ATS (HTTP/HTTPS)
• Quagga (BGP) • Quagga (BGP)
Saturday, April 23, 2011

15. Future!
• This stuff is fast.
• In the end, we’re not looking for faster servers,
we’re looking for improved user experience.
• Enter SPDY
• Google’s multi-channel HTTP super-protocol
• Allows multiplexing of concurrent HTTP(like)
request/response on a single TCP session.
• Defeats slow startup
• Allows for content prioritization on server
Saturday, April 23, 2011

16. Future: my thoughts
• SPDY is relatively simple to implement on the server
• SPDY is very very hard to leverage on the server
• If ATS implemented SPDY in and out
• and provided a robust configuration language
to leverage it
... the future would be today.
Saturday, April 23, 2011

17. Thank you.
• Thank you Олег Бунин
• Thanks to the Varnish and ATS developers.
• Спасибо.
Saturday, April 23, 2011