Whenever you build a dynamic site with user-specific content, each hit to the site causes a hit to the .Net/PHP/Ruby/Python backend, potentially causing scalability issues. In this talk, we’ll look at a new way of handling user-specific content which takes the load away from the backend. Be prepared to learn about some exciting tools that add a ‘turbo’ button to your infrastructure
IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdf
Nginx and friends - putting a turbo button on your site
1. Nginx and friends
aka 'Adding a turbo button to your site'
Wim Godden
Cu.be Solutions
2. Who am I ?
Wim Godden (@wimgtr)
Owner of Cu.be Solutions (http://cu.be)
Open source developer since 1997
Developer of OpenX
Zend Certified Engineer
Zend Framework Certified Engineer
MySQL Certified Developer
3. Who are you ?
Developers ?
System/network engineers ?
Managers ?
7. What is caching ?
x = 5, y = 2
Same result
n = 50
CACHE
select
*
from
article
join user Doesn't change
on article.user_id = user.id all the time
order by
created desc
limit
10
8. Caching storage - Memcache(d)
Facebook, Twitter, YouTube, … → need we say more ?
Distributed memory caching system
Multiple machines ↔ 1 big memory-based hash-table
Key-value storage system
Keys - max. 250bytes
Values - max. 1Mbyte
9. Caching storage - Memcache(d)
Facebook, Twitter, YouTube, … → need we say more ?
Distributed memory caching system
Key-value storage system
Keys - max. 250bytes
Values - max. 1Mbyte
Extremely fast... non-blocking, UDP (!)
10. Memcache - installation & running it
Installation
Distribution package
PECL
Windows : binaries
Running
No config-files
memcached -d -m <mem> -l <ip> -p <port>
ex. : memcached -d -m 2048 -l 172.16.1.91 -p 11211
11. Caching storage - Memcache - some notes
Not fault-tolerant
It's a cache !
Lose session data
Lose shopping cart data
...
12. Caching storage - Memcache - some notes
Not fault-tolerant
It's a cache !
Lose session data
Lose shopping cart data
…
Firewall your Memcache port !
13. Memcache in code
Initialization :
MemcachedClient.Setup("MyCache", new string[]{ "server1.example.com", "server2.example.com"});
Usage :
MemcachedClient cache = MemcachedClient.GetInstance("MyCache");
string myData = cache.Get("myKey") as string;
if (myData == null) { Miss
myData = GetMyDataFromDB();
cache.Set("myKey", myData);
}
// Use myData
14. Memcache slabs
(or why Memcache says it's full when it's not)
Multiple slabs of different sizes :
Slab 1 : 400 bytes
Slab 2 : 480 bytes (400 * 1.2)
Slab 3 : 576 bytes (480 * 1.2) (and so on...)
Multiplier (1.2 here) can be configured
Store a lot of very large objects
→ Large slabs : full
→ Rest : free
→ Eviction of data !
15. Memcache - Is it working ?
Connect to it using telnet STAT pid 2941
STAT uptime 10878
STAT time 1296074240
"stats" command → STAT version 1.4.5
STAT pointer_size 64
STAT rusage_user 20.089945
Use Cacti or other monitoring tools STAT rusage_system 58.499106
STAT curr_connections 16
STAT total_connections 276950
STAT connection_structures 96
STAT cmd_get 276931
STAT cmd_set 584148
STAT cmd_flush 0
STAT get_hits 211106
STAT get_misses 65825
STAT delete_misses 101
STAT delete_hits 276829
STAT incr_misses 0
STAT incr_hits 0
STAT decr_misses 0
STAT decr_hits 0
STAT cas_misses 0
STAT cas_hits 0
STAT cas_badval 0
STAT auth_cmds 0
STAT auth_errors 0
STAT bytes_read 613193860
STAT bytes_written 553991373
STAT limit_maxbytes 268435456
STAT accepting_conns 1
STAT listen_disabled_num 0
STAT threads 4
STAT conn_yields 0
STAT bytes 20418140
STAT curr_items 65826
STAT total_items 553856
STAT evictions 0
STAT reclaimed 0
16. Website X – news is changing !
Header
Latest news
Article content page
Navigation
Article content
28. Cache warmup scripts
Used to fill your cache when it's empty
Run it before starting Webserver !
2 ways :
Visit all URLs
Error-prone
Hard to maintain
Call all cache-updating methods
Make sure you have a warmup script !
29. Cache stampeding - what about locking ?
Seems like a nice idea, but...
While lock in place
What if the process that created the lock fails ?
37. A dynamically generated, but static page
Apache 2.2 + PHP (3 DB queries) 18
IIS 7.5 + .Net (3 DB queries) 16
Varnish 3.0 11400
38. Varnish - what can/can't be cached ?
Can :
Static pages
Images, js, css
Static parts of pages that don't change often (ESI)
Can't :
POST requests
Very large files (it's not a file server !)
Requests with Set-Cookie
User-specific content
39. ESI → no caching on user-specific content ?
Logged in as : Wim Godden
TTL = 0s ?
5 messages
TTL=1h TTL = 5min
40. Dynamic content
Request Varnish Ideally
user X : /top /top request to webserver same
user Y : /top /top request to webserver same
user X : /top /top request to webserver if unchanged : served from cache
user Y : /top /top request to webserver if unchanged : served from cache
49. Advantages
No repeated hits to webserver anymore !
Only the initial hit (unless you pre-heat/warm up the cache !)
No hits for user-specific content
Not even for non-specific content
50. News added
addnews() method
o ... se
i nt t(.
e rt ..)
ins
DB
/news
51. Advantages
No repeated hits to webserver anymore !
Only the initial hit (unless you pre-heat/warm up the cache !)
Not for user-specific content
Not even for non-specific content
No TTLs for non-specific content
Imagine doing it for the bid status on Ebay items ;-)
52. How many Memcache requests ?
Logged in as : Wim Godden
<esim:include src="/top" session="1" ttl="1h" />
5 messages
<esim:include
src="/menu" <esim:include src="/news" ttl="5m" />
ttl="1h" />
53. Why Nginx ?
Native Memcache support
Excellent and superfast subrequest system
Including parallel subrequests
Handles thousands of connections per worker
With minimal memory footprint
55. Figures
First customer :
No. of web servers : 18 → 4
No. of db servers : 6 → 2
Total : 24 → 6 (75% reduction !)
Second customer (already using Nginx + Memcache) :
No. of web servers : 72 → 8
No. of db servers : 15 → 4
Total : 87 → 12 (86% reduction !)
56. Availability
Stable at 2 customers
Still under heavy development
Beta : July 2012
Final : Sep 2012
58. So...
There's life outside the MS-box
And it's pretty awesome ;-)
Need caching in .Net (for DB queries, config-files, etc.) ?
Memcache
Velocity ?
Caching static pages
→ Varnish with ESI
Caching for authenticated users
→ Nginx with Dynamic ESI
There's a lot more outside the box
→ Maybe some other time ;-)