Altitude SF 2017: Optimizing your hit rate
•
5 likes•912 views
In this workshop, we’ll reveal how improving your caching strategy speeds up page loads with little effort. We’ll follow an HTTP request through Fastly and deep dive into cache hit ratios. In practical sessions, we’ll explain how to optimize your hit rate and still serve fresh content to users. We’ll cover caching headers, invalidation and the effect on user-perceived loading experience.
Recommended
Recommended
More Related Content
What's hot
What's hot (20)
Similar to Altitude SF 2017: Optimizing your hit rate
Similar to Altitude SF 2017: Optimizing your hit rate (20)
More from Fastly
More from Fastly (20)
Recently uploaded
Recently uploaded (20)
Altitude SF 2017: Optimizing your hit rate
- 1. Optimizing Your Cache Hit Ratio Léon Brocard Sales Engineer | Fastly Paolo Alvarado Customer Support Manager | Fastly
- 2. Léon Brocard Systems Engineer Fastly London Paolo Alvarado Customer Support Manager Fastly Tokyo
- 3. Cache hit ratio Why do we cache? Caching on Fastly HTTP caching Overriding caching Caching patterns Future
- 4. This is a talk workshop
- 7. Where is “Cache hit ratio” on https://www.fastly.com/
- 11. Name & Domain hitrate2017lb.global.ssl.fastly.net Address 35.197.9.99 No TLS, Port 3000
- 15. Why do we cache?
- 18. Level Capacity Latency Can on my desk 1 can ~1s My fridge 10 cans ~30s Local shop 500 cans ~5m Supermarket 10k cans ~90m
- 19. Level Capacity Latency Can on my desk 1 can ~1s My fridge 10 cans ~30s Local shop 500 cans ~5m Supermarket 10k cans ~90m
- 20. Level Capacity Latency Cash in my wallet $50 ~1s Local ATM $200 ~5m Bank $1,000 ~20m
- 21. Level Capacity Latency Plate 1 serving ~1s Serving dish 10 servings ~1m Kitchen 100 servings ~5m
- 23. Level Capacity Latency L1 cache 64 KiB per core ~1ns L2 cache 256 KiB per core ~4ns L3 cache Up to 2 MiB per core, shared ~20ns Main memory Up to 64 GB ~100ns
- 24. Level Capacity Latency Browser ~Megabytes ~0.1ms CDN ~Gigabytes ~5ms Origin ~Terabytes ~500ms
- 25. Cache?
- 26. L'écureuil roux cache ses provisions pour l'hiver
- 27. Hit Miss
- 31. Scenario Origin requests / sec EC2 instances No CDN 2,000 40 50% CHR 1,000 20 80% CHR 400 8 90% CHR 200 4 95% CHR 100 2
- 35. Why is latency so important?
- 38. GET /technology/2016/jul/15/ how-the-internet-was-invented -1976-arpa-kahn-cerf HTTP/1.1 Host: www.theguardian.com
- 39. HTTP/1.1 200 OK Content-Type: text/html; charset=utf-8 Content-Length: 110502 <!DOCTYPE html>...
- 41. Visit hitrate2017lb.global.ssl.fastly.net View ▶ Developer ▶ Developer Tools Network tab Reload, ⌘R, shift-⌘R
- 42. c
- 49. 1.5s 2.0s 3.5s 21.0s
- 51. www.webpagetest.org URL: docs.fastly.com EC2 location with low pending tests Connection DSL, capture video, keep test private
- 52. Here’s one I prepared earlier: http://fastly.us/2sybe4q
- 53. Back to developer tools Right click on table heading Response headers ▶ Add “Cache-Control” Response headers ▶ Manage header columns ▶ Add custom header → “X-Cache”
- 58. if users double their bandwidth without reducing their RTT significantly, the effect on Web Browsing will be a minimal improvement. However, decreasing RTT, regardless of current bandwidth always helps make web browsing faster
- 59. How can we decrease RTT?
- 60. How can we decrease RTT? latency?
- 61. Caching
- 65. HTTP caching
- 66. 1996 RFC 1945 HTTP/1.0 1997 RFC 2068 HTTP/1.1 1999 RFC 2616 HTTP/1.1 2014 RFC 7230 HTTP/1.1: Message Syntax and Routing 2014 RFC 7231 HTTP/1.1: Semantics and Content 2014 RFC 7232 HTTP/1.1: Conditional Requests 2014 RFC 7233 HTTP/1.1: Range Requests 2014 RFC 7234 HTTP/1.1: Caching 2014 RFC 7235 HTTP/1.1: Authentication 2015 RFC 7540 HTTP/2
- 67. Fresh Expired
- 68. Last-Modified: Fri, 13 Nov 2015 14:11:55 GMT If-Modified-Since: Fri, 13 Nov 2015 13:58:51 GMT HTTP/1.1 304 Not Modified
- 72. ETag: “3e8d1bf998b31cb811264f4789eac005” If-None-Match: “3e8d1bf998b31cb811264f4789eac005” HTTP/1.1 304 Not Modified
- 75. Visit hitrate2017lb.global.ssl.fastly.net Network tab Browser cache max-age=60 tab First time. Second time.
- 77. Cache-Control: max-age=60 Date: Thu, 25 May 2017 10:16:09 GMT Expires: Thu, 25 May 2017 10:16:58 GMT
- 83. Visit hitrate2017lb.global.ssl.fastly.net Network tab Browser cache ⌘R Cache-Control: (none) / private / max-age=60
- 84. Tool time
- 85. Install Chrome Extension “Browser Cache Hit Ratio”... https://fastly.us/browser-cachehitratio
- 86. Install Chrome Extension “Fastly Cache Hit Ratio”... https://fastly.us/cachehitratio
- 89. Break
- 93. Changing caching
- 94. On the origin with HTTP headers Override in Fastly
- 97. Fastly Service ▶ Configuration ▶ Clone active ▶ Settings ▶ Fallback TTL ▶ 10 ▶ Save ▶ Activate ▶ Purge ▶ Purge All ▶ Purge
- 100. default TTL?
- 101. if ( beresp.http.Expires || beresp.http.Surrogate-Control ~ "max-age" || beresp.http.Cache-Control ~"(s-maxage|max-age)" ) { # keep the ttl here } else { # apply the default ttl set beresp.ttl = 10s; }
- 104. if (req.url == "/dogs/max-age=60") { set beresp.http.Cache-Control = "max-age=10"; }
- 108. if (req.url == "/dogs/max-age=60") { set beresp.ttl = 10s; set beresp.http.Cache-Control = "max-age=10"; }
- 109. sub vcl_fetch { if (beresp.http.Cache-Control) { if (req.url.path ~ "^/users/flair/") { set beresp.http.Cache-Control = "public, max-age=180"; } else { set beresp.http.Cache-Control = "private"; } } Here’s an example of where VCL comes in very handy. Recently we deployed .NET 4.6.2 which had a very nasty bug that set max-age on cache responses to over 2000 years. The quickest way to mitigate this for all of our services affected was to override that cache header as-needed at the edge. As I write this, the following VCL is active:
- 110. What’s in a cache key?
- 111. req.http.host The incoming request’s host header req.url The full URL including query string #####GENERATION##### A generation number used for purging all
- 112. What do these URLs have in common? ● /search?query=dogs&page=1 ● /search?page=1&query=dogs
- 116. The uncacheable
- 118. “Dynamic content is really interesting” —Hooman 2014 Classically, dynamic content was uncacheable because it required interaction with a backend.
- 119. How can we cache dynamic content?
- 122. How do we force an update for cached content?
- 124. Purging
- 125. Single URL Purge Instant Purge an individual URL. Surrogate-Key Purge Instant Purge a particular service of items tagged with a Surrogate-Key. Purge All Instant Purge everything from a service.
- 130. HTTPS everywhere http://hitrate2017lb.global.ssl.fastly.net/ 301 Moved Permanently redirect to: https://hitrate2017lb.global.ssl.fastly.net/
- 131. HIT-SYNTH
- 134. Serving Stale
- 135. What does a negative number here mean?
- 138. “Worst-case scenario, we can always ghost-serve. Which actually happened: our Rolling Stone legacy origin was down for three days; the only reason readers could still visit our site without noticing the problem was because we used Fastly’s stale-if-error.” — Chris Boylan, Director of Engineering, Wenner Media
- 139. How can we purge while still offloading?
- 140. Soft Purge Single URL Purge Surrogate-Key Purge Soft Purging sets an object’s TTL to 0s, forcing revalidation. For best results, use in combination with stale-while-revalidate and/or stale-if-error
- 141. Levels of caches
- 142. × =
- 143. × =
- 146. × =
- 151. When the royal baby George Alexander Louis was born in 2013, the Guardian's traffic spiked from an average of 400 requests per second to over 1,000 requests per second. Fastly shielded the Guardian's origin servers from the hundreds of thousands of new requests pouring in, maintaining site uptime and delivering consistent performance for readers around the world. The company’s software architect, Matthew O’Brien, said that it’s difficult to predict news events, but Fastly helps them prepare accordingly.
- 152. Cache in browser and Fastly for different amount of time?
- 153. Cache-Control: private Surrogate-Control: max-age=60, stale-while-revalidate=5, stale-if-error=86400
- 155. “Fastly effectively handled the higher levels of traffic we experienced during the Apple Watch announcement.” Using Fastly’s stale-while-revalidate feature to serve readers slightly outdated content while the cached content was refreshed, WIRED knew readers would never see errors. Zack described stale-while-revalidate as one of Fastly’s “most exciting features.” He said, “It allowed us to successfully serve the live blog while constantly updating its content, ensuring our readers had access to the freshest content without seeing errors." — Zack Tollman, Lead Engineer at WIRED
- 160. X-Cache: HIT, HIT X-Cache-Hits: 649406, 922
- 161. fastly_info.state Jun 08 14:27:24 cache-ord1722 papertrail: MISS-CLUSTER "-" [08/Jun/2017:14:27:23 +0000] GET /recipes/?show=Beef%20brochettes HTTP/1.1 200 26
- 162. Logging at the edge Chris Jackel Systems Engineer | Fastly
- 163. vcl_hash vs Vary
- 164. Hash Vary
- 165. Accept-Encoding: gzip, deflate, br Content-Encoding: gzip Vary: Accept-Encoding
- 167. Future
- 169. HTTP/2 enables a more efficient use of network resources and a reduced perception of latency by introducing header field compression and allowing multiple concurrent exchanges on the same connection
- 170. Service Workers This specification describes a method that enables applications to take advantage of persistent background processing, including hooks to enable bootstrapping of web applications while offline
- 171. The power of the network Andrew Betts Principal Developer Advocate | Fastly
- 172. Emerging and future web trends Alex Russell Software Engineer | Google
- 173. Summary
- 174. Cache hit ratio Why do we cache? Caching on Fastly HTTP caching Overriding caching Caching patterns Future
- 175. Run live experiments
- 177. Thanks for participating Léon & Paolo