This document discusses measuring and analyzing web latency. It provides an overview of key latency metrics like server rendering time and time to load. It describes how to monitor latency over time and analyze trends. It also discusses testing latency through synthetic benchmarks and real user monitoring. The document emphasizes the importance of understanding client-side rendering times and how latency can be impacted by many factors.

1. Website Latency Diagnostics
alex.burciu @ amazon.com
bad robot.

2. Agenda
• Web Performance Optimization
• Measuring Web Latency
• Latency Monitoring and Analysis
• Testing Latency

3. Web Performance Optimization
• Customer satisfaction & loyalty
– White page of death
– “World Wide Wait”
• Faster pages = longer sessions
– Sales opportunities
– Ads impressions
• Competitive Advantage
– Better customer experience
– Factor in SEO rankings

4. Latency Metrics
• Understand what you’re optimizing and quantify impact.
– serverRenderTime (webserver render time)
• time delay on the webserver since receiving the HTTP
request to when generating and emitting the last byte of
content to the client.
– clientPageLoaded (browser render time)
• time delay spent on the browser since receiving the first
HTML response byte to when the DOM load event is fired.
– clickToPageLoaded (complete rendering time)
• time delay experienced by the customer since when
initiating navigation to when the page loaded.

5. Metrics Relations
(?) clickToPageLoaded =
serverRenderTime + clientPageLoaded
(?) serverRenderTime > clientPageLoaded

6. Browser-webserver Interaction
serverRenderTime
Click PL
Unload RTT Response Transfer DOM
Redirect
Cache clientPageLoaded
DNS
TCP
SSL

7. HTTP Flushing
serverRenderTime
Click PL
Unload RTT Response transfer DOM
Redirect
Cache clientPageLoaded
DNS
TCP
SSL

8. HTTP Caching
clientPageLoaded
Click PL
Unload DOM
Redirect
Cache

9. Back-Forward Rendering Cache
Click PL
Redirect
Cache

10. Pre-rendering
serverRenderTime
PL Click
Unload RTT Transfer DOM
Redirect
Cache clientPageLoaded
DNS
TCP
SSL

11. Navigation Timing (pre-request)
Click Redirect Cache DNS TCP Connection SSL Req...
Unload

12. Navigation Timing (pre-request)
navigationStart fetchStart
unloadEventStart domainLookupStart secureConnectionStart
redirectStart connectStart requestStart
Click Unload Redirect Cache DNS TCP Connection SSL Req...
redirectEnd connectEnd
unloadEventEnd domainLookupEnd

13. Navigation Timing (post-request)
Response
Request DOM Processing onLoad PL

14. Navigation Timing (post-request)
domLoading
requestStart domInteractive
responseStart domContentLoadedEventStart loadEventStart
Request Response DOM Processing onLoad PL
responseEnd domComplete
domContentLoadedEventEnd loadEventEnd

15. PageLoad Lifetime
navigationStart
redirect
fetchStart
connect
dns
requestStart
responseStart
domLoading
unloadEvent
domInteractive
domContentLoaded
domComplete
loadEvent
0 2,000 4,000 6,000 8,000 10,000 12,000 14,000

16. Navigation Timing JavaScript API
>> window.performance.timing
PerformanceTiming
connectEnd: 1344530318281
connectStart: 1344530318281
domComplete: 1344530330212
domContentLoadedEventEnd: 1344530321367
domContentLoadedEventStart: 1344530320211
domInteractive: 1344530320048
domLoading: 1344530318567
domainLookupEnd: 1344530317143
domainLookupStart: 1344530317143
fetchStart: 1344530318281
loadEventEnd: 1344530330214
loadEventStart: 1344530330212
navigationStart: 1344530317144
redirectEnd: 1344530318281
redirectStart: 1344530317149
requestStart: 1344530318283
responseEnd: 1344530319479
responseStart: 1344530318566
secureConnectionStart: 0
unloadEventEnd: 1344530319045
unloadEventStart: 1344530318567

17. Page Responsiveness
clickToFirstByte = responseStart - navigationStart
Influencing Backend Front-end
Network Server Browser
factor Application Application
Unload ✓ ✓
Redirect ✓ ✓ ✓
Cache ✓ ✓
DNS ✓ ✓
Connection ✓ ✓ ✓ ✓
SSL ✓ ✓
Request ✓ ✓ ✓
Response ✓ ✓ ✓

18. End-to-end Latency
clickToPageLoaded = loadEventEnd - navigationStart
Influencing Backend Front-end
Network Server Browser
factor Application Application
clickToFB ✓ ✓ ✓ ✓ ✓
DOM
✓ ✓ ✓ ✓ ✓
Interactive
DOM
Content ✓ ✓ ✓
Loaded
DOM
✓ ✓ ✓
Complete
DOM Load
✓ ✓
Event

19. Testing Web Performance
Synthetic Benchmarking
– Generated traffic
– Limited context
Real Users Monitoring
– Subset of pages
– Prod traffic
– On-demand
– Context-aware
– Expensive
– All visited pages
– Continuous
– Cheap

20. Average Daily Latency in 3 months

21. Average Daily Latency in 3 months

22. Avg Daily Latency vs Traffic

23. Avg Daily Latency vs Traffic

24. Avg Hourly Latency vs Traffic

25. Prev Week Avg Hourly Latency &Traffic

26. Percentile Metrics
• P(x) = variable value for which x% of the variables are
smaller
P25 = first quartile (Q1)
P50 = median (Q2)
P75 = third quartile (Q3)
• Example
– 10, 770, 40, 60, 50, 880, 9999, 30, 30, 20, 10
– 10, 10, 20, 30, 30, 40, 50, 60, 770, 880, 9999
• Average vs P50: 1,081 > 40

27. Prev Week Hourly Latency Percentiles

28. Understats and overstats
• The inverse of a percentile.
• Percentiles
– Constant percentage (e.g. 50%)
– Variable measurement (e.g. latency in ms)
• Understatistics and overstatistics
– Constant measurement (e.g. latency <1000ms)
– Variable percentage (e.g. percentage of hits)
• Understat x = percentage of hits with latency under x ms

29. Prev Week Traffic% under 2 seconds

30. Prev Week Traffic% under 2 seconds

31. Avg vs P50 5-minute Latency

32. P50 for components before response

33. P50 for components after response

34. domContentLoaded per browser

35. DOM Content Loaded
domLoadingToDomContentLoaded =
domContentLoadedEventStart - domLoading
Influencing Backend Front-end
Network Server Browser
Factor Application Application
DOM
Content ✓ ✓ ✓
Loaded

36. Root Cause: Content Change
Date Press Release Title
Updated Amazon Cloud Player Includes New Scan
7/31/2012 and Match Technology, Free Audio Quality
Upgrades, and More
Amazon.com Announces Second Quarter Sales
7/26/2012
up 29% to $12.83 Billion
Amazon Launches Innovative New Education
Initiative, Paying 95% of Tuition Costs for
7/23/2012
Employees to Pursue Their Aspirations – Whether
at Amazon, or in Another Industry
Amazon.com Books Editors Announce the Big Fall
7/23/2012
Books Preview
On Prime Instant Video First: Amazon Announces
Digital Video License Agreement with Warner
7/20/2012 Bros. Domestic Television Distribution, Bringing
Acclaimed TV Shows Exclusively to Kindle Fire
and Prime Customers
Amazon Web Services Introduces New Amazon “Jeff Letter”
7/19/2012
EC2 High I/O Instance Type on Gateway Page
Amazon.com to Webcast Second Quarter 2012
7/12/2012
Financial Results Conference Call

37. Example Latency Root Causes

38. A/B Latency Testing

39. Key Takeaways
• Invest in Latency Monitoring
– 80% of the latency is on the client
– Navigation Timing cross-browser API
• Browser rendering is complex
– Latency easily influenced by context
– Lab traffic is different from the mix from real customers
• Look at your data
– Average poorly represents a population
– Control releases using A/B testing

40. Q&A
“Premature optimization is the root of all evil.”
– Donald Knuth