When doing web application performance tests we usually present statistics from tools that simulates real users as a results from tests. This approach works for APIs and simple sites that does not require additional resource like javascript, css or image files. For most modern websites measurements collected that way give stakeholders information only about performance and stability of the first layer of application but give no information about performance seen from user perspective. To collect such information we need to use other tools and approaches. In my presentation I would like to share experience my team collected when designing and implanting performance tests strategy that aims to give 360 degrees view on application performance. Such solution gives developers fast feedback about performance changes, provide tools for testers to estimate site performance before going live and measures customer experience. In particular I would like to present: - rules for building high performance web sites - concepts of synthetic and real user measurements - when and what for use load generators - potential issues with test environment - metrics that could be collected - JavaScript Resource and Performance APIs - open source and commercial tools that can be used

1. QAFEST
Web Application
Performance From
User Perspective
JACEK OKROJEK
01.10.2016

2. 1. Why it matters
2. Tips for improving web page performance
3. Metrics
4. Measurements techniques
AGENDA

3. GFT Technologies SE 25.06.2016 3
Why it matters
W EB APPLICATION PERFORMANCE FROM USER PERSPECTIVE

4. GFT Technologies SE 25.06.2016 4
High Performance Web Sites Rules
 Gzip Components
 Put StyleSheets at the Top
 Put Scripts at the Bottom
 Minimize HTTP Requests
 Minify JavaScript and CSS
 Make JavaScript and CSS External
 Use a Content Delivery Network
 Avoid empty src or href
 Add an Expires or a Cache-Control Header
 Avoid CSS Expressions
 Reduce DNS Lookups
 Remove Duplicate Scripts
W EB APPLICATION PERFORMANCE FROM USER PERSPECTIVE
 Make AJAX Cacheable
 Use GET for AJAX Requests
 Reduce the Number of DOM Elements
 Use Cookie-Free Domains for Components
 Do Not Scale Images in HTML
 Make favicon.ico Small and Cacheable
 Configure ETags
 Avoid Redirects
 Reduce Cookie Size
 Avoid Filters
 No 404s

5. GFT Technologies SE 25.06.2016 5
Checking conformance to High Performance Web Sites Rules
 YSlow
 Google PageSpeed Insights
W EB APPLICATION PERFORMANCE FROM USER PERSPECTIVE

6. GFT Technologies SE 25.06.2016 8
Server Metrics
 HTML Document download time is 68ms
W EB APPLICATION PERFORMANCE FROM USER PERSPECTIVE

7. GFT Technologies SE 25.06.2016 10
USING REAL BROWSER IN PERFORMANCE TESTS
 Key advantages
 Provides metrics from real
browser
 You can run them when you
want
 Key disadvantage
 User perspective as if she use
test equipment
browser, hardware, connections
speed, location)
 Key challenges
 What metrics should be
collected
 Where to put test equipment
SYNTHETIC MEASUREMENTS

8. GFT Technologies SE 25.06.2016 11
Navigation API
BROW SER METRICS

9. GFT Technologies SE 25.06.2016 12
USING REAL BROWSER IN PERFORMANCE TESTS
 Key advantages
 Provides metrics from real
browser
 You can run them when you
want
 Key disadvantage
 User perspective as if she use
test equipment
browser, hardware, connections
speed, location)
 Key challenges
 What metrics should be
collected
 Where to put test equipment
SYNTHETIC MEASUREMENTS

10. GFT Technologies SE 25.06.2016 13
Navigation Timing API
var t = performance.timing || {};
var i = t.domInteractive - t.navigationStart,
d = t.domContentLoadedEventStart - t.navigationStart,
c = t.domComplete - t.navigationStart;
return i + ', ' + d + ', ' + c;
BROW SER METRICS

11. GFT Technologies SE 25.06.2016 14
Zasoby
 Typowe metryki
 Czas pobrania wybranego zasobu
 Czas pobrania ostatniego zasobu
BROW SER METRICS

12. GFT Technologies SE 25.06.2016 15
Resource Timing API
var resources = performance.getEntriesByType("resource");
for (var i=0; i < resources.length; i++) {
log("== Resource[" + i + "] - " + resources[i].name);
// Start until reponse end
t = (resources[i].startTime > 0) ? "0":
(resources[i].responseEnd - resources[i].startTime);
log("... Start until response end time = " + t);
}
BROW SER METRICS

13. GFT Technologies SE 25.06.2016 16
User Timing API
 Typowe metryki
 Wyświetlony wybrany element strony
BROW SER METRICS
<html>
<head>
<link rel="stylesheet" type="text/css" media="all" href="contentSearchUI.css">
</head>
…
<body>
<img src="hero.jpg" onload="performance.mark('hero1')">
<script>performance.mark('hero2')</script>
</body>
</html>

14. GFT Technologies SE 25.06.2016 17
Tools and services
 Open Source
 Selenium, BrowserMob i HarStorage
 SiteSpeed.io
 WebPageTest
 Płatne
 NewRelic
 CatchPoint
SYNTHETIC MEASUREMENTS

15. GFT Technologies SE 25.06.2016 18
Real User Measurements (RUM)
 Collected from real user devices
(mobile and desktop) and helps
understands how performance is
seen by real users
 Key advantages
 You know what performance is seen
by real users
 Solves most of the challenges of
synthetic monitoring
 Key disadvantage
 You can not run it when you want (for
example before release)
 Key challenges
 What to measure
W EB APPLICATION PERFORMANCE FROM USER PERSPECTIVE

16. GFT Technologies SE 25.06.2016 19
Dodatkowe informacje
 https://www.soasta.com/blog/measuring-web-performance-video/
 https://dzone.com/articles/a-brief-history-of-web-performance-roi
 https://developers.google.com/web/fundamentals/performance
 http://yslow.org/
 http://www.softwareishard.com/blog/har-viewer/
 https://www.soasta.com/blog/ebook-usertiming-performance-monitoring/
 https://codeascraft.com/
 https://www.soasta.com/blog/ebook-usertiming-performance-monitoring/
 https://code.facebook.com/posts/991252547593574/the-technology-behind-
preview-photos/
 https://blog.twitter.com/2012/improving-performance-on-twittercom
 http://www.stevesouders.com/blog/2015/05/12/hero-image-custom-metrics
Steve Souders
@Souders
Tammy Everts
@tameverts
Ilya Grigorik
@igrigorik
Tim Kaldec
@tkaldec

17. GFT Technologies SE
Jacek Okrojek
Email: jacek.okrojek@gft.com
Twitter: @Okrojek
Thank you