We all know Mobile is different, but by how much?
This presentation attempts to quantify the difference between mobile and non-mobile, focusing on CPU, network and browser differences.
Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...
Â
Quantifying The Mobile Difference
1. The Mobile Difference â In Numbers
Guy Podjarny
@guypod
http://www.guypo.com/ Akamai Confidential
2. Mobile Isnât Desktop
â˘âŻ Itâs different in many waysâŚ
â˘âŻ This presentation focuses on sizing the gap
â˘âŻ How big is the gap between Mobile & Non-Mobile?
â˘âŻ How much does it impact performance?
â˘âŻ We canât cover everything, so weâll focus on three topics:
â˘âŻ Mobile CPU != Desktop/Laptop CPU
â˘âŻ Cellular Networks != Broadband Networks
â˘âŻ Mobile Browsers != Desktop Browsers
Faster ForwardTM Š2012 Akamai
10. Test Time!
â˘âŻ Question:
Does mobile computing power impact page load time?
â˘âŻ Tested Sites: Alexa US Top 100
â˘âŻ Test Tool: Mobitest
â˘âŻ Details:
â˘âŻ Test iPhone 4, iPhone 4S and iOS Simulator on 2-yr old MacBook Pro
â˘âŻ Note: iOS Simulator uses the actual hardware
â˘âŻ Tested overnight
â˘âŻ High speed Cable connection, WiFi
Faster ForwardTM Š2012 Akamai
12. CPU Difference - Conclusions
â˘âŻ Mobile CPU is still underpowered compared to desktop
â˘âŻ It indeed impacts web performance, even for regular sites
â˘âŻ Impact on load time is lighter than on benchmarks
â˘âŻ Roughly 25-50% of impact on benchmarks
â˘âŻ Computing power is more than just CPU
â˘âŻ At least GPU and RAM also have an impact
â˘âŻ Stick around for Ariyaâs Presentation!
â˘âŻ Recommendation: Test on old devices too
Faster ForwardTM Š2012 Akamai
21. Cellular Network Speed â Conclusions
â˘âŻ Theoretical limits are NOT the reality
â˘âŻ Cellular bandwidth definitely less than broadband
â˘âŻ There is no single number for cellular network speed
â˘âŻ But we need SOME numbers for testingâŚ
â˘âŻ So hereâs a SWAG at numbers to use for testing
â˘âŻ Merging many data points, and rounding numbers for convenience
Test Case Download (Kbps) Upload (Kbps)
Poor Cell Network 400 250
Average Cell Network 1200 600
Faster ForwardTM Š2012 Akamai
23. Broadband Latency (FCC)
âFiber-to-the-home services provided 17 ms round-trip
latency on average, while cable-based services averaged
28 ms, and DSL-based services averaged 44 ms.â
Connection
RTT
Type
Fiber 17ms
Cable 28ms
DSL 44ms
Faster ForwardTM
http://www.fcc.gov/measuring-broadband-america Š2012 Akamai
24. Ofcom study, UK Mobile Networks, May â11
âLatency⌠was an average of 117 milliseconds. Nearly two
Static Tests: thirds of tests took more than 100 millisecondsâ
User Panel: âAverage latency was 192 milliseconds and around a third
of tests delivered latency of higher than 200 millisecondsâ
http://stakeholders.ofcom.org.uk/binaries/research/telecoms-research/bbspeeds2010/Mobile_BB_performance.pdf
Faster ForwardTM Š2012 Akamai
25. Cisco Cloud Index, 2010-2015
âGlobal average fixed latency is 125 and average mobile
latency is 290⌠Global mobile consumer average latency is
307.3 msâ
Faster ForwardTM Š2012 Akamai
26. Yep, Mobile Latency is different
Latency(by(Connec,on(Type(
â˘âŻ Chart showing latency on Mobile vs Broadband, Ofcom
350"
numbers
300"
250"
200"
150"
100"
50"
0"
Fiber" Cable" DSL" Mobile" Mobile" Mobile" Mobile"
(Ofcom"Tests)" (Ofcom" (Cisco)" Consumer"
Panel)" (Cisco)"
Faster ForwardTM Š2012 Akamai
27. Cellular Network Latency - Conclusions
â˘âŻ Not a pretty sightâŚ
â˘âŻ MUCH higher than broadband latency
â˘âŻ Measured averages around 100-300 ms
â˘âŻ Updated table for suggested testing numbers:
Test Case Download Upload Latency
(Kbps) (Kbps) (ms)
Poor Cell Network 500 250 300
Average Cell Network 1200 600 200
Faster ForwardTM Š2012 Akamai
29. Test Time!
â˘âŻ Question:
How much impact does the difference in Bandwidth & Latency
have on web performance?
â˘âŻ Tested Sites: Alexa US Top 100
â˘âŻ Test Tool: WebPageTest
â˘âŻ Details:
â˘âŻ Increase Latency, Download & Upload Bandwidth separately
â˘âŻ Set no limits on other two variables
â˘âŻ Run each test 3 times, use median result
Faster ForwardTM Š2012 Akamai
30. Download BW Impact on Load Time
Poor Cell Conn
DSL
Cable FIBER
Avg Cell Conn
Faster ForwardTM Š2012 Akamai
31. Upload BW Impact on Load Time
Poor Cell Conn Avg Cell Conn
DSL
Cable FIBER
Faster ForwardTM Š2012 Akamai
32. Latency Impact on Load Time
Poor Cell Conn
Avg Cell Conn
DSL
Cable
FIBER
Faster ForwardTM Š2012 Akamai
33. Bandwidth/Latency Difference - Conclusions
â˘âŻ Upload difference doesnât matter much
â˘âŻ Download difference DOES matter
â˘âŻ Latency difference matters the most
â˘âŻ Tip #1: Test using Cell Network speeds
â˘âŻ Know what youâre getting
â˘âŻ Tip #2: Reduce number of requests
â˘âŻ Especially blocking requests, e.g scripts, CSS
â˘âŻ Tip #3: Reduce number of bytes
Faster ForwardTM Š2012 Akamai
35. Packet Loss â Broadband vs Mobile
Average Packet Loss
Mobile 0.6%
Broadband
0.017%
Faster ForwardTM Š2012 Akamai
36. Packet Loss, Akamai/MIT Research
â˘âŻ Tested 149 Million connections in 51 countries
â˘âŻ Compared PL for mobile networks from non-mobile networks
â˘âŻ Most connections have no packet loss (Median PL is 0%).
â˘âŻ 75th percentile usually zero too
â˘âŻ Average connection packet loss: 2.5%
â˘âŻ Fixed network average is 2.2%
â˘âŻ Implies connections are usually fine, but sometimes terrible
â˘âŻ The average PL for the non-zero half is 5%
â˘âŻ Suggests âsmallâ connections
have higher Packet Loss
â˘âŻ HTTP Connections are often small
http://pam2012.ftw.at/papers/PAM2012paper6.pdf
Faster ForwardTM Š2012 Akamai
37. Packet Loss - Conclusions
â˘âŻ Packet Loss is significantly higher in Mobile, ButâŚ
â˘âŻ Itâs small (even in Mobile)
â˘âŻ It usually doesnât happen (even in Mobile)
â˘âŻ Smaller connections lose more packets
â˘âŻ Recommendations:
â˘âŻ Donât worry about Packet Loss too muchâŚ
â˘âŻ Include some Packet Loss in your tests
â˘âŻ Ideal: Lots of caching, good user-feedback on error and no SPOFs
SWAG Test Case Download Upload Latency Packet
(Kbps) (Kbps) (ms) Loss
Poor Cell Network 500 250 300 1%
Average Cell Network 1200 600 200 0%
Faster ForwardTM Š2012 Akamai
38. Apple Network Link Conditioner
780/330 Kbps
200ms RTT
850/420 Kbps
190ms RTT 780/330 Kbps
200ms RTT
1% Packet Loss
Faster ForwardTM Š2012 Akamai
41. Different protocols lead to different speeds
http://www.ericsson.com/res/docs/2012/traffic_and_market_report_june_2012.pdf
Faster Forward
TM
Š2012 Akamai
43. Different Carriers have different average speeds
Source (PCWorld): http://bit.ly/JX7MQw Source: Ofcom
http://stakeholders.ofcom.org.uk/binaries/research/telecoms-research/bbspeeds2010/Mobile_BB_performance.pdf
Faster ForwardTM Š2012 Akamai
44. Volatility - Conclusions
â˘âŻ Stop Guessing!
â˘âŻ You canât predict your userâs speeds, today or in the future
â˘âŻ Little value in doing something âonly on cellularâ
â˘âŻ Strategy #1: Aim at the lowest common denominator
â˘âŻ Design for a usable experience in a slow network
â˘âŻ Enhance the page when possible
â˘âŻ Strategy #2: Adapt to Network Conditions
â˘âŻ E.g. reduce image quality when conditions are bad
â˘âŻ Not easy to do, but doable
â˘âŻ Check your options with your Server/Host/CDN
Faster ForwardTM Š2012 Akamai
45. Cellular Networks - Summary
â˘âŻ Cellular Networks are kinda crappy
â˘âŻ Canât live with âem, canât live without âemâŚ
â˘âŻ You can assume latency will be high
â˘âŻ And download wonât be too awesome
â˘âŻ Donât worry about Packet Loss & Upload rate
â˘âŻ You have bigger things to worry about
â˘âŻ Make Peace with it â you canât predict cellular network speed
â˘âŻ Best bet is to aim low, and adapt to network conditions
SWAG Test Case Download Upload Latency Packet
(Kbps) (Kbps) (ms) Loss
Poor Cell Network 500 250 300 1%
Average Cell Network 1200 600 200 0%
Faster ForwardTM Š2012 Akamai
47. Actually, theyâre not that differentâŚ
â˘âŻ Mobile & Desktop browsers are actually similar
â˘âŻ Many share the same underlying engines (e.g. WebKit)
â˘âŻ Some share the browser itself (e.g. Safari, Chrome, Firefox)
â˘âŻ How are they different, then?
â˘âŻ HTTP Pipelining
â˘âŻ Connection Models
â˘âŻ Cache Sizes
â˘âŻ Fragmentation
Faster ForwardTM Š2012 Akamai
49. Mobile Software: Pipelining
â˘âŻ HTTP Pipelining is around since HTTP 1.1
â˘âŻ Send multiple requests on connection before receiving response
â˘âŻ Most useful in high latency environment
â˘âŻ Big in Mobile
â˘âŻ All big mobile browsers
â˘âŻiOS 5+, Android, Opera
â˘âŻ Hardly used on Desktop
â˘âŻOpera Only (Default)
â˘âŻChrome, Firefox (Opt-in)
49 Faster ForwardTM Š2012 Akamai
50. Network Capture
â˘âŻ Samsung Galaxy S
â˘âŻ Max Conn: 12
â˘âŻ Conn Per Host: 12
â˘âŻ Max Piped Reqs: 6
â˘âŻ Max âIn-Flightâ Reqs: 72 (!!!)
â˘âŻ Full Details:
http://www.guypo.com/technical/http-pipelining-
request-distribution-algorithms/
50 Faster ForwardTM Š2012 Akamai
52. Test Time!
â˘âŻ Question:
Does HTTP Pipelining help make websites faster?
â˘âŻ Tested Sites: Alexa US Top 500
â˘âŻ Test Tool: WebPageTest
â˘âŻ Details:
â˘âŻ Use Firefox agent, which supports pipelining
â˘âŻ Turn pipelining on and off
â˘âŻ Set bandwidth to DSL, Cable & FIOS
â˘âŻ Increase Latency to see if impact changes
Faster ForwardTM Š2012 Akamai
54. Why doesnât pipelining have an effect?
â˘âŻ Are you sure pipelining was used?
â˘âŻ Yes⌠confirmed via tcpdump and various other metrics
â˘âŻ I donât know for sure, but I can guessâŚ
â˘âŻ 18 domains referenced on a page (average)
â˘âŻ 9 domains used only for one request (average)
â˘âŻ 3-4 domains used only for two requests (average)
â˘âŻ Browsers prefer new connection to piping a request
â˘âŻ Other bottlenecksâŚ
â˘âŻ Saw similar results for SPDY in recent test
â˘âŻ Details: http://bit.ly/MptaBV
â˘âŻ Conclusion: Not much you can/should doâŚ
Faster ForwardTM Š2012 Akamai
56. Summary
â˘âŻ Those were A LOT of numbersâŚ
â˘âŻ Mobile is truly different, and it matters for web performance
â˘âŻ Who saw that coming?!
â˘âŻ Main differences for Web Performance:
â˘âŻ CPU delta, but its effect is not linear
â˘âŻ Network delta in Latency & Download BW, not upload/Packet-Loss
â˘âŻ Main takeaways for web developers:
â˘âŻ Test performance on old devices too
â˘âŻ Test with at least average and poor cell network performance
â˘âŻ Provide a decent experience in the worst of conditions
â˘âŻ Remember - we only looked at a handful of aspectsâŚ
Faster ForwardTM Š2012 Akamai
57. Thank You!
Questions?
The Mobile Difference â In Numbers
Guy Podjarny
@guypod
http://www.guypo.com/ Akamai Confidential