presented October 17 2018

1. Delivering Beautiful and Fast
Images and Video
Doug Sillars
@DougSillars
BordeauxJS
October 17, 2018

2. Contact Me:
@DougSillars
Doug.Sillars@gmail.com
www.dougsillars.com
Doug Sillars
Freelance Developer Relations
Performance Audits: Web/Native
Workshops:
Performance/Images/Video
http://bit.ly/HighPerformanceAndroidApps

4. 0.5
0.6
0.7
0.8
0.9
Standing in Line Standing on the
edge of a virtual
cliff
Experiencing
Mobile Delays
Solving a Math
Problem
https://www.ericsson.com/res/docs/2016/mobility-report/emr-feb-2016-the-stress-of-steaming-delays.pdf
Stress

5. Large Downloads Induce Delays in Rendering
3s: 53% of Users Abandon Mobile Sites
500ms: 26% Frustration
8% Engagement
100ms: 1% Revenue Walmart & Amazon (Desktop 2001)
4% Mobile Users Throw Their Phones
https://www.doubleclickbygoogle.com/articles/mobile-speed-matters
http://bit.ly/mobileWebStress
http://www.globaldots.com/how-website-speed-affects-conversion-rates/
https://www.mobilejoomla.com/blog/172-responsive-design-vs-server-side-solutions-infographic.html

6. Images & Video make up 75% of Web Content
HTTPArchive mobile websites 15/02/18

7. 4 Simple Image Optimizations

8. 4 Simple Image Optimizations
1. Quality
2. Format
3. Sizing
4. Lazy Loading

9. 4 Simple Image Optimizations
1. Quality
2. Format
3. Sizing
4. Lazy Loading

10. 4 Simple Image Optimizations
Httparchive.org Webpagetest.org

11. 4 Simple Image Optimizations
1. Quality
2. Format
3. Sizing
4. Lazy Loading

12. Image Quality
magick -quality 85 riga.jpg riga85.jpg
http://res.cloudinary.com/dougsillars/image/upload/q_85/v1520504964/IMG_20180301_114117_tzasan.jpg
Lighthouse:
Recommends 85% quality on all images
https://developers.google.com/speed/docs/insights/OptimizeImages

13. 100%
3.6 MB
http://res.cloudinary.com/dougsillars/image/upload/v1529005982/IMG_20180614_184507_ssuk1i.jpg

14. 85%
1.87 MB
q_85
http://res.cloudinary.com/dougsillars/image/upload/q_85/v1529005982/IMG_20180614_184507_ssuk1i.jpg

15. Image Quality Use “In The Wild”
500,000 mobile sites
Analyzed 3/15/18

16. Image Quality Use “In The Wild”
Median Savings (50th percentile):
• 2.83 seconds faster page load
• 419KB less data

17. 50%
914 KB
q_50
http://res.cloudinary.com/dougsillars/image/upload/q_50/v1529005982/IMG_20180614_184507_ssuk1i.jpg

18. 20%
513 KB
q_20
http://res.cloudinary.com/dougsillars/image/upload/q_20/v1529005982/IMG_20180614_184507_ssuk1i.jpg

19. Graphing the Results
https://developers.google.com/speed/docs/insights/OptimizeImages

20. Graphing the Results
https://developers.google.com/speed/docs/insights/OptimizeImages

21. Automate Quality vs. File Size
• Butteraugli
• SSIM: Structural SIMilarity
https://github.com/technopagan/cjpeg-dssim
cjpeg-dssim jpegoptim riga.jpg
http://res.cloudinary.com/dougsillars/image/upload/q_auto/v1529005982/IMG_20180614_184507_ssuk1i.jpg

22. SSIM
1.46 MB
http://res.cloudinary.com/dougsillars/image/upload/q_auto/v1529005982/IMG_20180614_184507_ssuk1i.jpg

23. Graphing the Results
https://developers.google.com/speed/docs/insights/OptimizeImages

24. Results:
• Test Load on Motorola G4:

25. 4 Simple Image Optimizations
1. Quality
2. Format
3. Sizing
4. Lazy Loading

26. Image Formats – Average Size
HTTPArchive mobile websites 15/02/18

27. Image Formats – Average Size
HTTPArchive mobile websites 15/02/18

28. Scalable Vector Graphics (SVG)
Images drawn as shapes
infinitely scalable
XML - Can be added inline to HTML document
https://gist.github.com/hail2u/2884613?short_path=66a60ff

29. Scalable Vector Graphics (SVG)
Images drawn as shapes
infinitely scalable
XML - Can be added inline to HTML document
https://gist.github.com/hail2u/2884613?short_path=66a60ff

30. Scalable Vector Graphics (SVG)

31. Scalable Vector Graphics (SVG)

32. Scalable Vector Graphics (SVG)
https://github.com/google/brotli

33. Image Formats – Average Size
HTTPArchive mobile websites 15/02/18

34. File Format: WebP
https://caniuse.com/#feat=webp

37. SSIM + WebP
986 KB
http://res.cloudinary.com/dougsillars/image/upload/q_auto,f_auto/v1529005982/IMG_20180614_184507_

38. File Format: Web
<picture>
<source width = "100%" type="image/webp" srcset=”riga.webp">
<img width = "100%" src=”riga_cjpeg_dssim.jpg"
alt=”Riga, Latvia">
</picture>

39. Image Format Use “In The Wild”
500,000 mobile sites
Analyzed 3/15/18

40. Image Format Use “In The Wild”
500,000 mobile sites
Analyzed 3/15/18

41. 4 Simple Image Optimizations
1. Quality
2. Format
3. Sizing
4. Lazy Loading

42. Image Sizing
-

43. Image Sizing

44. Image Sizing
-

45. Image Sizing
624
832
-

46. Image Sizing
624
832
12,979,000 pixels
519,000 pixels
_________________
-
12,460,000 pixels

47. Image Sizing
624
832
12,979,000 pixels
519,000 pixels
_________________
-
12,460,000 pixels

48. Image Sizing
https://twitter.com/paulcalvano/status/928751141843808256

49. Image Sizing
-
Responsive Images:
Generate a set of images
25 KB difference in size

50. Responsive Images
624
832
625,000 pixels
-
106,000 pixels
519,000 pixels
_________________

51. Responsive Breakpoint Generation
https://github.com/cloudinary/responsive_breakpoints_generator
http://www.responsivebreakpoints.com/

52. Responsive Images

53. Responsive Images

54. Responsive Images Use “In The Wild”
500,000 mobile sites
Analyzed 3/15/18

55. Responsive Images Use “In The Wild”
442,000 mobile sites
Analyzed 3/15/18

56. 4 Simple Image Optimizations
1. Quality
2. Format
3. Sizing
4. Lazy Loading

57. Lazy Load
https://calendar.perfplanet.com/2017/progressive-image-loading-using-intersection-observer-and-sqip/

58. Lazy Load
https://calendar.perfplanet.com/2017/progressive-image-loading-using-intersection-observer-and-sqip/

59. Lazy Loading Use “In The Wild”
442,000 mobile sites
Analyzed 3/15/18

60. Lazy Loading Use “In The Wild”
442,000 mobile sites
Analyzed 3/15/18

61. Preview Images

62. Optimizing Content Delivery
Preview Images
https://github.com/technopagan/sqip

63. Lazy Loading: Experiments

64. Lazy Loading: Experiments

65. Lazy Loading: Experiments

66. Animated GIFs
Original MP4
1.4 MB

67. Animated GIFs

68. Animated GIFs
Animated GIF
3.8 MB
270% larger

69. Animated GIFs

70. Animated GIFs
MP4: 256 colors
247KB
93% smaller

71. Animated GIFs
Video Tags:
<video loop autoplay muted playsinline controls = "false” src="goats.mp4”/>
Video is not pre-loaded, will be last to download
Img tags are fast!
<picture>
<source type="video/mp4" srcset=”goats.mp4">
<source type="image/webp" srcset=”goats.webp">
<img src=”goats.gif">
</picture>
https://calendar.perfplanet.com/2017/animated-gif-without-the-gif/

72. Animated GIFs

73. Video

74. 74
Video
Buffer Rage: a state of uncontrollable
fury or violent anger induced by delayed
or interrupted streaming video content
http://www.ineoquest.com/press-releases/new-research-reveals-buffer-rage-as-techs-newest-epidemic

75. Video Quality Metrics
1.Does the Video Start?
2.Does the Video Stall?
3.Does it Look Good?

76. Video Startup
Conviva creen_Streaming_TV_Census_Report_FINAL.pdf
Q1 2018:

77. Video Startup
Conviva creen_Streaming_TV_Census_Report_FINAL.pdf
Q1 2018: Video Startup
16.9B total Video plays
400M Fail to Start
2B Abandoned before Start
~800M hours of video
playback lost

78. Video Startup
Conviva creen_Streaming_TV_Census_Report_FINAL.pdf
Q1 2018: Video Startup
16.9B total Video plays
400M Fail to Start
2B Abandoned before Start
~800M hours of video
playback lost

79. Video Fails To Start

80. Video Startup Failure

81. Video Startup
Conviva creen_Streaming_TV_Census_Report_FINAL.pdf
Q1 2018: Video Startup
16.9B total Video plays
400M Fail to Start
2B Abandoned before Start

82. Video Startup Delay
After 2 seconds,
every additional
second corresponds
to 5.8% increase in
abandonment
https://www.akamai.com/kr/ko/multimedia/documents/technical-publication/video-stream-quality-impacts-viewer-behavior-inferring-causality-using-quasi-experimental-designs-technical-publication.pdf

83. Video Startup Delay
https://www.akamai.com/kr/ko/multimedia/documents/technical-publication/video-stream-quality-impacts-viewer-behavior-inferring-causality-using-quasi-experimental-designs-technical-publication.pdf

84. Preload When It Makes Sense
Ad Playback Video
Download

85. Preload When It Makes Sense
Ad Playback Video
Download

86. Preload When It Makes Sense
Ad Playback
Video
Download

87. ..be Careful with Video Preload
Preload = “auto”

88. …be Careful with Preload

89. Background Video

90. Video Background
Video (as Downloaded): 5.3 MB
Video 5 MB
Audio 250 KB 5% of file
Best Practice:
To save bandwidth, remove the audio stream from videos that
are played silently.

91. Video Background: Mobile
Best Practice:
If Viewport will not support Video…
Don’t Download it

92. Video Background

93. Video Background
33.6 MB
27s
2560 x 1226
10 MBPS

94. Video Background
Best Practice:
Resize Video to reasonable size.
33.6 MB
27s
2560 x 1226
10 MBPS

95. Video Background
Best Practice:
Resize Video to reasonable size.
PROTIP: Renaming the file is not enough…

96. Video Resizing
• 1920x1080: 8.1 MB
• 1280x720: 4.3 MB
• 1080x608: 3.3 MB
• 720x405: 1.76 MB
http://res.cloudinary.com/dougsillars/video/upload/vc_auto,w_720/v1533591785/depend_p2ryou.mp4

97. Video Third Parties

98. Video Third Parties

99. Video Third Parties

100. Video Third Parties

101. Video Streaming
Manifest File:
List of Available
Streams
Player Chooses a Stream Stream Manifest:
List of Video Segments
Player Downloads
Segments into buffer
Video Plays

102. Video Plays
Optimizing Video Delivery
Video Streaming
Player can estimate
network throughput
Optimal video Bitrate
HTTPArchive
Mobile: 1.6 MBPS
Desktop 5 MBPS

103. Video Streaming: Manifest File
https://hls.ted.com/talks/2208.m3u8
Video Tracks
Video Tracks (iFrames)
Audio Tracks &
Subtitles

104. Video Streaming: Manifest File
https://hls.ted.com/talks/2208.m3u8
Video Tracks

105. Video Streaming: Manifest File
https://hls.ted.com/talks/2208.m3u8

106. Optimizing Video Delivery
Video Streaming
Manifest File:
List of Available
Streams
Player selects high
bitrate stream Stream Manifest:
List of Video Segments
Buffer takes a long
time to fill
Video Does Not Play
Player chooses low
bitrate
Buffer Fills Quickly
Video Plays

107. Video Startup
Conviva creen_Streaming_TV_Census_Report_FINAL.pdf
Q1 2018: Video Startup
Average Video Start Time:
N. America: 4.91s
Asia: 3.00s
Europe: 4.30s

108. Video Streaming: Startup Time
Low -> High
High => Low
Middle (Goldilocks)

109. Video Startup
10s 11s 12s 13s 14s 15s 16s 17s
4k Low->High:
4K Middle
(Goldilocks):
4k:High->Low:

110. Video Startup
4k Low->High:
4K Middle
(Goldilocks):
4k:High->Low:
PROs: CONS:
Fast Startup Initial Quality: Low
Other Connections:
Initial Quality: Low
Very slow startup
Fast Connections:
Initial Quality: High
Initial Quality: Good
Initial Startup: Not
fast, but not slow

111. Video Streaming: Manifest File
https://hls.ted.com/talks/2208.m3u8 Testing with WebPageTest 3G

112. Video Streaming: Manifest File
https://hls.ted.com/talks/2208.m3u8

113. Video Streaming: Manifest File
https://hls.ted.com/talks/2208.m3u8

114. Video Streaming: Manifest File
https://hls.ted.com/talks/2208.m3u8
Testing with WebPageTest 3G

115. Video Streaming: Manifest File
600k.m3u8

116. Video Streaming: Manifest File
600k.m3u8

117. Video Streaming: Manifest File
https://hls.ted.com/talks/2208.m3u8

118. Conclusion
Optimize Image:
Quality
Format
Sizing
Lazy Load if Possible
Only download Video when displayed
Strip audio if silent
Resize Videos for Mobile
Audit 3rd Party Videos
Streaming: Start with lower bitrates to speed video playback
Streaming: Conservative bitrates *may* reduce stalls, but will lower quality
Images
Video

119. Summary
Testing:
WebPageTest https://www.webpagetest.org
HttpArchive https://httparchive.org
Images:
ImageMagick https://www.imagemagick.org
SSIM https://github.com/technopagan/cjpeg-dssim
LazySizes https://github.com/aFarkas/lazysizes
Responsive Breakpoints http://www.responsivebreakpoints.com/
Cloudinary https://www.cloudinary.com
Tooling

120. Conclusion
Images and Video
Can Be Beautiful AND Fast