Presented 11/2/2019

1. Delivering Beautiful and Fast
Images and Video
Doug Sillars
@DougSillars
Edinburgh UX
February 11, 2019

2. Contact Me:
@DougSillars
Doug.Sillars@gmail.com
www.dougsillars.com
Doug Sillars
Freelance Developer Relations
Performance Audits: Web/Native
Workshops:
Performance/Images/Video
http://bit.ly/HighPerformanceAndroidApps

4. 0.5
0.6
0.7
0.8
0.9
Standing in Line Standing on the
edge of a virtual
cliff
Experiencing
Mobile Delays
Solving a Math
Problem
https://www.ericsson.com/res/docs/2016/mobility-report/emr-feb-2016-the-stress-of-steaming-delays.pdf
Stress

5. What Is Fast?
https://commons.wikimedia.org/wiki/File:Lightning_Pritzerbe_01_(MK).jpg

6. Fast is a Human Perception
• 100ms:
• 1s:
• 10s:
https://www.nngroup.com/articles/response-times-3-important-limits/
Perceived as instantaneous
Limit for uninterrupted train of thought: Delay is
noticed, but generally accepted
Limit for keeping focus

7. Fast is a Human Perception
In 1993:

8. Fast is a Human Perception
• 100ms:
• 1s:
• 10s: 3-5s:
Perceived as instantaneous
Limit for uninterrupted train of thought: Delay is noticed,
but generally accepted
Limit for keeping focus

9. Fast is a Human Perception
• 100ms:
• 1s:
• 10s: 3-5s:
Perceived as instantaneous
Limit for uninterrupted train of thought: Delay is noticed,
but generally accepted
Limit for keeping focus
65ms: ranged from 34-164ms
https://calendar.perfplanet.com/2018/magic-numbers/

10. Large Downloads Induce Delays in Rendering
3s: 53% of Users Abandon Mobile Sites
500ms: 26% Frustration
8% Engagement
100ms: 1% Revenue Walmart & Amazon (Desktop 2001)
4% Mobile Users Throw Their Phones
https://www.doubleclickbygoogle.com/articles/mobile-speed-matters
http://bit.ly/mobileWebStress
http://www.globaldots.com/how-website-speed-affects-conversion-rates/
https://www.mobilejoomla.com/blog/172-responsive-design-vs-server-side-solutions-infographic.html

11. Images & Video make up 75% of Web Content
HTTPArchive mobile websites 15/02/18

12. Deliver Original Content

13. Is Content In Ideal Web Format?

14. Screenshot: File Size Optimizations?
Tested 1300 “Screen Shot” images on the web
https://dougsillars.com/2019/02/10/using-screenshots-in-production/

15. 4 Simple Image Optimizations

16. 4 Simple Image Optimizations
1. Quality
2. Format
3. Sizing
4. Lazy Loading

17. 4 Simple Image Optimizations
1. Quality
2. Format
3. Sizing
4. Lazy Loading

18. 4 Simple Image Optimizations
Httparchive.org Webpagetest.org

19. 4 Simple Image Optimizations
1. Quality
2. Format
3. Sizing
4. Lazy Loading

20. Image Quality
magick -quality 85 riga.jpg riga85.jpg
http://res.cloudinary.com/dougsillars/image/upload/q_85/v1520504964/IMG_20180301_114117_tzasan.jpg
Lighthouse:
Recommends 85% quality on all images
https://developers.google.com/speed/docs/insights/OptimizeImages

21. 100%
3.6 MB
http://res.cloudinary.com/dougsillars/image/upload/v1529005982/IMG_20180614_184507_ssuk1i.jpg

22. 85%
1.87 MB
q_85
http://res.cloudinary.com/dougsillars/image/upload/q_85/v1529005982/IMG_20180614_184507_ssuk1i.jpg

23. Image Quality Use “In The Wild”
500,000 mobile sites
Analyzed 3/15/18

24. Image Quality Use “In The Wild”
Median Savings (50th percentile):
• 2.83 seconds faster page load
• 419KB less data

25. 20%
513 KB
q_20
http://res.cloudinary.com/dougsillars/image/upload/q_20/v1529005982/IMG_20180614_184507_ssuk1i.jpg

26. Graphing the Results
https://developers.google.com/speed/docs/insights/OptimizeImages

27. Graphing the Results
https://developers.google.com/speed/docs/insights/OptimizeImages

28. Automate Quality vs. File Size
• Butteraugli
• SSIM: Structural SIMilarity
https://github.com/technopagan/cjpeg-dssim
cjpeg-dssim jpegoptim riga.jpg
http://res.cloudinary.com/dougsillars/image/upload/q_auto/v1529005982/IMG_20180614_184507_ssuk1i.jpg

29. SSIM
1.46 MB
http://res.cloudinary.com/dougsillars/image/upload/q_auto/v1529005982/IMG_20180614_184507_ssuk1i.jpg

30. Graphing the Results
https://developers.google.com/speed/docs/insights/OptimizeImages

31. Results:
• Test Load on Motorola G4:

32. 4 Simple Image Optimizations
1. Quality
2. Format
3. Sizing
4. Lazy Loading

33. Image Formats – Average Size
HTTPArchive mobile websites 15/02/18

34. Image Formats – Average Size
HTTPArchive mobile websites 15/02/18

35. Scalable Vector Graphics (SVG)
Images drawn as shapes
infinitely scalable
XML - Can be added inline to HTML document
https://gist.github.com/hail2u/2884613?short_path=66a60ff

36. Scalable Vector Graphics (SVG)
Images drawn as shapes
infinitely scalable
XML - Can be added inline to HTML document
https://gist.github.com/hail2u/2884613?short_path=66a60ff

37. Scalable Vector Graphics (SVG)

38. Scalable Vector Graphics (SVG)

39. Scalable Vector Graphics (SVG)
KB
Original 946
Optimized 1

40. Scalable Vector Graphics (SVG)
KB
Original 946
Optimized 1

41. Scalable Vector Graphics (SVG)
KB
Original 946
Optimized 1
GZip 687 bytes

42. Scalable Vector Graphics (SVG)
KB
Original 946
Optimized 1
GZip 687 bytes
Brotli 525 bytes

43. Scalable Vector Graphics (SVG)
KB
Original 946
Optimized 1
GZip 687 bytes
Brotli 525 bytes

44. Scalable Vector Graphics (SVG)
<style >
.svgorange {
filter: invert(.5) sepia(1)
saturate(5) hue-rotate(5deg);
}
</style>
<img class="svgorange" src ="map-
marker-circle.svg">
https://medium.com/@union_io/swapping-fill-color-on-image-tag-svgs-using-css-filters-fa4818bf7ec6

45. Scalable Vector Graphics (SVG)

46. Image Formats – Average Size
HTTPArchive mobile websites 15/02/18

47. File Format: WebP
https://caniuse.com/#feat=webp

48. SSIM + WebP
986 KB
http://res.cloudinary.com/dougsillars/image/upload/q_auto,f_auto/v1529005982/IMG_20180614_184507_

49. File Format: Web
<picture>
<source width = "100%" type="image/webp" srcset=”riga.webp">
<img width = "100%" src=”riga_cjpeg_dssim.jpg"
alt=”Riga, Latvia">
</picture>

50. Image Format Use “In The Wild”
500,000 mobile sites
Analyzed 3/15/18

51. Image Format Use “In The Wild”
500,000 mobile sites
Analyzed 3/15/18

52. 4 Simple Image Optimizations
1. Quality
2. Format
3. Sizing
4. Lazy Loading

53. Image Sizing
-

54. Image Sizing

55. Image Sizing
-

56. Image Sizing
624
832
-

57. Image Sizing
624
832
12,979,000 pixels
519,000 pixels
_________________
-
12,460,000 pixels

58. Image Sizing
624
832
12,979,000 pixels
519,000 pixels
_________________
-
12,460,000 pixels

59. Image Sizing
https://twitter.com/paulcalvano/status/928751141843808256

60. Image Sizing
-
Responsive Images:
Generate a set of images
25 KB difference in size

61. Responsive Images
624
832
625,000 pixels
-
106,000 pixels
519,000 pixels
_________________

62. Responsive Breakpoint Generation
https://github.com/cloudinary/responsive_breakpoints_generator
http://www.responsivebreakpoints.com/

63. Responsive Images

64. Responsive Images

65. Responsive Images Use “In The Wild”
500,000 mobile sites
Analyzed 3/15/18

66. Responsive Images Use “In The Wild”
442,000 mobile sites
Analyzed 3/15/18

67. 4 Simple Image Optimizations
1. Quality
2. Format
3. Sizing
4. Lazy Loading

68. Lazy Load
https://calendar.perfplanet.com/2017/progressive-image-loading-using-intersection-observer-and-sqip/

69. Lazy Load
https://calendar.perfplanet.com/2017/progressive-image-loading-using-intersection-observer-and-sqip/

70. Lazy Loading Use “In The Wild”
442,000 mobile sites
Analyzed 3/15/18

71. Lazy Loading Use “In The Wild”
442,000 mobile sites
Analyzed 3/15/18

72. Preview Images

73. Optimizing Content Delivery
Preview Images
https://github.com/technopagan/sqip

74. Lazy Loading: Experiments

75. Lazy Loading: Chrome Experiments

76. Lazy Loading: Chrome Experiments

77. Animated GIFs

78. Animated GIFs
“The Graphics Interchange Format is not
intended as a platform for animation, even
though it can be done in a limited way.”
-GIF89a Specification
https://www.w3.org/Graphics/GIF/spec-gif89a.txt

79. Animated GIFs
Original MP4
1.4 MB

80. Animated GIFs

81. Animated GIFs
Animated GIF
3.8 MB
270% larger

82. Animated GIFs

83. Animated GIFs
MP4: 256 colors
247KB
93% smaller

84. Animated GIFs
Video Tags:
<video loop autoplay muted playsinline controls = "false” src="goats.mp4”/>
Video is not pre-loaded, will be last to download
Img tags are fast!
<picture>
<source type="video/mp4" srcset=”goats.mp4">
<source type="image/webp" srcset=”goats.webp">
<img src=”goats.gif">
</picture>
https://calendar.perfplanet.com/2017/animated-gif-without-the-gif/

85. Animated GIFs

86. Animated GIFs: Social Media

87. Animated GIFs: Social Media
#FAKENEWS

88. Animated GIFs: Social Media

89. Animated GIFs
https://dougsillars.com/2019/01/15/state-of-the-web-animated-gifs/

90. Addendum: What Are Your Customers Saying?

91. Addendum: Save-Data
Save-Data: on

92. Addendum: Save-Data

93. Addendum: Save-Data

94. Addendum: Save-Data

95. Addendum: Save-Data
q_auto -> q_auto:eco
180KB -> 135 KB
https://res.cloudinary.com/dougsillars/image/upload/w_1400,q_auto/v1540978150/IMG_20181028_153301_okeb2b.jpg

96. Addendum 2: Network Info

97. Addendum 2: Network Info
// Network type that browser uses
navigator.connection.type;
// Effective bandwidth estimate
navigator.connection.downlink
// Effective round-trip time estimate
navigator.connection.rtt
// Upper bound on the downlink speed of the first network hop
navigator.connection.downlinkMax

98. Addendum 2: Network Info
// Network type that browser uses
navigator.connection.type;
// Effective bandwidth estimate
navigator.connection.downlink
// Effective round-trip time estimate
navigator.connection.rtt
// Upper bound on the downlink speed of the first network hop
navigator.connection.downlinkMax
500 KBPS

99. Addendum 3: Base 64 Encoding
Images embedded as Base64 in your HTML/CSS/JS
• Fewer Requests
• Images Now Block Rendering of Page
• Images are 20-30% larger
• Caching is limited
• Difficult to reference more than once

100. Addendum 3: Base 64 Encoding
• 91KB CSS Shared as University Template
• 48KB is SVG
• Never Appears on Any Page
Departments, colleges and other units of
the University of Nebraska–Lincoln should
use the Nebraska N as a main identifier,
not the University seal, on all publications,
invitations, websites and other electronic
media.
https://unlcms.unl.edu/wdn/templates_4.1/css/all.css?dep=4.1.36

101. Addendum 3: Base 64 Encoding
https://cdn.glowing.com/generated/css/base.474240e8485dbff13fd3652d24ef83bc.css

102. Conclusion
Optimize Image:
Quality
Format
Sizing
Lazy Load if Possible
aGIFs to movies
Honor customer requests for less data
Observe Network Conditions
Avoid Base64 encoding
Images

103. Video

104. 104
Video
Buffer Rage: a state of uncontrollable
fury or violent anger induced by delayed
or interrupted streaming video content
http://www.ineoquest.com/press-releases/new-research-reveals-buffer-rage-as-techs-newest-epidemic

105. Video Quality Metrics
1.Does the Video Start?
2.Does the Video Stall?
3.Does it Look Good?

106. Video Startup
Conviva creen_Streaming_TV_Census_Report_FINAL.pdf
Q1 2018:

107. Video Startup
Conviva creen_Streaming_TV_Census_Report_FINAL.pdf
Q1 2018: Video Startup
16.9B total Video plays
400M Fail to Start
2B Abandoned before Start
~800M hours of video
playback lost

108. Video Startup
Conviva creen_Streaming_TV_Census_Report_FINAL.pdf
Q1 2018: Video Startup
16.9B total Video plays
400M Fail to Start
2B Abandoned before Start
~800M hours of video
playback lost

109. Video Fails To Start

110. Video Startup Failure

111. Video Startup
Conviva creen_Streaming_TV_Census_Report_FINAL.pdf
Q1 2018: Video Startup
16.9B total Video plays
400M Fail to Start
2B Abandoned before Start

112. Video Startup Delay
After 2 seconds,
every additional
second corresponds
to 5.8% increase in
abandonment
https://www.akamai.com/kr/ko/multimedia/documents/technical-publication/video-stream-quality-impacts-viewer-behavior-inferring-causality-using-quasi-experimental-designs-technical-publication.pdf

113. Video Startup Delay
https://www.akamai.com/kr/ko/multimedia/documents/technical-publication/video-stream-quality-impacts-viewer-behavior-inferring-causality-using-quasi-experimental-designs-technical-publication.pdf

114. Preload When It Makes Sense
Ad Playback Video
Download

115. Preload When It Makes Sense
Ad Playback Video
Download

116. Preload When It Makes Sense
Ad Playback
Video
Download

117. ..be Careful with Video Preload
Preload = “auto”

118. …be Careful with Preload

119. Background Video

120. Video Background
Video (as Downloaded): 5.3 MB
Video 5 MB
Audio 250 KB 5% of file
Best Practice:
To save bandwidth, remove the audio stream from videos that
are played silently.

121. Video Background: Mobile
Best Practice:
If Viewport will not support Video…
Don’t Download it

122. Video Background

123. Video Background
33.6 MB
27s
2560 x 1226
10 MBPS

124. Video Background
Best Practice:
Resize Video to reasonable size.
33.6 MB
27s
2560 x 1226
10 MBPS

125. Video Background
Best Practice:
Resize Video to reasonable size.
PROTIP: Renaming the file is not enough…

126. Video Resizing
• 1920x1080: 8.1 MB
• 1280x720: 4.3 MB
• 1080x608: 3.3 MB
• 720x405: 1.76 MB
http://res.cloudinary.com/dougsillars/video/upload/vc_auto,w_720/v1533591785/depend_p2ryou.mp4

127. Video Is Not Cheap

128. Video Is Not Cheap

129. Video Is Not Cheap

130. Video Is Not Cheap

131. Video Is Not Cheap
After 20 free videos
Each Video Download costs:

132. Video Is Not Cheap

133. Video Is Not Cheap

134. Third Parties
No initial Video Download, but expect 700KB -1MB
additional JavaScript on page load

135. Third Parties

136. Video Third Parties

137. Video Third Parties

138. Video Third Parties

139. Video Third Parties

140. Conclusion
Optimize Image:
Quality
Format
Sizing
Lazy Load if Possible
Only download Video when displayed
Strip audio if silent
Resize Videos for Mobile
Audit 3rd Party Videos
Streaming: Start with lower bitrates to speed video playback
Streaming: Conservative bitrates *may* reduce stalls, but will lower quality
Images
Video

141. Summary
Testing:
WebPageTest https://www.webpagetest.org
HttpArchive https://httparchive.org
Images:
ImageMagick https://www.imagemagick.org
SSIM https://github.com/technopagan/cjpeg-dssim
LazySizes https://github.com/aFarkas/lazysizes
Responsive Breakpoints http://www.responsivebreakpoints.com/
Cloudinary https://www.cloudinary.com
Tooling

142. Conclusion
Images and Video
Can Be Beautiful AND Fast

143. Video Streaming
Manifest File:
List of Available
Streams
Player Chooses a Stream Stream Manifest:
List of Video Segments
Player Downloads
Segments into buffer
Video Plays

144. Video Plays
Optimizing Video Delivery
Video Streaming
Player can estimate
network throughput
Optimal video Bitrate
HTTPArchive
Mobile: 1.6 MBPS
Desktop 5 MBPS

145. Video Streaming: Manifest File
https://hls.ted.com/talks/2208.m3u8
Video Tracks
Video Tracks (iFrames)
Audio Tracks &
Subtitles

146. Video Streaming: Manifest File
https://hls.ted.com/talks/2208.m3u8
Video Tracks

147. Video Streaming: Manifest File
https://hls.ted.com/talks/2208.m3u8

148. Optimizing Video Delivery
Video Streaming
Manifest File:
List of Available
Streams
Player selects high
bitrate stream Stream Manifest:
List of Video Segments
Buffer takes a long
time to fill
Video Does Not Play
Player chooses low
bitrate
Buffer Fills Quickly
Video Plays

149. Video Startup
Conviva creen_Streaming_TV_Census_Report_FINAL.pdf
Q1 2018: Video Startup
Average Video Start Time:
N. America: 4.91s
Asia: 3.00s
Europe: 4.30s

150. Video Streaming: Startup Time
Low -> High
High => Low
Middle (Goldilocks)

151. Video Startup
10s 11s 12s 13s 14s 15s 16s 17s
4k Low->High:
4K Middle
(Goldilocks):
4k:High->Low:

152. Video Startup
4k Low->High:
4K Middle
(Goldilocks):
4k:High->Low:
PROs: CONS:
Fast Startup Initial Quality: Low
Other Connections:
Initial Quality: Low
Very slow startup
Fast Connections:
Initial Quality: High
Initial Quality: Good
Initial Startup: Not
fast, but not slow

153. Video Streaming: Manifest File
https://hls.ted.com/talks/2208.m3u8 Testing with WebPageTest 3G

154. Video Streaming: Manifest File
https://hls.ted.com/talks/2208.m3u8

155. Video Streaming: Manifest File
https://hls.ted.com/talks/2208.m3u8

156. Video Streaming: Manifest File
https://hls.ted.com/talks/2208.m3u8
Testing with WebPageTest 3G

157. Video Streaming: Manifest File
600k.m3u8

158. Video Streaming: Manifest File
600k.m3u8

159. Video Streaming: Manifest File
https://hls.ted.com/talks/2208.m3u8