Open Source Tools to Image and Video Optimization

1. Open Source Tools:
Delivering Beautiful and Fast
Images and Video Doug Sillars
@DougSillars
OSCAL

3. 0.5
0.6
0.7
0.8
0.9
Standing in Line Standing on the
edge of a virtual
cliff
Experiencing
Mobile Delays
Solving a Math
Problem
https://www.ericsson.com/res/docs/2016/mobility-report/emr-feb-2016-the-stress-of-steaming-delays.pdf
Stress

4. Large Downloads Induce Delays in Rendering
3s: 53% of Users Abandon Mobile Sites
500ms: 26% Frustration
8% Engagement
100ms: 1% Revenue Walmart & Amazon (Desktop 2001)
4% Mobile Users Throw Their Phones
https://www.doubleclickbygoogle.com/articles/mobile-speed-matters
http://bit.ly/mobileWebStress
http://www.globaldots.com/how-website-speed-affects-conversion-rates/
https://www.mobilejoomla.com/blog/172-responsive-design-vs-server-side-solutions-infographic.html

5. Images & Video make up 75% of Web Content
HTTPArchive mobile websites 15/02/18

6. Image Format Usage in Bytes
HTTPArchive mobile websites 15/02/18

7. Images & Video make up 75% of Web Content
HTTPArchive mobile websites 15/02/18
JPGPNG
GIF

8. Tools We’ll Be Using
• Examples https://github.com/dougsillars/OSCAL
• Monitoring & Testing
• Image Manipulations
• Demos
https://www.webpagetest.org
https://www.imagemagick.org
https://www.cloudinary.com

9. Image Formats – Average Size
HTTPArchive mobile websites 15/02/18

10. Image Formats – Average Size
HTTPArchive mobile websites 15/02/18

11. Scalable Vector Graphics (SVG)
Images drawn as shapes
infinitely scalable
XML - Can be added inline to HTML document
https://gist.github.com/hail2u/2884613?short_path=66a60ff

12. Scalable Vector Graphics (SVG)
Images drawn as shapes
infinitely scalable
XML - Can be added inline to HTML document
https://gist.github.com/hail2u/2884613?short_path=66a60ff

13. Scalable Vector Graphics (SVG)
Images drawn as shapes
infinitely scalable
XML - Can be added inline to HTML document
https://gist.github.com/hail2u/2884613?short_path=66a60ff

14. Scalable Vector Graphics (SVG)

15. Scalable Vector Graphics (SVG)

16. Scalable Vector Graphics (SVG)

17. Scalable Vector Graphics (SVG)
https://commons.wikimedia.org/wiki/User:Quibik/Cleaning_up_SVG_files_manually

18. Scalable Vector Graphics (SVG)
https://github.com/google/brotli

19. Image Compression
Lossy
Image size is reduced by
removing pixel information
magick -quality 85 ireland.jpg ireland85.jpg
http://res.cloudinary.com/dougsillars/image/upload/q_85/v1520504964/IMG_20180301_114117_tzasan.jpg
https://developers.google.com/speed/docs/insights/OptimizeImages

20. Image Compression
Lossy
Image size is reduced by
removing pixel information
magick -quality 85 ireland.jpg ireland85.jpg
http://res.cloudinary.com/dougsillars/image/upload/q_85/v1520504964/IMG_20180301_114117_tzasan.jpg
Google:
Recommends 85%
compression on all
images
https://developers.google.com/speed/docs/insights/OptimizeImages

21. Image Compression Use “In The Wild”
442,000 mobile sites
Analyzed 3/15/18

22. Image Quality Use “In The Wild”
Median Savings (50th percentile):
• 2.83 seconds faster page load
• 419KB less data

23. 100%
2.8 MB
http://res.cloudinary.com/dougsillars/image/upload/v1526279646/ireland_mwnkwo.jpg

24. 85%
1.3 MB
q_85
http://res.cloudinary.com/dougsillars/image/q_85/upload/v1526279646/ireland_mwnkwo.jpg

25. 50%
551KB
q_50
http://res.cloudinary.com/dougsillars/image/q_50/upload/v1526279646/ireland_mwnkwo.jpg

26. 20%
274 KB
q_20
http://res.cloudinary.com/dougsillars/image/q_20/upload/v1526279646/ireland_mwnkwo.jpg

27. Graphing the Results
https://developers.google.com/speed/docs/insights/OptimizeImages

28. Graphing the Results
https://developers.google.com/speed/docs/insights/OptimizeImages

29. Automate Quality vs. File Size
• Butteraugli
• SSIM: Structural SIMilarity
https://github.com/technopagan/cjpeg-dssim
cjpeg-dssim jpegoptim ireland.jpg
http://res.cloudinary.com/dougsillars/image/upload/q_auto/v1526279646/ireland_mwnkwo.jpg

30. SSIM
628-934 KB
http://res.cloudinary.com/dougsillars/image/upload/q_auto/v1526279646/ireland_mwnkwo.jpg

31. Results:
• Full Size Image: https://dougsillars.github.io/OSCAL/
• 85% Optimized: https://dougsillars.github.io/OSCAL/index85.html
• SSIM Optimized: https://dougsillars.github.io/OSCAL/index_ssim.html

32. Results:
• Full Size Image: https://dougsillars.github.io/OSCAL/
• 85% Optimized: https://dougsillars.github.io/OSCAL/index85.html
• SSIM Optimized: https://dougsillars.github.io/OSCAL/index_ssim.html
• Test Load on Motorola G4:

33. Graphing the Results
https://developers.google.com/speed/docs/insights/OptimizeImages

34. Workshop
• GitHub: https://github.com/dougsillars/OSCAL
• Full Size Image: https://dougsillars.github.io/OSCAL/
• Optimized: https://dougsillars.github.io/OSCAL/index_ssim.html
• https://res.cloudinary.com/dougsillars/image/fetch/q_85/<yourImageURL>
• http://res.cloudinary.com/dougsillars/image/upload/q_85/v1526279646/ir
eland_mwnkwo.jpg

35. Image Formats – Average Size
HTTPArchive mobile websites 15/02/18

36. Image Formats – Average Size
HTTPArchive mobile websites 15/02/18

37. File Format: WebP
https://caniuse.com/#feat=webp

38. Convert File Format
http://res.cloudinary.com/dougsillars/image/upload/q_auto,f_auto/v1526279646/ireland_mwnkwo.jpg
convert ireland_cjpeg_dssim.jpg ireland.webp

39. SSIM + WebP
433-505 KB
http://res.cloudinary.com/dougsillars/image/upload/q_auto,f_auto/v1526279646/ireland_mwnkwo.jpg

40. File Format: Web
https://dougsillars.github.io/OSCAL/index_webp.html
<picture>
<source width = "100%" type="image/webp" srcset="ireland.webp">
<img width = "100%" src="ireland_cjpeg_dssim.jpg"
alt="Mizen Head, Ireland">
</picture>

41. Image Format Use “In The Wild”
442,000 mobile sites
Analyzed 3/15/18

42. Image Format Use “In The Wild”
442,000 mobile sites
Analyzed 3/15/18

43. Workshop
• GitHub: https://github.com/dougsillars/OSCAL
• Full Size Image: https://dougsillars.github.io/OSCAL/
• Optimized & Webp: https://dougsillars.github.io/OSCAL/index_webp.html
• https://res.cloudinary.com/dougsillars/image/fetch/q_autp,f_auto/<yourImageU
RL>
• http://res.cloudinary.com/dougsillars/image/upload/q_auto,f_auto/v152627964
6/ireland_mwnkwo.jpg

44. Images & Screens
-

45. Images & Screens

46. Images & Screens
-

47. Images & Screens
624
832
-

48. Images & Screens
624
832
12,979,000 pixels
519,000 pixels
_________________
-
12,460,000 pixels

49. Images & Screens
624
832
12,979,000 pixels
519,000 pixels
_________________
-
12,460,000 pixels

50. Responsive Images
https://twitter.com/paulcalvano/status/928751141843808256

51. Responsive Images
-
Generate a set of images
25 KB difference in size

52. Responsive Images
624
832
625,000 pixels
-
106,000 pixels
519,000 pixels
_________________

53. Responsive Breakpoint Generation
https://github.com/cloudinary/responsive_breakpoints_generator
http://www.responsivebreakpoints.com/

54. Responsive Images

55. Responsive Images
+
Image Format
Create responsive WebP
mogrify -format webp *.jpg

56. Responsive Images + Client Hints

57. Responsive Images

58. Responsive Images Use “In The Wild”
442,000 mobile sites
Analyzed 3/15/18

59. Responsive Images Use “In The Wild”
442,000 mobile sites
Analyzed 3/15/18

60. Workshop
• GitHub: https://github.com/dougsillars/OSCAL
• Full Size Image: https://dougsillars.github.io/OSCAL/
• Optimized & Webp:https://dougsillars.github.io/OSCAL/index_responsive.html
• Client Hints: https://dougsillars.github.io/OSCAL/index_responsive_adv.html
• Fun Example: https://dougsillars.github.io/OSCAL/index_responsive_example.html
• https://res.cloudinary.com/dougsillars/image/fetch/q_autp,f_auto,w_700/<yourImageURL>
• http://res.cloudinary.com/dougsillars/image/upload/q_auto,f_auto,w_700/v1526279646/ir
eland_mwnkwo.jpg

61. Sample Website
https://dougsillars.github.io/OSCAL/index.html

62. Sample Website
https://dougsillars.github.io/OSCAL/index.html https://dougsillars.github.io/OSCAL/index2.html

63. Web Usage
https://www.nngroup.com/articles/scrolling-and-attention/
https://calendar.perfplanet.com/2017/progressive-image-loading-using-intersection-observer-and-sqip/

64. Web Usage
https://www.nngroup.com/articles/scrolling-and-attention/
https://calendar.perfplanet.com/2017/progressive-image-loading-using-intersection-observer-and-sqip/

65. Lazy Loading
https://www.nngroup.com/articles/scrolling-and-attention/
https://calendar.perfplanet.com/2017/progressive-image-loading-using-intersection-observer-and-sqip/

66. Lazy Loading
https://github.com/aFarkas/lazysizes
Time for Some JavaScript!
<img
sizes="(max-width: 1400px) 100vw, 1400px"
class="lazyload"
data-src=<image URL>
alt=”Always use alt text">
<script src="lazysizes.min.js"></script>

67. Lazy Loading - Results
Recall: We added 1 JS
file, so we removed 2
images!

68. Lazy Loading Use “In The Wild”
442,000 mobile sites
Analyzed 3/15/18

69. Lazy Loading Use “In The Wild”
442,000 mobile sites
Analyzed 3/15/18

70. Lazy Loading
https://www.nngroup.com/articles/scrolling-and-attention/
https://calendar.perfplanet.com/2017/progressive-image-loading-using-intersection-observer-and-sqip/

71. Preview Images

72. Preview Images
SVG
979 bytes!
Jpg/Webp
~50KB
https://github.com/technopagan/sqip
<img src=“IMG_20160619_173136306.jpg”
style="background-size: cover; background-image:
url(data:image/svg+xml;base64,<svg text>);"

73. Preview Images

74. Workshop
• GitHub: https://github.com/dougsillars/OSCAL
• Site: https://dougsillars.github.io/OSCAL/index2.html
• Lazy: https://dougsillars.github.io/OSCAL/index2_lazy.html
• Lazy + Preview: https://dougsillars.github.io/OSCAL/index_preview.html
Hint:
In Chrome DevTools, set
speed to “Slow 3G”

75. Animated GIFs
Original MP4
1.4 MB

76. Animated GIFs

77. Animated GIFs
Animated GIF
3.8 MB
270% larger

78. Animated GIFs

79. Animated GIFs
MP4: 256 colors
247KB
93% smaller

80. Animated GIFs
Video Tags are slow:
Video is not pre-loaded, will be last to download
<video autoplay loop muted playsinline controls = "false” src="goats.mp4”/>
Img tags are fast!
<picture>
<source type="video/mp4" srcset=”goats.mp4">
<source type="image/webp" srcset=”goats.webp">
<img src=”goats.gif">
</picture>
https://calendar.perfplanet.com/2017/animated-gif-without-the-gif/

81. Animated GIFs

82. Workshop
• GitHub: https://github.com/dougsillars/OSCAL
• GIF: https://dougsillars.github.io/OSCAL/aGIF.html
• Video: https://dougsillars.github.io/OSCAL/aVideo.html
• Optimized & Webp: https://dougsillars.github.io/OSCAL/aGIF_options.html

83. Video

84. Video Startup Delay

85. Optimizing Content Delivery
Video Requests are deferred

86. Optimizing Content Delivery
Video Requests are deferred

87. 87
Buffer Rage
a state of uncontrollable fury or violent
anger induced by delayed or interrupted
streaming video content
http://www.ineoquest.com/press-releases/new-research-reveals-buffer-rage-as-techs-newest-epidemic

88. Video Startup Delay
After 2 seconds,
every additional
second corresponds
to 5.8% increase in
abandonment
https://www.akamai.com/kr/ko/multimedia/documents/technical-publication/video-stream-quality-impacts-viewer-behavior-inferring-causality-using-quasi-experimental-designs-technical-publication.pdf

89. Video Startup Delay
https://www.akamai.com/kr/ko/multimedia/documents/technical-publication/video-stream-quality-impacts-viewer-behavior-inferring-causality-using-quasi-experimental-designs-technical-publication.pdf

90. Optimizing Video Delivery
What Leads to Startup Delay?
Manifest File:
List of Available
Streams
Player Chooses a Stream Stream Manifest:
List of Video Segments
Player Downloads
Segments into buffer
Video Plays

91. Optimizing Video Delivery
What Leads to Startup Delay?
Example Manifest file:

92. Optimizing Video Delivery
Video Streaming
Manifest File:
List of Available
Streams
Player selects
8.5 MBPS stream Stream Manifest:
List of Video Segments
Buffer takes a long
time to fill
Video Does Not Play
Player immediately changes the
stream choice to 192 KBPS

93. Video Startup

94. Optimizing Video Delivery
What Leads to Startup Delay?
Example Manifest file:

95. Optimizing Video Delivery
What Leads to Startup Delay?
Example Manifest file:

96. Video Startup

97. Video Startup

98. Video Startup

99. Video Startup
10s 11s 12s 13s 14s 15s 16s 17s
4k High->Low:
4K Middle:
4k:Low->High:

100. Video Startup “In the Wild”
75%
20%
5%
https://dougsillars.com/2018/03/29/streaming-video-whats-on-the-web-today/

101. Optimizing Content Delivery
Balancing the Network with Playback
Player uses available streams to balance download with playback

102. Workshop
• GitHub: https://github.com/dougsillars/OSCAL
• 4k High->Low: https://dougsillars.github.io/OSCAL/streaming_4k.html
• 4k:Low->High: https://dougsillars.github.io/OSCAL/streaming_4k_low.html
• 4K Middle: https://dougsillars.github.io/OSCAL/streaming_4k_middle.html

103. Summary
Testing:
WebPageTest https://www.webpagetest.org
WebsiteSpeedTest https://Webspeedtest.cloudinary.com
Images:
ImageMagick https://www.imagemagick.org
SSIM https://github.com/technopagan/cjpeg-dssim
LazySizes https://github.com/aFarkas/lazysizes
Responsive Breakpoints http://www.responsivebreakpoints.com/
Samples: https://github.com/dougsillars/OSCAL
Cloudinary https://www.cloudinary.com
Tooling

104. Conclusion
Images and Video
can be Beautiful AND Fast

105. Contest – Win a €100 Amazon Gift Card
Optimize Images for Fast Page Load
Submit your entry by: May 26 for a chance to win!
Code and tips: https://github.com/dougsillars/dougsillars.github.io
Entry form: http://bit.ly/FastImages
https://dougsillars.github.io