The document discusses next-generation image formats for the internet. It focuses on JPEG XL, a new image format currently being standardized that aims to replace JPEG, PNG, and GIF. JPEG XL provides state-of-the-art compression, supports high bit depths, alpha channels, animation and is designed to be responsive by encoding images into a single file at multiple quality levels. It is also intended to be legacy-friendly by never producing larger files than the original formats and avoiding additional quality loss for existing images.
Nelamangala Call Girls: 🍓 7737669865 🍓 High Profile Model Escorts | Bangalore...
Imagecon 2019 - Jon Sneyers
1. TRANSFORMING THE DIGITAL MEDIA EXPERIENCE
MAY 2019
NEXT-GENERATION
IMAGE FORMATS
FOR THE INTERNET
Jon Sneyers
jon@cloudinary.com
2. N E X T- G E N E R AT I O N I M A G E F O R M AT S
F O R T H E I N T E R N E T
JPEG
3. TRANSFORMING THE DIGITAL MEDIA EXPERIENCE
J P E G
ISO/IEC JTC1/SC29/WG1 aka “Joint Photographic Experts Group”
International
Standardization
Organization
International
Electrotechnical
Commission
International
Telecommunication
UnionJoint Technical Committee 1:
Information Technology
Subcommittee 29:
Coding of audio, picture, multimedia and hypermedia information
Working Group 1:
Coding of still pictures
“Joint”
4. TRANSFORMING THE DIGITAL MEDIA EXPERIENCE
J P E G
JPEG is great!
4
In 1986, the Joint Photographic Experts Group started developing what became the 1992 JPEG standard.
It is still the most widely used image format, and arguably the most successful image format ever.
Features:
• Lossy compression:
• YCbCr, optional chroma subsampling (4:2:0)
• 8x8 DCT (discrete cosine transform)
• Sequential and progressive mode
• Standard:
• 8-bit and 12-bit lossy
• lossless mode up to 16-bit
• up to 4 components (not really specified what they are)
• two choices of entropy coders: Huffman (no patents) or Arithmetic Coding (patent-encumbered)
de facto standard:
grayscale, RGB or CMYK
5. TRANSFORMING THE DIGITAL MEDIA EXPERIENCE
J P E G
But…
5
With a good JPEG encoder like the (de facto) JPEG standard is still quite good!
BUT… it has limitations:
• Only lossy
• Bad for non-photographs (sharp edges, text)
• No alpha channel (transparency)
• Only 8-bit (problem for wide-gamut)
• No animation
• Not quite state-of-the-art entropy coding (Huffman)
• At lower bitrates: obvious compression artifacts (blockiness, color banding, ringing, DCT noise)
}Use PNG instead
⟶ This is why GIF is still around
6. N E X T- G E N E R AT I O N I M A G E F O R M AT S
F O R T H E I N T E R N E T
ALTERNATIVES
TO JPEG
7. TRANSFORMING THE DIGITAL MEDIA EXPERIENCE
A LT E R N AT I V E S T O J P E G
Attempts to replace JPEG
There have been many attempts to replace JPEG
• JPEG 2000
• JPEG XR
• JPEG XT
• WebP
• BPG
• HEIF (HEIC)
• AVIF
So far, none of them has really succeeded (yet)
(although some of them have had some success)
7
}Video codecs used as image codec
8. TRANSFORMING THE DIGITAL MEDIA EXPERIENCE
A LT E R N AT I V E S T O J P E G
Attempts to replace JPEG
There have been many attempts to replace JPEG
• JPEG 2000 (1997-…)
• JPEG XR
• JPEG XT
• WebP
• BPG
• HEIF (HEIC)
• AVIF
8
👍
• Supports lossless
• Alpha
• High bit depth
• Less blockiness
👎
• Slow
• Blur
• Not that much better
• Initially no open
source software
9. TRANSFORMING THE DIGITAL MEDIA EXPERIENCE
A LT E R N AT I V E S T O J P E G
Attempts to replace JPEG
There have been many attempts to replace JPEG
• JPEG 2000
• JPEG XR (2007-2012)
• JPEG XT
• WebP
• BPG
• HEIF (HEIC)
• AVIF
9
👍
• Supports lossless
• Alpha
• High bit depth
• Less blockiness
• Faster than J2K
👎
• Worse than J2K
• Single vendor
pushing it (Microsoft)
10. TRANSFORMING THE DIGITAL MEDIA EXPERIENCE
A LT E R N AT I V E S T O J P E G
Attempts to replace JPEG
There have been many attempts to replace JPEG
• JPEG 2000
• JPEG XR
• JPEG XT (2013-2016)
• WebP
• BPG
• HEIF (HEIC)
• AVIF
10
👍
• Backwards compatible
• Supports lossless
• Alpha
• High bit depth
👎
• Backwards compatible
• Little incentive to
upgrade decoders
• No compression
improvement
11. TRANSFORMING THE DIGITAL MEDIA EXPERIENCE
A LT E R N AT I V E S T O J P E G
Attempts to replace JPEG
There have been many attempts to replace JPEG
• JPEG 2000
• JPEG XR
• JPEG XT
• WebP (2010-2019)
• BPG
• HEIF (HEIC)
• AVIF
11
👍
• Supports lossless
• Alpha
• Animation
• Open source
• Royalty-free
👎
• Only 8-bit
• No progressive mode
• Obligatory 4:2:0
• Max dimensions are
16383x16383
• No formal standard
12. TRANSFORMING THE DIGITAL MEDIA EXPERIENCE
A LT E R N AT I V E S T O J P E G
Attempts to replace JPEG
There have been many attempts to replace JPEG
• JPEG 2000
• JPEG XR
• JPEG XT
• WebP
• BPG (2014-2018)
• HEIF (HEIC) (2013-2017)
• AVIF
12
👍
• Supports lossless
• Alpha, Depth
• High bit depth
• Animation
• Very good compression
👎
• No progressive mode
• Patent mess
13. TRANSFORMING THE DIGITAL MEDIA EXPERIENCE
A LT E R N AT I V E S T O J P E G
Attempts to replace JPEG
There have been many attempts to replace JPEG
• JPEG 2000
• JPEG XR
• JPEG XT
• WebP
• BPG
• HEIF (HEIC)
• AVIF (2018-…)
13
👍
• Supports lossless
• Alpha
• Up to 12-bit
• Animation
• Very good compression
• Royalty-free
👎
• No progressive mode
• Slow
• Complicated
14. TRANSFORMING THE DIGITAL MEDIA EXPERIENCE
A LT E R N AT I V E S T O J P E G
Attempts to replace JPEG
14
Patent
mess
Limitations
(8 bit, 4:2:0)
Complexity
No progressive, only sequential
Compression improvement: “meh”
15. 15
A LT E R N AT I V E S T O J P E G
Attempts to replace JPEG
1. JPEG
2. GIF
3. PNG
4. TIFF
5. BMP
6. PPM
7. JPEG 2000
8. WebP
9. JPEG XR
10. JPEG XT
11. BPG
12. FLIF
13. HEIC
14. AVIF
1. JPEG
2. GIF
3. PNG
4. TIFF
5. BMP
6. PPM
7. JPEG 2000
8. WebP
9. JPEG XR
10. JPEG XT
11. BPG
12. FLIF
13. HEIC
14. AVIF
15. JPEG XL
xkcd.com/927
16. N E X T- G E N E R AT I O N I M A G E F O R M AT S
F O R T H E I N T E R N E T
JPEG XL
17. TRANSFORMING THE DIGITAL MEDIA EXPERIENCE 17
Pik Image Compression
♠
Alexander Rhatushnyak, Evgenii Kliuchnikov, Jan Wassenberg, Jeffrey Lim,
Jyrki Alakuijala, Krzysztof Potempa, Lode Vandevenne, Renata Khasanova,
Robert Obryk, Sami Boukortt, Thomas Fischbacher, Zoltan Szabadka
= +
proposal 2
proposal 3
proposal 4
proposal 5
proposal 6
proposal 1
proposal 7
>
ISO/IEC JTC1/SC29/WG1
J P E G X L
History
20. TRANSFORMING THE DIGITAL MEDIA EXPERIENCE
J P E G X L
Features / Goals
20
• Royalty-free, Free & Open Source software available from the start
• All the features expected for a modern image codec:
• Alpha (and other extra channels like Depth)
• High bit depth (up to 16-bit)
• Supports lossless
• State-of-the-art compression
• Animation
• Progressive mode / “Responsive by design”
• Legacy-friendly
• High quality
• Can replace JPEG, PNG and GIF
22. TRANSFORMING THE DIGITAL MEDIA EXPERIENCE
J P E G X L
Responsive by design
22
In the 1990s…
Internet was slow
for everyone
Screen sizes and
resolutions didn’t
vary much
14.4 - 33.6 kbit/s
14 to 21 inch
800x600 or 1024x768
23. TRANSFORMING THE DIGITAL MEDIA EXPERIENCE
J P E G X L
Responsive by design
23
Today
Internet speed and screen sizes and resolutions vary a lot!
Smartwatch
2 inch, 360x480
8K Smart TV
80 inch, 7680x4320
25. TRANSFORMING THE DIGITAL MEDIA EXPERIENCE
J P E G X L
Responsive by design
25
w4x.jpg, 1.9 MBw2x.jpg, 490 KBw1x.jpg, 110 KBw0.5x.jpg, 30 KB
LQIP.gif
271 bytes
Current
approach:
many files
26. TRANSFORMING THE DIGITAL MEDIA EXPERIENCE
J P E G X L
Responsive by design
26
w.jxl, 1.7 MBw.jxl, first 806 KBw.jxl, first 206 KBw.jxl, first 50 KB
w.jxl
first 200 bytes
JPEG XL
approach:
one file!
33. TRANSFORMING THE DIGITAL MEDIA EXPERIENCE
J P E G X L
Legacy-friendly
33
JPEG PNG GIF
New format
Pixels
Legacy decoder
New encoder
Additional loss and/or more bytes than original!
Existing image files
34. TRANSFORMING THE DIGITAL MEDIA EXPERIENCE
J P E G X L
Legacy-friendly
34
sunset.jpg
6173 bytes
sunset.webp
11110 bytes
35. TRANSFORMING THE DIGITAL MEDIA EXPERIENCE
J P E G X L
Legacy-friendly
35
JPEG PNG GIF
New format
Pixels
Legacy decoder
New encoder
Additional loss and/or more bytes than original!
Existing image files
36. TRANSFORMING THE DIGITAL MEDIA EXPERIENCE
J P E G X L
Legacy-friendly
36
JPEG PNG8 GIF
JPEG XL
No additional loss, always smaller than original!
Existing image files
PNG24
Pixels Palette pixelsDCT coefficients
37. TRANSFORMING THE DIGITAL MEDIA EXPERIENCE
J P E G X L
Legacy-friendly
37
sunset.jpg
6173 bytes
sunset.jxl
3320 bytes
sunset.jpg
6173 bytes reversible
38. TRANSFORMING THE DIGITAL MEDIA EXPERIENCE
J P E G X L
Legacy-friendly
38
sunset.jpg
6173 bytes
sunset.jxl
3320 bytes
sunset.jpg
6173 bytes reversible
Improved dequantization
(reduces blockiness / color banding)
40. 40TRANSFORMING THE DIGITAL MEDIA EXPERIENCE
J P E G X L
High quality
• Video codecs focus on low bitrates
• They have a lot of pixels to encode!
• You only see a single frame for 16-42 ms
• A lot of smoothing and “distilling”
• Great for video
• Not so good for high-quality still images
Original HEIC
41. 41TRANSFORMING THE DIGITAL MEDIA EXPERIENCE
J P E G X L
High quality
• Visually lossless = lossy, but human eyes cannot see the difference
• XYB color space models human vision
• Adaptive quantization (guided by a perceptual metric)
Distribution of cone cells in the fovea
42. 42TRANSFORMING THE DIGITAL MEDIA EXPERIENCE
J P E G X L
How it works: adaptive quantization
• JPEG:
Same quality in every region
• JPEG XL:
Quality can be different in
different regions; it is adjusted
automatically based on
perceptual metrics
45. 45TRANSFORMING THE DIGITAL MEDIA EXPERIENCE
J P E G X L
Universal
JPEG XL is designed to be a universal and future-proof image codec
• Any kind of image content: photographs, illustrations, screenshots, rendered images, document
scans, medical imaging, game graphics, UI elements, …
• Support for extra channels like alpha, depth, spot colors, …
• No limits on the image dimensions
• High bit depth / wide gamut / HDR
• Covers the whole quality spectrum from very low bitrates all the way to lossless
• Various trade-offs between encode/decode speed and compression density
46. 46
J P E G X L
Universal
Non-photographic 8-bit photo 12-bit photo
TRANSFORMING THE DIGITAL MEDIA EXPERIENCE