13. TRANSFORMING THE DIGITAL MEDIA EXPERIENCE
Operational Transparency
● http://status.cloudinary.com
● Server-Timing APIs
● Public security bug bounty program
18. TRANSFORMING THE DIGITAL MEDIA EXPERIENCE
Media manipulation
AI and the impact on image management
Accuracy Does it scale
Manual editing
service
Accurate No
Classic Computer
Vision
Not Accurate Yes
Modern Machine
Learning
Accurate Yes
Choose your option
19. TRANSFORMING THE DIGITAL MEDIA EXPERIENCE
Media manipulation
AI and the impact on image management
Accuracy Does it scale
Manual editing
service
Accurate No
Classic Computer
Vision
Not Accurate Yes
Modern Machine
Learning
Accurate Yes
Choose your option
20. TRANSFORMING THE DIGITAL MEDIA EXPERIENCE
Media manipulation
AI and the impact on image management
Accuracy Does it scale
Manual editing
service
Accurate No
Classic Computer
Vision
Not Accurate Yes
Modern Machine
Learning
Accurate Yes
Choose your option
“Car”
21. TRANSFORMING THE DIGITAL MEDIA EXPERIENCE
Smart auto-cropping
AI and the impact on image management
22. TRANSFORMING THE DIGITAL MEDIA EXPERIENCE
Smart auto-cropping
AI and the impact on image management
23. TRANSFORMING THE DIGITAL MEDIA EXPERIENCE
AI-based subject identification
AI and the impact on image management
Deep Neural Network
Neural Network
Compression
methods
AWS Elastic
Inference
Model runs x10 faster! GPU acceleration
24. TRANSFORMING THE DIGITAL MEDIA EXPERIENCE
Smart Video Cropping
AI and the impact on image management
Original Clip AI generated heat-map
25. TRANSFORMING THE DIGITAL MEDIA EXPERIENCE
Smart Video Cropping
AI and the impact on image management
Square Portrait
AI-Based dynamic
cropping
26. TRANSFORMING THE DIGITAL MEDIA EXPERIENCE
AI and the impact on image management
AI-based background removal
Original Background Removed
27. TRANSFORMING THE DIGITAL MEDIA EXPERIENCE
AI and the impact on image management
How we do it?
Original Background Removed
Foreground
Detection
Model
Foreground
Segmentation
Model
Refinement
Model
+
Post
Processing
29. TRANSFORMING THE DIGITAL MEDIA EXPERIENCE
AI and the impact on image management
Video background removal
30. TRANSFORMING THE DIGITAL MEDIA EXPERIENCE
AI and the impact on image management
Few-Shot object detector
Few Shot
Object
Detector
Create your own object detector with only a few examples
Small amount of examples API Calls
31. TRANSFORMING THE DIGITAL MEDIA EXPERIENCE
Areas of Focus
Innovation
Machine
Learning
Data
Insights
Web
Standards
& OSS
32. JPEG XL
Next Gen Image Format
Jon Sneyers
Senior image researcher
33. TRANSFORMING THE DIGITAL MEDIA EXPERIENCE
= +
proposal 2
proposal 3
proposal 4
proposal 5
proposal 6
proposal 1
proposal 7
>
ISO/IEC JTC1/SC29/WG1
34. TRANSFORMING THE DIGITAL MEDIA EXPERIENCE
Responsive by design
Legacy-friendlyUniversal
High quality
35. TRANSFORMING THE DIGITAL MEDIA EXPERIENCE
w4x.jpg, 1.9 MBw2x.jpg, 490 KBw1x.jpg, 110 KBw0.5x.jpg, 30 KB
LQIP.gif
271 bytes
Many Files
Current Approach
36. TRANSFORMING THE DIGITAL MEDIA EXPERIENCE
full 1.7 MBfirst 806 KBfirst 206 KBfirst 50 KB
first 200
bytes
One File
Responsive By Design
37. TRANSFORMING THE DIGITAL MEDIA EXPERIENCE
sunset.webp
11,110 bytes
sunset.jpg
6,173 bytes
Legacy Friendly
JPEG XL
38. TRANSFORMING THE DIGITAL MEDIA EXPERIENCE
sunset.jxl
3,320 bytes
sunset.jpg
6,173 bytes reversible
Legacy Friendly
JPEG XL
39. TRANSFORMING THE DIGITAL MEDIA EXPERIENCE
sunset.jxl
3,320 bytes
sunset.jpg
6,173 bytes reversible
Improved dequantization
(reduces blockiness / color banding)
Legacy Friendly
JPEG XL
40. TRANSFORMING THE DIGITAL MEDIA EXPERIENCE
Areas of Focus
Innovation
Machine
Learning
Data
Insights
Web
Standards
& OSS
50. TRANSFORMING THE DIGITAL MEDIA EXPERIENCE
Use of Microdata
State of the Media
<meta name="description" content="...">
<meta name="keywords" content="...">
<meta property="og:title" content="..." />
<meta property="og:description" content="..." />
<meta name="twitter:title" content="...">
<meta name="twitter:description" content="...">
<meta property="og:image"
content="https://res.cloudinary.com/.../hero-img.png" />
<meta name="twitter:image:src"
content="https://res.cloudinary.com/.../hero-img.png">