SlideShare ist ein Scribd-Unternehmen logo
1 von 23
Downloaden Sie, um offline zu lesen
All rights reserved. ©2020
All rights reserved. ©2020
Machine Learning Based Video Coding
Enhancements for HTTP Adaptive Streaming
ACM MMSys’21 Doctoral Symposium
September 30, 2021
Ekrem Çetinkaya
Christian Doppler Laboratory ATHENA | Alpen-Adria-Universität Klagenfurt | Austria
ekrem.cetinkaya@aau.at | athena.itec.aau.at
1
All rights reserved. ©2020
● Introduction
● Research Questions
● Methodology & Existing Results
● Ongoing & Future Work
● Q & A
Agenda
All rights reserved. ©2020
2
Introduction
All rights reserved. ©2020
3
All rights reserved. ©2020
Video Streaming
Share in the Internet Traffic
82%
Content Characteristics
1 Million
minutes
Video Streamed Every Second
As of 2021
* Cisco VNI Forecast Highlights (2021)
All rights reserved. ©2020
4
All rights reserved. ©2020
HTTP Adaptive Streaming (HAS)
Very Nice Video
Play
Play
5
240
kbps
Client HAS Server
1200
kbps
3500
kbps
480
kbps
2500
kbps
7000
kbps
All rights reserved. ©2020
Video Encoding
Block
Partitioning
Motion
Compensation
Transformation
& Quantization
Entropy Coding
Entropy
Decoding
Inverse
Transformation &
Inverse
Quantization
Inter or Intra
Prediction
Picture Buffer In-loop Filtering
6
All rights reserved. ©2020
Video Codecs
C. Feldmann, “State of Compression Standards - VVC”, 2020, https://bitmovin.com/compression-standards-vvc-2020/
Vanne et.al., “Comparative Rate-Distortion-Complexity Analysis of HEVC and AVC Video Codecs”, TCSVT, 2012
High Efficiency Video
Coding (HEVC)
2003
Advanced Video
Coding (AVC)
2013
Versatile Video
Coding (VVC)
2020
170 % 954 %
37 % 35 %
Block size 16x16
Quaternary tree
Supports up to 4K
Block size 64x64
Quaternary tree
Supports up to 8K
Block size 128x128
Multi-type tree
Supports up to
16K, 360° videos
7
All rights reserved. ©2020
Video Encoding with Machine Learning
Block
Partitioning
Motion
Compensation
Transformation
& Quantization
Entropy
Coding
Entropy
Decoding
Inverse
Transformation
& Inverse
Quantization
Inter or Intra
Prediction
Picture Buffer
In-loop
Filtering
Block
Partitioning
Decision
Prediction
Optical Flow
Detection
Mode
Prediction
Angular
Direction
Prediction
Deblocking
with ML
Denoising
with ML
Super-resolution
8
Research Questions
All rights reserved. ©2020
9
All rights reserved. ©2020
10
RQ-1 How to efficiently provide multi-rate
representations over a wide range of resolutions for HAS?
RQ-2 How to improve the performance of video codecs
using machine learning?
RQ-4 How to use machine learning to improve
perceptual quality assessment for videos?
RQ-3 How to improve the visual quality of videos using
machine learning?
Why?
🔋High-resolution content is getting more common,
required number of representations for HAS is increasing.
Literature
🗂 ML based approaches are utilized in video codecs to
speed up encoder decision.
🗂 Some attempts in end-to-end ML based video codecs.
Literature
🗂 ML based refinement techniques applied.
🗂 Post-processing in decoded frames to improve
quality.
🗂 Super-resolution for images and videos.
Literature
🗂 ML model is used in VMAF.
🗂 Several more attempts for non-reference perceptual
quality assessment.
Why?
🔋ML based image restoration methods are improving,
however video is mostly ignored. QoE can be increased.
Why?
🔋Finding a reliable metric for perceptual quality is
important as current objective metrics are problematic.
Why?
🔋More complex codecs, many possibilities to apply ML,
still much room for improvement.
Literature
🗂 Choose a reference representation and use its
information to speed up remaining encodings.
Methodology & Existing Results
All rights reserved. ©2020
11
All rights reserved. ©2020
Design and Abstraction Methodology
Design
Propose a solution (algorithm,
concept, protocol, etc.) for a
given problem
Implement
Prototype software
implementation using the
proposed solution
Analyze
Qualitative and quantitative analysis
of the solution
Repeat the cycle
to improve the
solutions
12
All rights reserved. ©2020
● State-of-the-art:
○ Encode the highest quality 1
or the lowest
quality 2
as the reference first then use these
information
● Proposed Method 3
:
○ Encode the highest quality first,
○ Use its information to encode the lowest
quality
○ Use information from both representations to
encode the remaining representations
○ Double bound for CTU search ranges
1
Schroeder, Damien, et al. "Efficient multi-rate video encoding for HEVC-based adaptive HTTP streaming." IEEE Transactions on Circuits and systems for
Video Technology 28.1 (2016): 143-157.
2
B. Guo, Y. Han, J. Wen, "Fast Block Structure Determination in AV1-based Multiple Resolutions Video Encoding," in 2018 IEEE International Conference on
Multimedia and Expo (ICME), San Diego, CA, USA, July 2018.
3
H. Amirpour, E. Çetinkaya, C. Timmerer and M. Ghanbari, "Fast Multi-rate Encoding for Adaptive HTTP Streaming," 2020 Data Compression Conference
(DCC), Snowbird, UT, USA, 2020, pp. 358-358
QP1
QPN
QPN-1
QP3
QP2
...
13
Fast Multi-rate Encoding (DCC’20)
All rights reserved. ©2020
14
Fast Multi-rate Encoding (DCC’20)
All rights reserved. ©2020
● State-of-the-art:
○ Encode the highest quality 1
or the lowest
quality 2
as the reference first then use these
information
● Proposed Method 3
:
○ Try different quality levels as the reference
representation to determine the best starting
point for parallel encoding
○ Encode the middle quality first and use its
information.
○ Upper or lower bound depending on the quality
level
Towards Optimal Multirate Encoding (MMM’21)
1
Schroeder, Damien, et al. "Efficient multi-rate video encoding for HEVC-based adaptive HTTP streaming." IEEE Transactions on Circuits and systems for
Video Technology 28.1 (2016): 143-157.
2
B. Guo, Y. Han, J. Wen, "Fast Block Structure Determination in AV1-based Multiple Resolutions Video Encoding," in 2018 IEEE International Conference on
Multimedia and Expo (ICME), San Diego, CA, USA, July 2018.
3
H. Amirpour, E. Çetinkaya, C. Timmerer and M. Ghanbari, "Towards Optimal Multirate Encoding for HTTP Adaptive Streaming," The International MultiMedia
Modeling Conference (MMM), Prague, Czech Republic, 2021
QPN/2
QPN
QP2
QP1
...
15
All rights reserved. ©2020
16
Towards Optimal Multirate Encoding (MMM’21)
All rights reserved. ©2020
● State-of-the-art:
○ Use a CNN to predict CTU depth decisions 1
● Proposed Method 2
:
○ Train a CNN with encoding information
obtained from the reference representation and
use its decision to encode dependent
representations.
○ Focus on parallel encoding, thus only apply for
bottleneck situations
○ Train different CNNs for different QP targets
1
Kim, Kyungah, and Won Woo Ro. "Fast CU depth decision for HEVC using neural networks." IEEE Transactions on Circuits and Systems for Video Technology
29.5 (2018): 1462-1473.
2
E. Çetinkaya, H. Amirpour, C. Timmerer and M. Ghanbari, “FaME-ML: Fast Multirate Encoding for HTTP Adaptive Streaming Using Machine Learning,” 2020
IEEE International Conference on Visual Communications and Image Processing (VCIP), Macau, 2020, pp. 87-90.
QPN
CNN
QPN-1
QP1
QP2
...
HEVC
HEVC
HEVC
CNN
HEVC HEVC
17
Fast Multi-rate Encoding with ML (VCIP’20)
All rights reserved. ©2020
18
Fast Multi-rate Encoding with ML (VCIP’20)
All rights reserved. ©2020
● State-of-the-art:
○ Use the highest quality representation as the
reference 1
● Proposed Method 2
:
○ Train a CNN with encoding information obtained
from the reference representation (the highest
quality from the lowest resolution) and use its
decision to encode dependent representations
○ Improves parallel encoding as well as serial
encoding
○ Train different CNNs for different QP and resolution
targets
1
Schroeder, Damien, et al. "Efficient multi-rate video encoding for HEVC-based adaptive HTTP streaming." IEEE Transactions on Circuits and systems for
Video Technology 28.1 (2016): 143-157.
2
E. Çetinkaya, H. Amirpour, C. Timmerer and M. Ghanbari, "Fast Multi-Resolution and Multi-Rate Encoding for HTTP Adaptive Streaming Using Machine
Learning," in IEEE Open Journal of Signal Processing, vol. 2, pp. 484-495, 2021, doi: 10.1109/OJSP.2021.3078657.
19
Fast Multi-rate and Multi-resolution
Encoding with ML (IEEE OJ-SP)
HEVC
QP1
HEVC
QP2
CNN
HEVC
QPN
CNN
HEVC
..
CNN
HEVC
QP2
CNN
HEVC
QPN
CNN
HEVC
..
CNN
HEVC
QP2
CNN
HEVC
QPN
CNN
HEVC
..
CNN
HEVC
QP1
CNN
HEVC
QP1
CNN
540p
540p
1080p
2160p
All rights reserved. ©2020
20
Fast Multi-rate and Multi-resolution Encoding with ML (IEEE OJ-SP)
Normalized
Encoding
time
HM 16.21 Lower Bound FaRes-ML
Ongoing & Future Work
All rights reserved. ©2020
21
All rights reserved. ©2020
Work Plan
2019
Q4
2020 2021 2022
Q1 Q2 Q3 Q4 Q1 Q2 Q3 Q4 Q1 Q2 Q3 Q4 Q1 Q2
1. How to efficiently provide multi-bitrate representations over a wide
range of resolutions for HAS ?
2. How to improve performance of video codecs using machine learning ?
3. How to improve quality of videos using machine learning ?
4. How to use machine learning to improve perceptual quality assessment
for videos ?
Literature review
DCC’20 Paper
MMM’21 Paper
VCIP’20 Paper
Multi-rate and Multi-resolution Encoding
IEEE OJSP Paper
RQ1
RQ2
RQ3
RQ4
Super-resolution
Literature
Review
Perceptual Quality Assessment with ML
2023
Thesis
Bitrate Ladder Prediction
Literature
Review
Improvement in In-loop Filtering
with ML
Mobile Player Optimization with SR
22
Fast multi-rate encoding for adaptive http streaming
Towards optimal multirate encoding for HTTP
adaptive streaming
FaME-ML: Fast multirate encoding for HTTP adaptive
streaming using machine learning
Fast Multi-Resolution and Multi-Rate Encoding for
HTTP Adaptive Streaming Using Machine Learning
All rights reserved. ©2020
Thank you!
ekrem.cetinkaya@aau.at @ekremcetinkaya_ linkedin.com/in/ekrcet

Weitere ähnliche Inhalte

Was ist angesagt?

CSDN: CDN-Aware QoE Optimization in SDN-Assisted HTTP Adaptive Video Streaming
CSDN: CDN-Aware QoE Optimization in SDN-Assisted HTTP Adaptive Video StreamingCSDN: CDN-Aware QoE Optimization in SDN-Assisted HTTP Adaptive Video Streaming
CSDN: CDN-Aware QoE Optimization in SDN-Assisted HTTP Adaptive Video StreamingAlpen-Adria-Universität
 
Bandwidth Prediction in Low-Latency Chunked Streaming
Bandwidth Prediction in Low-Latency Chunked StreamingBandwidth Prediction in Low-Latency Chunked Streaming
Bandwidth Prediction in Low-Latency Chunked StreamingAlpen-Adria-Universität
 
Press Release of 131st WG11 (MPEG) Meeting
Press Release of 131st WG11 (MPEG) MeetingPress Release of 131st WG11 (MPEG) Meeting
Press Release of 131st WG11 (MPEG) MeetingAlpen-Adria-Universität
 
Where to Encode: A Performance Analysis of Intel x86 and Arm-based Amazon EC2...
Where to Encode: A Performance Analysis of Intel x86 and Arm-based Amazon EC2...Where to Encode: A Performance Analysis of Intel x86 and Arm-based Amazon EC2...
Where to Encode: A Performance Analysis of Intel x86 and Arm-based Amazon EC2...Alpen-Adria-Universität
 
Video complexity analyzer (VCA) for streaming applications
 Video complexity analyzer (VCA) for streaming applications Video complexity analyzer (VCA) for streaming applications
Video complexity analyzer (VCA) for streaming applicationsAlpen-Adria-Universität
 
FaME-ML: Fast Multirate Encoding for HTTP Adaptive Streaming Using Machine Le...
FaME-ML: Fast Multirate Encoding for HTTP Adaptive Streaming Using Machine Le...FaME-ML: Fast Multirate Encoding for HTTP Adaptive Streaming Using Machine Le...
FaME-ML: Fast Multirate Encoding for HTTP Adaptive Streaming Using Machine Le...Alpen-Adria-Universität
 
EADAS: Edge Assisted Adaptation Scheme for HTTP Adaptive Streaming
EADAS: Edge Assisted Adaptation Scheme for HTTP Adaptive StreamingEADAS: Edge Assisted Adaptation Scheme for HTTP Adaptive Streaming
EADAS: Edge Assisted Adaptation Scheme for HTTP Adaptive StreamingAlpen-Adria-Universität
 
EPIQ'21: Days of Future Past: An Optimization-based Adaptive Bitrate Algorith...
EPIQ'21: Days of Future Past: An Optimization-based Adaptive Bitrate Algorith...EPIQ'21: Days of Future Past: An Optimization-based Adaptive Bitrate Algorith...
EPIQ'21: Days of Future Past: An Optimization-based Adaptive Bitrate Algorith...Minh Nguyen
 
Video Coding Enhancements for HTTP Adaptive Streaming
Video Coding Enhancements for HTTP Adaptive StreamingVideo Coding Enhancements for HTTP Adaptive Streaming
Video Coding Enhancements for HTTP Adaptive StreamingAlpen-Adria-Universität
 
Video Coding for Large-Scale HTTP Adaptive Streaming Deployments: State of th...
Video Coding for Large-Scale HTTP Adaptive Streaming Deployments: State of th...Video Coding for Large-Scale HTTP Adaptive Streaming Deployments: State of th...
Video Coding for Large-Scale HTTP Adaptive Streaming Deployments: State of th...Alpen-Adria-Universität
 
Docker-Based Evaluation Framework for Video Streaming QoE in Broadband Networks
 Docker-Based Evaluation Framework for Video Streaming QoE in Broadband Networks Docker-Based Evaluation Framework for Video Streaming QoE in Broadband Networks
Docker-Based Evaluation Framework for Video Streaming QoE in Broadband NetworksAlpen-Adria-Universität
 
Quality Optimization of Live Streaming Services over HTTP with Reinforcement ...
Quality Optimization of Live Streaming Services over HTTP with Reinforcement ...Quality Optimization of Live Streaming Services over HTTP with Reinforcement ...
Quality Optimization of Live Streaming Services over HTTP with Reinforcement ...Alpen-Adria-Universität
 
Policy-driven Dynamic HTTP Adaptive Streaming Player Environment
Policy-driven Dynamic HTTP Adaptive Streaming Player EnvironmentPolicy-driven Dynamic HTTP Adaptive Streaming Player Environment
Policy-driven Dynamic HTTP Adaptive Streaming Player EnvironmentMinh Nguyen
 
A Channel Allocation Algorithm for Cognitive Radio Users Based on Channel Sta...
A Channel Allocation Algorithm for Cognitive Radio Users Based on Channel Sta...A Channel Allocation Algorithm for Cognitive Radio Users Based on Channel Sta...
A Channel Allocation Algorithm for Cognitive Radio Users Based on Channel Sta...Alpen-Adria-Universität
 
INCEPT: Intra CU Depth Prediction for HEVC
INCEPT: Intra CU Depth Prediction for HEVCINCEPT: Intra CU Depth Prediction for HEVC
INCEPT: Intra CU Depth Prediction for HEVCAlpen-Adria-Universität
 
ES-HAS: An Edge- and SDN-Assisted Framework for HTTP Adaptive Video Streaming
ES-HAS: An Edge- and SDN-Assisted Framework for HTTP Adaptive Video StreamingES-HAS: An Edge- and SDN-Assisted Framework for HTTP Adaptive Video Streaming
ES-HAS: An Edge- and SDN-Assisted Framework for HTTP Adaptive Video StreamingAlpen-Adria-Universität
 
ComplexCTTP: Complexity Class Based Transcoding Time Prediction for Video Seq...
ComplexCTTP: Complexity Class Based Transcoding Time Prediction for Video Seq...ComplexCTTP: Complexity Class Based Transcoding Time Prediction for Video Seq...
ComplexCTTP: Complexity Class Based Transcoding Time Prediction for Video Seq...Alpen-Adria-Universität
 

Was ist angesagt? (20)

CSDN: CDN-Aware QoE Optimization in SDN-Assisted HTTP Adaptive Video Streaming
CSDN: CDN-Aware QoE Optimization in SDN-Assisted HTTP Adaptive Video StreamingCSDN: CDN-Aware QoE Optimization in SDN-Assisted HTTP Adaptive Video Streaming
CSDN: CDN-Aware QoE Optimization in SDN-Assisted HTTP Adaptive Video Streaming
 
Bandwidth Prediction in Low-Latency Chunked Streaming
Bandwidth Prediction in Low-Latency Chunked StreamingBandwidth Prediction in Low-Latency Chunked Streaming
Bandwidth Prediction in Low-Latency Chunked Streaming
 
Press Release of 131st WG11 (MPEG) Meeting
Press Release of 131st WG11 (MPEG) MeetingPress Release of 131st WG11 (MPEG) Meeting
Press Release of 131st WG11 (MPEG) Meeting
 
Where to Encode: A Performance Analysis of Intel x86 and Arm-based Amazon EC2...
Where to Encode: A Performance Analysis of Intel x86 and Arm-based Amazon EC2...Where to Encode: A Performance Analysis of Intel x86 and Arm-based Amazon EC2...
Where to Encode: A Performance Analysis of Intel x86 and Arm-based Amazon EC2...
 
Video complexity analyzer (VCA) for streaming applications
 Video complexity analyzer (VCA) for streaming applications Video complexity analyzer (VCA) for streaming applications
Video complexity analyzer (VCA) for streaming applications
 
FaME-ML: Fast Multirate Encoding for HTTP Adaptive Streaming Using Machine Le...
FaME-ML: Fast Multirate Encoding for HTTP Adaptive Streaming Using Machine Le...FaME-ML: Fast Multirate Encoding for HTTP Adaptive Streaming Using Machine Le...
FaME-ML: Fast Multirate Encoding for HTTP Adaptive Streaming Using Machine Le...
 
PEMWN'21 - ANGELA
PEMWN'21 - ANGELAPEMWN'21 - ANGELA
PEMWN'21 - ANGELA
 
EADAS: Edge Assisted Adaptation Scheme for HTTP Adaptive Streaming
EADAS: Edge Assisted Adaptation Scheme for HTTP Adaptive StreamingEADAS: Edge Assisted Adaptation Scheme for HTTP Adaptive Streaming
EADAS: Edge Assisted Adaptation Scheme for HTTP Adaptive Streaming
 
EPIQ'21: Days of Future Past: An Optimization-based Adaptive Bitrate Algorith...
EPIQ'21: Days of Future Past: An Optimization-based Adaptive Bitrate Algorith...EPIQ'21: Days of Future Past: An Optimization-based Adaptive Bitrate Algorith...
EPIQ'21: Days of Future Past: An Optimization-based Adaptive Bitrate Algorith...
 
Video Coding Enhancements for HTTP Adaptive Streaming
Video Coding Enhancements for HTTP Adaptive StreamingVideo Coding Enhancements for HTTP Adaptive Streaming
Video Coding Enhancements for HTTP Adaptive Streaming
 
Video Coding for Large-Scale HTTP Adaptive Streaming Deployments: State of th...
Video Coding for Large-Scale HTTP Adaptive Streaming Deployments: State of th...Video Coding for Large-Scale HTTP Adaptive Streaming Deployments: State of th...
Video Coding for Large-Scale HTTP Adaptive Streaming Deployments: State of th...
 
SLFC: Scalable Light Field Coding
SLFC: Scalable Light Field CodingSLFC: Scalable Light Field Coding
SLFC: Scalable Light Field Coding
 
Docker-Based Evaluation Framework for Video Streaming QoE in Broadband Networks
 Docker-Based Evaluation Framework for Video Streaming QoE in Broadband Networks Docker-Based Evaluation Framework for Video Streaming QoE in Broadband Networks
Docker-Based Evaluation Framework for Video Streaming QoE in Broadband Networks
 
What’s new in MPEG?
What’s new in MPEG?What’s new in MPEG?
What’s new in MPEG?
 
Quality Optimization of Live Streaming Services over HTTP with Reinforcement ...
Quality Optimization of Live Streaming Services over HTTP with Reinforcement ...Quality Optimization of Live Streaming Services over HTTP with Reinforcement ...
Quality Optimization of Live Streaming Services over HTTP with Reinforcement ...
 
Policy-driven Dynamic HTTP Adaptive Streaming Player Environment
Policy-driven Dynamic HTTP Adaptive Streaming Player EnvironmentPolicy-driven Dynamic HTTP Adaptive Streaming Player Environment
Policy-driven Dynamic HTTP Adaptive Streaming Player Environment
 
A Channel Allocation Algorithm for Cognitive Radio Users Based on Channel Sta...
A Channel Allocation Algorithm for Cognitive Radio Users Based on Channel Sta...A Channel Allocation Algorithm for Cognitive Radio Users Based on Channel Sta...
A Channel Allocation Algorithm for Cognitive Radio Users Based on Channel Sta...
 
INCEPT: Intra CU Depth Prediction for HEVC
INCEPT: Intra CU Depth Prediction for HEVCINCEPT: Intra CU Depth Prediction for HEVC
INCEPT: Intra CU Depth Prediction for HEVC
 
ES-HAS: An Edge- and SDN-Assisted Framework for HTTP Adaptive Video Streaming
ES-HAS: An Edge- and SDN-Assisted Framework for HTTP Adaptive Video StreamingES-HAS: An Edge- and SDN-Assisted Framework for HTTP Adaptive Video Streaming
ES-HAS: An Edge- and SDN-Assisted Framework for HTTP Adaptive Video Streaming
 
ComplexCTTP: Complexity Class Based Transcoding Time Prediction for Video Seq...
ComplexCTTP: Complexity Class Based Transcoding Time Prediction for Video Seq...ComplexCTTP: Complexity Class Based Transcoding Time Prediction for Video Seq...
ComplexCTTP: Complexity Class Based Transcoding Time Prediction for Video Seq...
 

Ähnlich wie Machine Learning Based Video Coding Enhancements for HTTP Adaptive Streaming

QoE- and Energy-aware Content Consumption for HTTP Adaptive Streaming
QoE- and Energy-aware Content Consumption for HTTP Adaptive StreamingQoE- and Energy-aware Content Consumption for HTTP Adaptive Streaming
QoE- and Energy-aware Content Consumption for HTTP Adaptive StreamingDanieleLorenzi6
 
Research@Lunch_Presentation.pdf
Research@Lunch_Presentation.pdfResearch@Lunch_Presentation.pdf
Research@Lunch_Presentation.pdfVignesh V Menon
 
Video Coding Enhancements for HTTP Adaptive Streaming Using Machine Learning
Video Coding Enhancements for HTTP Adaptive Streaming Using Machine LearningVideo Coding Enhancements for HTTP Adaptive Streaming Using Machine Learning
Video Coding Enhancements for HTTP Adaptive Streaming Using Machine LearningEkrem Çetinkaya
 
Video Coding Enhancements for HTTP Adaptive Streaming Using Machine Learning
Video Coding Enhancements for HTTP Adaptive Streaming Using Machine LearningVideo Coding Enhancements for HTTP Adaptive Streaming Using Machine Learning
Video Coding Enhancements for HTTP Adaptive Streaming Using Machine LearningAlpen-Adria-Universität
 
HTTP Adaptive Streaming – Quo Vadis? (2023)
HTTP Adaptive Streaming – Quo Vadis? (2023)HTTP Adaptive Streaming – Quo Vadis? (2023)
HTTP Adaptive Streaming – Quo Vadis? (2023)Alpen-Adria-Universität
 
MHV'22 - Super-resolution Based Bitrate Adaptation for HTTP Adaptive Streamin...
MHV'22 - Super-resolution Based Bitrate Adaptation for HTTP Adaptive Streamin...MHV'22 - Super-resolution Based Bitrate Adaptation for HTTP Adaptive Streamin...
MHV'22 - Super-resolution Based Bitrate Adaptation for HTTP Adaptive Streamin...Minh Nguyen
 
OPTE: Online Per-title Encoding for Live Video Streaming.pdf
OPTE: Online Per-title Encoding for Live Video Streaming.pdfOPTE: Online Per-title Encoding for Live Video Streaming.pdf
OPTE: Online Per-title Encoding for Live Video Streaming.pdfVignesh V Menon
 
Energy-Efficient Multi-Codec Bitrate-Ladder Estimation for Adaptive Video Str...
Energy-Efficient Multi-Codec Bitrate-Ladder Estimation for Adaptive Video Str...Energy-Efficient Multi-Codec Bitrate-Ladder Estimation for Adaptive Video Str...
Energy-Efficient Multi-Codec Bitrate-Ladder Estimation for Adaptive Video Str...Alpen-Adria-Universität
 
Immersive Video Delivery: From Omnidirectional Video to Holography
Immersive Video Delivery: From Omnidirectional Video to HolographyImmersive Video Delivery: From Omnidirectional Video to Holography
Immersive Video Delivery: From Omnidirectional Video to HolographyAlpen-Adria-Universität
 
Content-adaptive Video Coding for HTTP Adaptive Streaming
Content-adaptive Video Coding for HTTP Adaptive StreamingContent-adaptive Video Coding for HTTP Adaptive Streaming
Content-adaptive Video Coding for HTTP Adaptive StreamingAlpen-Adria-Universität
 
Motion Vector Recovery for Real-time H.264 Video Streams
Motion Vector Recovery for Real-time H.264 Video StreamsMotion Vector Recovery for Real-time H.264 Video Streams
Motion Vector Recovery for Real-time H.264 Video StreamsIDES Editor
 
The impact of jitter on the HEVC video streaming with Multiple Coding
The impact of jitter on the HEVC video streaming with  Multiple CodingThe impact of jitter on the HEVC video streaming with  Multiple Coding
The impact of jitter on the HEVC video streaming with Multiple CodingHakimSahour
 
Paper id 2120148
Paper id 2120148Paper id 2120148
Paper id 2120148IJRAT
 
MMSys'21 - Multi-access edge computing for adaptive bitrate video streaming
MMSys'21 - Multi-access edge computing for adaptive bitrate video streamingMMSys'21 - Multi-access edge computing for adaptive bitrate video streaming
MMSys'21 - Multi-access edge computing for adaptive bitrate video streamingJesus Aguilar
 
IEEE MMSP'21: INCEPT: Intra CU Depth Prediction for HEVC
IEEE MMSP'21: INCEPT: Intra CU Depth Prediction for HEVCIEEE MMSP'21: INCEPT: Intra CU Depth Prediction for HEVC
IEEE MMSP'21: INCEPT: Intra CU Depth Prediction for HEVCVignesh V Menon
 
Optimal coding unit decision for early termination in high efficiency video c...
Optimal coding unit decision for early termination in high efficiency video c...Optimal coding unit decision for early termination in high efficiency video c...
Optimal coding unit decision for early termination in high efficiency video c...IJECEIAES
 
LwTE-Live: Light-weight Transcoding at the Edge for Live Streaming
LwTE-Live: Light-weight Transcoding at the Edge for Live StreamingLwTE-Live: Light-weight Transcoding at the Edge for Live Streaming
LwTE-Live: Light-weight Transcoding at the Edge for Live StreamingAlpen-Adria-Universität
 
An Overview of High Efficiency Video Codec HEVC (H.265)
An Overview of High Efficiency Video Codec HEVC (H.265)An Overview of High Efficiency Video Codec HEVC (H.265)
An Overview of High Efficiency Video Codec HEVC (H.265)Varun Ravi
 
Are you Digitized Files Really OK? Levels of QC and Film Digitization (SCHALL...
Are you Digitized Files Really OK? Levels of QC and Film Digitization (SCHALL...Are you Digitized Files Really OK? Levels of QC and Film Digitization (SCHALL...
Are you Digitized Files Really OK? Levels of QC and Film Digitization (SCHALL...FIAT/IFTA
 

Ähnlich wie Machine Learning Based Video Coding Enhancements for HTTP Adaptive Streaming (20)

HTTP Adaptive Streaming – Quo Vadis?
HTTP Adaptive Streaming – Quo Vadis?HTTP Adaptive Streaming – Quo Vadis?
HTTP Adaptive Streaming – Quo Vadis?
 
QoE- and Energy-aware Content Consumption for HTTP Adaptive Streaming
QoE- and Energy-aware Content Consumption for HTTP Adaptive StreamingQoE- and Energy-aware Content Consumption for HTTP Adaptive Streaming
QoE- and Energy-aware Content Consumption for HTTP Adaptive Streaming
 
Research@Lunch_Presentation.pdf
Research@Lunch_Presentation.pdfResearch@Lunch_Presentation.pdf
Research@Lunch_Presentation.pdf
 
Video Coding Enhancements for HTTP Adaptive Streaming Using Machine Learning
Video Coding Enhancements for HTTP Adaptive Streaming Using Machine LearningVideo Coding Enhancements for HTTP Adaptive Streaming Using Machine Learning
Video Coding Enhancements for HTTP Adaptive Streaming Using Machine Learning
 
Video Coding Enhancements for HTTP Adaptive Streaming Using Machine Learning
Video Coding Enhancements for HTTP Adaptive Streaming Using Machine LearningVideo Coding Enhancements for HTTP Adaptive Streaming Using Machine Learning
Video Coding Enhancements for HTTP Adaptive Streaming Using Machine Learning
 
HTTP Adaptive Streaming – Quo Vadis? (2023)
HTTP Adaptive Streaming – Quo Vadis? (2023)HTTP Adaptive Streaming – Quo Vadis? (2023)
HTTP Adaptive Streaming – Quo Vadis? (2023)
 
MHV'22 - Super-resolution Based Bitrate Adaptation for HTTP Adaptive Streamin...
MHV'22 - Super-resolution Based Bitrate Adaptation for HTTP Adaptive Streamin...MHV'22 - Super-resolution Based Bitrate Adaptation for HTTP Adaptive Streamin...
MHV'22 - Super-resolution Based Bitrate Adaptation for HTTP Adaptive Streamin...
 
OPTE: Online Per-title Encoding for Live Video Streaming.pdf
OPTE: Online Per-title Encoding for Live Video Streaming.pdfOPTE: Online Per-title Encoding for Live Video Streaming.pdf
OPTE: Online Per-title Encoding for Live Video Streaming.pdf
 
Energy-Efficient Multi-Codec Bitrate-Ladder Estimation for Adaptive Video Str...
Energy-Efficient Multi-Codec Bitrate-Ladder Estimation for Adaptive Video Str...Energy-Efficient Multi-Codec Bitrate-Ladder Estimation for Adaptive Video Str...
Energy-Efficient Multi-Codec Bitrate-Ladder Estimation for Adaptive Video Str...
 
Immersive Video Delivery: From Omnidirectional Video to Holography
Immersive Video Delivery: From Omnidirectional Video to HolographyImmersive Video Delivery: From Omnidirectional Video to Holography
Immersive Video Delivery: From Omnidirectional Video to Holography
 
Content-adaptive Video Coding for HTTP Adaptive Streaming
Content-adaptive Video Coding for HTTP Adaptive StreamingContent-adaptive Video Coding for HTTP Adaptive Streaming
Content-adaptive Video Coding for HTTP Adaptive Streaming
 
Motion Vector Recovery for Real-time H.264 Video Streams
Motion Vector Recovery for Real-time H.264 Video StreamsMotion Vector Recovery for Real-time H.264 Video Streams
Motion Vector Recovery for Real-time H.264 Video Streams
 
The impact of jitter on the HEVC video streaming with Multiple Coding
The impact of jitter on the HEVC video streaming with  Multiple CodingThe impact of jitter on the HEVC video streaming with  Multiple Coding
The impact of jitter on the HEVC video streaming with Multiple Coding
 
Paper id 2120148
Paper id 2120148Paper id 2120148
Paper id 2120148
 
MMSys'21 - Multi-access edge computing for adaptive bitrate video streaming
MMSys'21 - Multi-access edge computing for adaptive bitrate video streamingMMSys'21 - Multi-access edge computing for adaptive bitrate video streaming
MMSys'21 - Multi-access edge computing for adaptive bitrate video streaming
 
IEEE MMSP'21: INCEPT: Intra CU Depth Prediction for HEVC
IEEE MMSP'21: INCEPT: Intra CU Depth Prediction for HEVCIEEE MMSP'21: INCEPT: Intra CU Depth Prediction for HEVC
IEEE MMSP'21: INCEPT: Intra CU Depth Prediction for HEVC
 
Optimal coding unit decision for early termination in high efficiency video c...
Optimal coding unit decision for early termination in high efficiency video c...Optimal coding unit decision for early termination in high efficiency video c...
Optimal coding unit decision for early termination in high efficiency video c...
 
LwTE-Live: Light-weight Transcoding at the Edge for Live Streaming
LwTE-Live: Light-weight Transcoding at the Edge for Live StreamingLwTE-Live: Light-weight Transcoding at the Edge for Live Streaming
LwTE-Live: Light-weight Transcoding at the Edge for Live Streaming
 
An Overview of High Efficiency Video Codec HEVC (H.265)
An Overview of High Efficiency Video Codec HEVC (H.265)An Overview of High Efficiency Video Codec HEVC (H.265)
An Overview of High Efficiency Video Codec HEVC (H.265)
 
Are you Digitized Files Really OK? Levels of QC and Film Digitization (SCHALL...
Are you Digitized Files Really OK? Levels of QC and Film Digitization (SCHALL...Are you Digitized Files Really OK? Levels of QC and Film Digitization (SCHALL...
Are you Digitized Files Really OK? Levels of QC and Film Digitization (SCHALL...
 

Mehr von Alpen-Adria-Universität

Empowerment of Atypical Viewers via Low-Effort Personalized Modeling of Video...
Empowerment of Atypical Viewers via Low-Effort Personalized Modeling of Video...Empowerment of Atypical Viewers via Low-Effort Personalized Modeling of Video...
Empowerment of Atypical Viewers via Low-Effort Personalized Modeling of Video...Alpen-Adria-Universität
 
Empowerment of Atypical Viewers via Low-Effort Personalized Modeling of Vid...
Empowerment of Atypical Viewers  via Low-Effort Personalized Modeling  of Vid...Empowerment of Atypical Viewers  via Low-Effort Personalized Modeling  of Vid...
Empowerment of Atypical Viewers via Low-Effort Personalized Modeling of Vid...Alpen-Adria-Universität
 
Optimizing Video Streaming for Sustainability and Quality: The Role of Prese...
Optimizing Video Streaming  for Sustainability and Quality: The Role of Prese...Optimizing Video Streaming  for Sustainability and Quality: The Role of Prese...
Optimizing Video Streaming for Sustainability and Quality: The Role of Prese...Alpen-Adria-Universität
 
Machine Learning Based Resource Utilization Prediction in the Computing Conti...
Machine Learning Based Resource Utilization Prediction in the Computing Conti...Machine Learning Based Resource Utilization Prediction in the Computing Conti...
Machine Learning Based Resource Utilization Prediction in the Computing Conti...Alpen-Adria-Universität
 
Evaluation of Quality of Experience of ABR Schemes in Gaming Stream
Evaluation of Quality of Experience of ABR Schemes in Gaming StreamEvaluation of Quality of Experience of ABR Schemes in Gaming Stream
Evaluation of Quality of Experience of ABR Schemes in Gaming StreamAlpen-Adria-Universität
 
Network-Assisted Delivery of Adaptive Video Streaming Services through CDN, S...
Network-Assisted Delivery of Adaptive Video Streaming Services through CDN, S...Network-Assisted Delivery of Adaptive Video Streaming Services through CDN, S...
Network-Assisted Delivery of Adaptive Video Streaming Services through CDN, S...Alpen-Adria-Universität
 
Multi-access Edge Computing for Adaptive Video Streaming
Multi-access Edge Computing for Adaptive Video StreamingMulti-access Edge Computing for Adaptive Video Streaming
Multi-access Edge Computing for Adaptive Video StreamingAlpen-Adria-Universität
 
Policy-Driven Dynamic HTTP Adaptive Streaming Player Environment
Policy-Driven Dynamic HTTP Adaptive Streaming Player EnvironmentPolicy-Driven Dynamic HTTP Adaptive Streaming Player Environment
Policy-Driven Dynamic HTTP Adaptive Streaming Player EnvironmentAlpen-Adria-Universität
 
VE-Match: Video Encoding Matching-based Model for Cloud and Edge Computing In...
VE-Match: Video Encoding Matching-based Model for Cloud and Edge Computing In...VE-Match: Video Encoding Matching-based Model for Cloud and Edge Computing In...
VE-Match: Video Encoding Matching-based Model for Cloud and Edge Computing In...Alpen-Adria-Universität
 
Energy Consumption in Video Streaming: Components, Measurements, and Strategies
Energy Consumption in Video Streaming: Components, Measurements, and StrategiesEnergy Consumption in Video Streaming: Components, Measurements, and Strategies
Energy Consumption in Video Streaming: Components, Measurements, and StrategiesAlpen-Adria-Universität
 
Exploring the Energy Consumption of Video Streaming: Components, Challenges, ...
Exploring the Energy Consumption of Video Streaming: Components, Challenges, ...Exploring the Energy Consumption of Video Streaming: Components, Challenges, ...
Exploring the Energy Consumption of Video Streaming: Components, Challenges, ...Alpen-Adria-Universität
 
SARENA: SFC-Enabled Architecture for Adaptive Video Streaming Applications
SARENA: SFC-Enabled Architecture for Adaptive Video Streaming ApplicationsSARENA: SFC-Enabled Architecture for Adaptive Video Streaming Applications
SARENA: SFC-Enabled Architecture for Adaptive Video Streaming ApplicationsAlpen-Adria-Universität
 
LLL-CAdViSE: Live Low-Latency Cloud-based Adaptive Video Streaming Evaluation...
LLL-CAdViSE: Live Low-Latency Cloud-based Adaptive Video Streaming Evaluation...LLL-CAdViSE: Live Low-Latency Cloud-based Adaptive Video Streaming Evaluation...
LLL-CAdViSE: Live Low-Latency Cloud-based Adaptive Video Streaming Evaluation...Alpen-Adria-Universität
 
How to Optimize Dynamic Adaptive Video Streaming? Challenges and Solutions
How to Optimize Dynamic Adaptive Video Streaming? Challenges and SolutionsHow to Optimize Dynamic Adaptive Video Streaming? Challenges and Solutions
How to Optimize Dynamic Adaptive Video Streaming? Challenges and SolutionsAlpen-Adria-Universität
 
MPEC2: Multilayer and Pipeline Video Encoding on the Computing Continuum
MPEC2: Multilayer and Pipeline Video Encoding on the Computing ContinuumMPEC2: Multilayer and Pipeline Video Encoding on the Computing Continuum
MPEC2: Multilayer and Pipeline Video Encoding on the Computing ContinuumAlpen-Adria-Universität
 
Collaborative Edge-Assisted Systems for HTTP Adaptive Video Streaming
Collaborative Edge-Assisted Systems for HTTP Adaptive Video StreamingCollaborative Edge-Assisted Systems for HTTP Adaptive Video Streaming
Collaborative Edge-Assisted Systems for HTTP Adaptive Video StreamingAlpen-Adria-Universität
 
OTEC: An Optimized Transcoding Task Scheduler for Cloud and Fog Environments
OTEC: An Optimized Transcoding Task Scheduler for Cloud and Fog EnvironmentsOTEC: An Optimized Transcoding Task Scheduler for Cloud and Fog Environments
OTEC: An Optimized Transcoding Task Scheduler for Cloud and Fog EnvironmentsAlpen-Adria-Universität
 
ETPS: Efficient Two-pass Encoding Scheme for Adaptive Live Streaming
ETPS: Efficient Two-pass Encoding Scheme for Adaptive Live StreamingETPS: Efficient Two-pass Encoding Scheme for Adaptive Live Streaming
ETPS: Efficient Two-pass Encoding Scheme for Adaptive Live StreamingAlpen-Adria-Universität
 
OPSE: Online Per-Scene Encoding for Adaptive HTTP Live Streaming
OPSE: Online Per-Scene Encoding for Adaptive HTTP Live StreamingOPSE: Online Per-Scene Encoding for Adaptive HTTP Live Streaming
OPSE: Online Per-Scene Encoding for Adaptive HTTP Live StreamingAlpen-Adria-Universität
 

Mehr von Alpen-Adria-Universität (20)

Empowerment of Atypical Viewers via Low-Effort Personalized Modeling of Video...
Empowerment of Atypical Viewers via Low-Effort Personalized Modeling of Video...Empowerment of Atypical Viewers via Low-Effort Personalized Modeling of Video...
Empowerment of Atypical Viewers via Low-Effort Personalized Modeling of Video...
 
Empowerment of Atypical Viewers via Low-Effort Personalized Modeling of Vid...
Empowerment of Atypical Viewers  via Low-Effort Personalized Modeling  of Vid...Empowerment of Atypical Viewers  via Low-Effort Personalized Modeling  of Vid...
Empowerment of Atypical Viewers via Low-Effort Personalized Modeling of Vid...
 
Optimizing Video Streaming for Sustainability and Quality: The Role of Prese...
Optimizing Video Streaming  for Sustainability and Quality: The Role of Prese...Optimizing Video Streaming  for Sustainability and Quality: The Role of Prese...
Optimizing Video Streaming for Sustainability and Quality: The Role of Prese...
 
Machine Learning Based Resource Utilization Prediction in the Computing Conti...
Machine Learning Based Resource Utilization Prediction in the Computing Conti...Machine Learning Based Resource Utilization Prediction in the Computing Conti...
Machine Learning Based Resource Utilization Prediction in the Computing Conti...
 
Evaluation of Quality of Experience of ABR Schemes in Gaming Stream
Evaluation of Quality of Experience of ABR Schemes in Gaming StreamEvaluation of Quality of Experience of ABR Schemes in Gaming Stream
Evaluation of Quality of Experience of ABR Schemes in Gaming Stream
 
Network-Assisted Delivery of Adaptive Video Streaming Services through CDN, S...
Network-Assisted Delivery of Adaptive Video Streaming Services through CDN, S...Network-Assisted Delivery of Adaptive Video Streaming Services through CDN, S...
Network-Assisted Delivery of Adaptive Video Streaming Services through CDN, S...
 
Multi-access Edge Computing for Adaptive Video Streaming
Multi-access Edge Computing for Adaptive Video StreamingMulti-access Edge Computing for Adaptive Video Streaming
Multi-access Edge Computing for Adaptive Video Streaming
 
Policy-Driven Dynamic HTTP Adaptive Streaming Player Environment
Policy-Driven Dynamic HTTP Adaptive Streaming Player EnvironmentPolicy-Driven Dynamic HTTP Adaptive Streaming Player Environment
Policy-Driven Dynamic HTTP Adaptive Streaming Player Environment
 
VE-Match: Video Encoding Matching-based Model for Cloud and Edge Computing In...
VE-Match: Video Encoding Matching-based Model for Cloud and Edge Computing In...VE-Match: Video Encoding Matching-based Model for Cloud and Edge Computing In...
VE-Match: Video Encoding Matching-based Model for Cloud and Edge Computing In...
 
Energy Consumption in Video Streaming: Components, Measurements, and Strategies
Energy Consumption in Video Streaming: Components, Measurements, and StrategiesEnergy Consumption in Video Streaming: Components, Measurements, and Strategies
Energy Consumption in Video Streaming: Components, Measurements, and Strategies
 
Exploring the Energy Consumption of Video Streaming: Components, Challenges, ...
Exploring the Energy Consumption of Video Streaming: Components, Challenges, ...Exploring the Energy Consumption of Video Streaming: Components, Challenges, ...
Exploring the Energy Consumption of Video Streaming: Components, Challenges, ...
 
SARENA: SFC-Enabled Architecture for Adaptive Video Streaming Applications
SARENA: SFC-Enabled Architecture for Adaptive Video Streaming ApplicationsSARENA: SFC-Enabled Architecture for Adaptive Video Streaming Applications
SARENA: SFC-Enabled Architecture for Adaptive Video Streaming Applications
 
LLL-CAdViSE: Live Low-Latency Cloud-based Adaptive Video Streaming Evaluation...
LLL-CAdViSE: Live Low-Latency Cloud-based Adaptive Video Streaming Evaluation...LLL-CAdViSE: Live Low-Latency Cloud-based Adaptive Video Streaming Evaluation...
LLL-CAdViSE: Live Low-Latency Cloud-based Adaptive Video Streaming Evaluation...
 
How to Optimize Dynamic Adaptive Video Streaming? Challenges and Solutions
How to Optimize Dynamic Adaptive Video Streaming? Challenges and SolutionsHow to Optimize Dynamic Adaptive Video Streaming? Challenges and Solutions
How to Optimize Dynamic Adaptive Video Streaming? Challenges and Solutions
 
MPEC2: Multilayer and Pipeline Video Encoding on the Computing Continuum
MPEC2: Multilayer and Pipeline Video Encoding on the Computing ContinuumMPEC2: Multilayer and Pipeline Video Encoding on the Computing Continuum
MPEC2: Multilayer and Pipeline Video Encoding on the Computing Continuum
 
Collaborative Edge-Assisted Systems for HTTP Adaptive Video Streaming
Collaborative Edge-Assisted Systems for HTTP Adaptive Video StreamingCollaborative Edge-Assisted Systems for HTTP Adaptive Video Streaming
Collaborative Edge-Assisted Systems for HTTP Adaptive Video Streaming
 
OTEC: An Optimized Transcoding Task Scheduler for Cloud and Fog Environments
OTEC: An Optimized Transcoding Task Scheduler for Cloud and Fog EnvironmentsOTEC: An Optimized Transcoding Task Scheduler for Cloud and Fog Environments
OTEC: An Optimized Transcoding Task Scheduler for Cloud and Fog Environments
 
ETPS: Efficient Two-pass Encoding Scheme for Adaptive Live Streaming
ETPS: Efficient Two-pass Encoding Scheme for Adaptive Live StreamingETPS: Efficient Two-pass Encoding Scheme for Adaptive Live Streaming
ETPS: Efficient Two-pass Encoding Scheme for Adaptive Live Streaming
 
An Introduction to OMNeT++ 6.0
An Introduction to OMNeT++ 6.0An Introduction to OMNeT++ 6.0
An Introduction to OMNeT++ 6.0
 
OPSE: Online Per-Scene Encoding for Adaptive HTTP Live Streaming
OPSE: Online Per-Scene Encoding for Adaptive HTTP Live StreamingOPSE: Online Per-Scene Encoding for Adaptive HTTP Live Streaming
OPSE: Online Per-Scene Encoding for Adaptive HTTP Live Streaming
 

Kürzlich hochgeladen

Empowering Africa's Next Generation: The AI Leadership Blueprint
Empowering Africa's Next Generation: The AI Leadership BlueprintEmpowering Africa's Next Generation: The AI Leadership Blueprint
Empowering Africa's Next Generation: The AI Leadership BlueprintMahmoud Rabie
 
UiPath Studio Web workshop series - Day 7
UiPath Studio Web workshop series - Day 7UiPath Studio Web workshop series - Day 7
UiPath Studio Web workshop series - Day 7DianaGray10
 
OpenShift Commons Paris - Choose Your Own Observability Adventure
OpenShift Commons Paris - Choose Your Own Observability AdventureOpenShift Commons Paris - Choose Your Own Observability Adventure
OpenShift Commons Paris - Choose Your Own Observability AdventureEric D. Schabell
 
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...DianaGray10
 
COMPUTER 10: Lesson 7 - File Storage and Online Collaboration
COMPUTER 10: Lesson 7 - File Storage and Online CollaborationCOMPUTER 10: Lesson 7 - File Storage and Online Collaboration
COMPUTER 10: Lesson 7 - File Storage and Online Collaborationbruanjhuli
 
Nanopower In Semiconductor Industry.pdf
Nanopower  In Semiconductor Industry.pdfNanopower  In Semiconductor Industry.pdf
Nanopower In Semiconductor Industry.pdfPedro Manuel
 
Bird eye's view on Camunda open source ecosystem
Bird eye's view on Camunda open source ecosystemBird eye's view on Camunda open source ecosystem
Bird eye's view on Camunda open source ecosystemAsko Soukka
 
The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...
The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...
The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...Aggregage
 
Comparing Sidecar-less Service Mesh from Cilium and Istio
Comparing Sidecar-less Service Mesh from Cilium and IstioComparing Sidecar-less Service Mesh from Cilium and Istio
Comparing Sidecar-less Service Mesh from Cilium and IstioChristian Posta
 
Linked Data in Production: Moving Beyond Ontologies
Linked Data in Production: Moving Beyond OntologiesLinked Data in Production: Moving Beyond Ontologies
Linked Data in Production: Moving Beyond OntologiesDavid Newbury
 
activity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdf
activity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdf
activity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdfJamie (Taka) Wang
 
UiPath Platform: The Backend Engine Powering Your Automation - Session 1
UiPath Platform: The Backend Engine Powering Your Automation - Session 1UiPath Platform: The Backend Engine Powering Your Automation - Session 1
UiPath Platform: The Backend Engine Powering Your Automation - Session 1DianaGray10
 
Computer 10: Lesson 10 - Online Crimes and Hazards
Computer 10: Lesson 10 - Online Crimes and HazardsComputer 10: Lesson 10 - Online Crimes and Hazards
Computer 10: Lesson 10 - Online Crimes and HazardsSeth Reyes
 
20230202 - Introduction to tis-py
20230202 - Introduction to tis-py20230202 - Introduction to tis-py
20230202 - Introduction to tis-pyJamie (Taka) Wang
 
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCostKubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCostMatt Ray
 
Introduction to Matsuo Laboratory (ENG).pptx
Introduction to Matsuo Laboratory (ENG).pptxIntroduction to Matsuo Laboratory (ENG).pptx
Introduction to Matsuo Laboratory (ENG).pptxMatsuo Lab
 
Building AI-Driven Apps Using Semantic Kernel.pptx
Building AI-Driven Apps Using Semantic Kernel.pptxBuilding AI-Driven Apps Using Semantic Kernel.pptx
Building AI-Driven Apps Using Semantic Kernel.pptxUdaiappa Ramachandran
 
COMPUTER 10 Lesson 8 - Building a Website
COMPUTER 10 Lesson 8 - Building a WebsiteCOMPUTER 10 Lesson 8 - Building a Website
COMPUTER 10 Lesson 8 - Building a Websitedgelyza
 

Kürzlich hochgeladen (20)

Empowering Africa's Next Generation: The AI Leadership Blueprint
Empowering Africa's Next Generation: The AI Leadership BlueprintEmpowering Africa's Next Generation: The AI Leadership Blueprint
Empowering Africa's Next Generation: The AI Leadership Blueprint
 
UiPath Studio Web workshop series - Day 7
UiPath Studio Web workshop series - Day 7UiPath Studio Web workshop series - Day 7
UiPath Studio Web workshop series - Day 7
 
OpenShift Commons Paris - Choose Your Own Observability Adventure
OpenShift Commons Paris - Choose Your Own Observability AdventureOpenShift Commons Paris - Choose Your Own Observability Adventure
OpenShift Commons Paris - Choose Your Own Observability Adventure
 
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...
 
COMPUTER 10: Lesson 7 - File Storage and Online Collaboration
COMPUTER 10: Lesson 7 - File Storage and Online CollaborationCOMPUTER 10: Lesson 7 - File Storage and Online Collaboration
COMPUTER 10: Lesson 7 - File Storage and Online Collaboration
 
Nanopower In Semiconductor Industry.pdf
Nanopower  In Semiconductor Industry.pdfNanopower  In Semiconductor Industry.pdf
Nanopower In Semiconductor Industry.pdf
 
201610817 - edge part1
201610817 - edge part1201610817 - edge part1
201610817 - edge part1
 
20150722 - AGV
20150722 - AGV20150722 - AGV
20150722 - AGV
 
Bird eye's view on Camunda open source ecosystem
Bird eye's view on Camunda open source ecosystemBird eye's view on Camunda open source ecosystem
Bird eye's view on Camunda open source ecosystem
 
The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...
The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...
The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...
 
Comparing Sidecar-less Service Mesh from Cilium and Istio
Comparing Sidecar-less Service Mesh from Cilium and IstioComparing Sidecar-less Service Mesh from Cilium and Istio
Comparing Sidecar-less Service Mesh from Cilium and Istio
 
Linked Data in Production: Moving Beyond Ontologies
Linked Data in Production: Moving Beyond OntologiesLinked Data in Production: Moving Beyond Ontologies
Linked Data in Production: Moving Beyond Ontologies
 
activity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdf
activity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdf
activity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdf
 
UiPath Platform: The Backend Engine Powering Your Automation - Session 1
UiPath Platform: The Backend Engine Powering Your Automation - Session 1UiPath Platform: The Backend Engine Powering Your Automation - Session 1
UiPath Platform: The Backend Engine Powering Your Automation - Session 1
 
Computer 10: Lesson 10 - Online Crimes and Hazards
Computer 10: Lesson 10 - Online Crimes and HazardsComputer 10: Lesson 10 - Online Crimes and Hazards
Computer 10: Lesson 10 - Online Crimes and Hazards
 
20230202 - Introduction to tis-py
20230202 - Introduction to tis-py20230202 - Introduction to tis-py
20230202 - Introduction to tis-py
 
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCostKubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
 
Introduction to Matsuo Laboratory (ENG).pptx
Introduction to Matsuo Laboratory (ENG).pptxIntroduction to Matsuo Laboratory (ENG).pptx
Introduction to Matsuo Laboratory (ENG).pptx
 
Building AI-Driven Apps Using Semantic Kernel.pptx
Building AI-Driven Apps Using Semantic Kernel.pptxBuilding AI-Driven Apps Using Semantic Kernel.pptx
Building AI-Driven Apps Using Semantic Kernel.pptx
 
COMPUTER 10 Lesson 8 - Building a Website
COMPUTER 10 Lesson 8 - Building a WebsiteCOMPUTER 10 Lesson 8 - Building a Website
COMPUTER 10 Lesson 8 - Building a Website
 

Machine Learning Based Video Coding Enhancements for HTTP Adaptive Streaming

  • 1. All rights reserved. ©2020 All rights reserved. ©2020 Machine Learning Based Video Coding Enhancements for HTTP Adaptive Streaming ACM MMSys’21 Doctoral Symposium September 30, 2021 Ekrem Çetinkaya Christian Doppler Laboratory ATHENA | Alpen-Adria-Universität Klagenfurt | Austria ekrem.cetinkaya@aau.at | athena.itec.aau.at 1
  • 2. All rights reserved. ©2020 ● Introduction ● Research Questions ● Methodology & Existing Results ● Ongoing & Future Work ● Q & A Agenda All rights reserved. ©2020 2
  • 4. All rights reserved. ©2020 Video Streaming Share in the Internet Traffic 82% Content Characteristics 1 Million minutes Video Streamed Every Second As of 2021 * Cisco VNI Forecast Highlights (2021) All rights reserved. ©2020 4
  • 5. All rights reserved. ©2020 HTTP Adaptive Streaming (HAS) Very Nice Video Play Play 5 240 kbps Client HAS Server 1200 kbps 3500 kbps 480 kbps 2500 kbps 7000 kbps
  • 6. All rights reserved. ©2020 Video Encoding Block Partitioning Motion Compensation Transformation & Quantization Entropy Coding Entropy Decoding Inverse Transformation & Inverse Quantization Inter or Intra Prediction Picture Buffer In-loop Filtering 6
  • 7. All rights reserved. ©2020 Video Codecs C. Feldmann, “State of Compression Standards - VVC”, 2020, https://bitmovin.com/compression-standards-vvc-2020/ Vanne et.al., “Comparative Rate-Distortion-Complexity Analysis of HEVC and AVC Video Codecs”, TCSVT, 2012 High Efficiency Video Coding (HEVC) 2003 Advanced Video Coding (AVC) 2013 Versatile Video Coding (VVC) 2020 170 % 954 % 37 % 35 % Block size 16x16 Quaternary tree Supports up to 4K Block size 64x64 Quaternary tree Supports up to 8K Block size 128x128 Multi-type tree Supports up to 16K, 360° videos 7
  • 8. All rights reserved. ©2020 Video Encoding with Machine Learning Block Partitioning Motion Compensation Transformation & Quantization Entropy Coding Entropy Decoding Inverse Transformation & Inverse Quantization Inter or Intra Prediction Picture Buffer In-loop Filtering Block Partitioning Decision Prediction Optical Flow Detection Mode Prediction Angular Direction Prediction Deblocking with ML Denoising with ML Super-resolution 8
  • 9. Research Questions All rights reserved. ©2020 9
  • 10. All rights reserved. ©2020 10 RQ-1 How to efficiently provide multi-rate representations over a wide range of resolutions for HAS? RQ-2 How to improve the performance of video codecs using machine learning? RQ-4 How to use machine learning to improve perceptual quality assessment for videos? RQ-3 How to improve the visual quality of videos using machine learning? Why? 🔋High-resolution content is getting more common, required number of representations for HAS is increasing. Literature 🗂 ML based approaches are utilized in video codecs to speed up encoder decision. 🗂 Some attempts in end-to-end ML based video codecs. Literature 🗂 ML based refinement techniques applied. 🗂 Post-processing in decoded frames to improve quality. 🗂 Super-resolution for images and videos. Literature 🗂 ML model is used in VMAF. 🗂 Several more attempts for non-reference perceptual quality assessment. Why? 🔋ML based image restoration methods are improving, however video is mostly ignored. QoE can be increased. Why? 🔋Finding a reliable metric for perceptual quality is important as current objective metrics are problematic. Why? 🔋More complex codecs, many possibilities to apply ML, still much room for improvement. Literature 🗂 Choose a reference representation and use its information to speed up remaining encodings.
  • 11. Methodology & Existing Results All rights reserved. ©2020 11
  • 12. All rights reserved. ©2020 Design and Abstraction Methodology Design Propose a solution (algorithm, concept, protocol, etc.) for a given problem Implement Prototype software implementation using the proposed solution Analyze Qualitative and quantitative analysis of the solution Repeat the cycle to improve the solutions 12
  • 13. All rights reserved. ©2020 ● State-of-the-art: ○ Encode the highest quality 1 or the lowest quality 2 as the reference first then use these information ● Proposed Method 3 : ○ Encode the highest quality first, ○ Use its information to encode the lowest quality ○ Use information from both representations to encode the remaining representations ○ Double bound for CTU search ranges 1 Schroeder, Damien, et al. "Efficient multi-rate video encoding for HEVC-based adaptive HTTP streaming." IEEE Transactions on Circuits and systems for Video Technology 28.1 (2016): 143-157. 2 B. Guo, Y. Han, J. Wen, "Fast Block Structure Determination in AV1-based Multiple Resolutions Video Encoding," in 2018 IEEE International Conference on Multimedia and Expo (ICME), San Diego, CA, USA, July 2018. 3 H. Amirpour, E. Çetinkaya, C. Timmerer and M. Ghanbari, "Fast Multi-rate Encoding for Adaptive HTTP Streaming," 2020 Data Compression Conference (DCC), Snowbird, UT, USA, 2020, pp. 358-358 QP1 QPN QPN-1 QP3 QP2 ... 13 Fast Multi-rate Encoding (DCC’20)
  • 14. All rights reserved. ©2020 14 Fast Multi-rate Encoding (DCC’20)
  • 15. All rights reserved. ©2020 ● State-of-the-art: ○ Encode the highest quality 1 or the lowest quality 2 as the reference first then use these information ● Proposed Method 3 : ○ Try different quality levels as the reference representation to determine the best starting point for parallel encoding ○ Encode the middle quality first and use its information. ○ Upper or lower bound depending on the quality level Towards Optimal Multirate Encoding (MMM’21) 1 Schroeder, Damien, et al. "Efficient multi-rate video encoding for HEVC-based adaptive HTTP streaming." IEEE Transactions on Circuits and systems for Video Technology 28.1 (2016): 143-157. 2 B. Guo, Y. Han, J. Wen, "Fast Block Structure Determination in AV1-based Multiple Resolutions Video Encoding," in 2018 IEEE International Conference on Multimedia and Expo (ICME), San Diego, CA, USA, July 2018. 3 H. Amirpour, E. Çetinkaya, C. Timmerer and M. Ghanbari, "Towards Optimal Multirate Encoding for HTTP Adaptive Streaming," The International MultiMedia Modeling Conference (MMM), Prague, Czech Republic, 2021 QPN/2 QPN QP2 QP1 ... 15
  • 16. All rights reserved. ©2020 16 Towards Optimal Multirate Encoding (MMM’21)
  • 17. All rights reserved. ©2020 ● State-of-the-art: ○ Use a CNN to predict CTU depth decisions 1 ● Proposed Method 2 : ○ Train a CNN with encoding information obtained from the reference representation and use its decision to encode dependent representations. ○ Focus on parallel encoding, thus only apply for bottleneck situations ○ Train different CNNs for different QP targets 1 Kim, Kyungah, and Won Woo Ro. "Fast CU depth decision for HEVC using neural networks." IEEE Transactions on Circuits and Systems for Video Technology 29.5 (2018): 1462-1473. 2 E. Çetinkaya, H. Amirpour, C. Timmerer and M. Ghanbari, “FaME-ML: Fast Multirate Encoding for HTTP Adaptive Streaming Using Machine Learning,” 2020 IEEE International Conference on Visual Communications and Image Processing (VCIP), Macau, 2020, pp. 87-90. QPN CNN QPN-1 QP1 QP2 ... HEVC HEVC HEVC CNN HEVC HEVC 17 Fast Multi-rate Encoding with ML (VCIP’20)
  • 18. All rights reserved. ©2020 18 Fast Multi-rate Encoding with ML (VCIP’20)
  • 19. All rights reserved. ©2020 ● State-of-the-art: ○ Use the highest quality representation as the reference 1 ● Proposed Method 2 : ○ Train a CNN with encoding information obtained from the reference representation (the highest quality from the lowest resolution) and use its decision to encode dependent representations ○ Improves parallel encoding as well as serial encoding ○ Train different CNNs for different QP and resolution targets 1 Schroeder, Damien, et al. "Efficient multi-rate video encoding for HEVC-based adaptive HTTP streaming." IEEE Transactions on Circuits and systems for Video Technology 28.1 (2016): 143-157. 2 E. Çetinkaya, H. Amirpour, C. Timmerer and M. Ghanbari, "Fast Multi-Resolution and Multi-Rate Encoding for HTTP Adaptive Streaming Using Machine Learning," in IEEE Open Journal of Signal Processing, vol. 2, pp. 484-495, 2021, doi: 10.1109/OJSP.2021.3078657. 19 Fast Multi-rate and Multi-resolution Encoding with ML (IEEE OJ-SP) HEVC QP1 HEVC QP2 CNN HEVC QPN CNN HEVC .. CNN HEVC QP2 CNN HEVC QPN CNN HEVC .. CNN HEVC QP2 CNN HEVC QPN CNN HEVC .. CNN HEVC QP1 CNN HEVC QP1 CNN 540p 540p 1080p 2160p
  • 20. All rights reserved. ©2020 20 Fast Multi-rate and Multi-resolution Encoding with ML (IEEE OJ-SP) Normalized Encoding time HM 16.21 Lower Bound FaRes-ML
  • 21. Ongoing & Future Work All rights reserved. ©2020 21
  • 22. All rights reserved. ©2020 Work Plan 2019 Q4 2020 2021 2022 Q1 Q2 Q3 Q4 Q1 Q2 Q3 Q4 Q1 Q2 Q3 Q4 Q1 Q2 1. How to efficiently provide multi-bitrate representations over a wide range of resolutions for HAS ? 2. How to improve performance of video codecs using machine learning ? 3. How to improve quality of videos using machine learning ? 4. How to use machine learning to improve perceptual quality assessment for videos ? Literature review DCC’20 Paper MMM’21 Paper VCIP’20 Paper Multi-rate and Multi-resolution Encoding IEEE OJSP Paper RQ1 RQ2 RQ3 RQ4 Super-resolution Literature Review Perceptual Quality Assessment with ML 2023 Thesis Bitrate Ladder Prediction Literature Review Improvement in In-loop Filtering with ML Mobile Player Optimization with SR 22 Fast multi-rate encoding for adaptive http streaming Towards optimal multirate encoding for HTTP adaptive streaming FaME-ML: Fast multirate encoding for HTTP adaptive streaming using machine learning Fast Multi-Resolution and Multi-Rate Encoding for HTTP Adaptive Streaming Using Machine Learning
  • 23. All rights reserved. ©2020 Thank you! ekrem.cetinkaya@aau.at @ekremcetinkaya_ linkedin.com/in/ekrcet