Diese Präsentation wurde erfolgreich gemeldet.
Die SlideShare-Präsentation wird heruntergeladen. ×

JRS-Presentation-Optical-Flow-Algorithms-for-Restoration.pptx

Anzeige
Anzeige
Anzeige
Anzeige
Anzeige
Anzeige
Anzeige
Anzeige
Anzeige
Anzeige
Anzeige
Anzeige

Hier ansehen

1 von 8 Anzeige
Anzeige

Weitere Verwandte Inhalte

Aktuellste (20)

Anzeige

JRS-Presentation-Optical-Flow-Algorithms-for-Restoration.pptx

  1. 1. 1 A qualitative investigation of optical flow algorithms for video denoising Hannes Fassold, JOANNEUM RESEARCH 2022-11-28
  2. 2. Introduction & Motivation • Optical flow = pixel-wise motion field between two images (e.g. video frames) • Good optical flow estimation us crucial for many video processing tasks • E.g. object tracking, denoising, super-resolution … • Focus of work was to investigate how well optical flow algorithms perform qualitatively when integrated into a state of the art video denoising algorithm. • Both classic optical flow algorithms (e.g. TV-L1) as well as recent deep learning based algorithm (like RAFT or BMBC) are taken into account. • Tests are done with our own wavelet-based video denoising algorithm • Is integrated (in a significantly extended version – safeguards etc.) in the film restoration software “DIAMANT” by HS-Art. • HS-Art was also the host for this (virtual) secondment • Spin-off of JOANNEUM RESEARCH, located in Graz 2
  3. 3. Optical flow algorithm used in experiments • The TV-L1 optical flow algorithm (GPU) is a very popular method for optical flow estimation and employs variational calculus. • The Dense Inverse Search (DIS) optical flow algorithm (CPU) is designed specifically for runtime efficiency and integrated in OpenCV library. • The NV optical flow algorithm is a very fast hardware-accelerated optical flow provided on recent NVIDIA GPUs (Ampere are newer). • The RAFT method is a recent deep learning based method • One of best optical flow methods currently on the benchmark datasets. • The BMBC algorithm is actually a frame interpolation algorithm, but it can be employed also as an optical flow algorithm. • The LIFE optical flow method is a neural network which is explicitly designed to be robust against content with large motion or large brightness variations. 3
  4. 4. Experiment setup • The 2-phase JR video denoiser is employed (with safeguards disabled) • Phase 1 – wavelet-denoising using semi-local shrinkage functions • Phase 2 – temporal fusion within a 3-frame sliding window • In both phases, neighbor images are motion-compensated via optical flow • We employ different optical flow algorithms (TV-L1, DIS, NV, RAFT, BMBC, LIFE) for the motion compensation step within the denoiser • Qualitative experiments are done with different kinds of content • local motion only • fast motion • heavy noise • strong flicker / brightness variations 4
  5. 5. Comparison for video sequence with fast motion 5
  6. 6. Conclusions from qualitative investigation • No clear ”winner” can be determined, at least for our application scenario • Typical motion-compensation artifacts: halos, blurring, … • Newer deep learning based optical flow algorithm (especially RAFT) are not necessarily better than the classic methods like TV-L1 or NV • Indicates also that there is a gap between quantitative measures of optical flow performance (like endpoint error) on benchmark datasets and a qualitative evaluation in a certain application context (e.g. for video denoising) • There is a need for more realistic quantitative measures for the faithful evaluation of optical flow algorithms 6
  7. 7. Acknowledgment • This work was supported by European Union´s Horizon 2020 research and innovation programme under grant number 951911 - AI4Media. 7

×