SlideShare ist ein Scribd-Unternehmen logo
1 von 55
Yonsei University Severance Hospital CCIDS
ICCV 2019 Review
Dongmin Choi
ICCV
- ICCV : International Conference on Computer Vision
- 2019년 국내에서 첫 개최
: 10.27 ~ 11.2, COEX
- 2019 ICCV
: 약 7,500명 참가
총 1,077편의 논문
Yonsei University Severance Hospital CCIDS
ICCV
ICCV 2019 Summary
1. Visual Recognition for Medical Images
2. Image and Video Synthesis: How, Why and What if?
3. Interpretating and Explaining Visual AI Models
4. SinGAN: Learning a Generative Model from a Single Natural Images (Best Paper)
Yonsei University Severance Hospital CCIDS
Yonsei University Severance Hospital CCIDS
Visual Recognition
for Medical Images
https://sites.google.com/view/iccv19-vrmi
Visual Recognition for Medical Images
Yonsei University Severance Hospital CCIDS
Deep Learning in Neuroimaging and
Radiotherapy
Dinggang Shen
University of North Carolina at Chapel Hill
Visual Recognition for Medical Images
Yonsei University Severance Hospital CCIDS
Image Registration
Template T Subject S
Deep-learning based
Registration
J. Fan, X. Cao, Z. Xue, PT. Yap, D. Shen, “Adversarial Similarity Network for Evaluating Image Alignment in Deep Learning based Registration”
Visual Recognition for Medical Images
Yonsei University Severance Hospital CCIDS
Train the registration network by Similarity Metric
Template T
Subject S
Deep Registration
Network (U-Net)
T
S
68 * 68 * 68
68 * 68 * 68
Deformation
Field
3*28*28*28
Spatial
Transformation Warped
subject
28*28*28
Dissimilarity
Loss LD(T, S')
S'φ
J. Fan, X. Cao, Z. Xue, PT. Yap, D. Shen, “Adversarial Similarity Network for Evaluating Image Alignment in Deep Learning based Registration”
Visual Recognition for Medical Images
Yonsei University Severance Hospital CCIDS
Train the registration network by Similarity Metric (inspired by GAN)
Template T
Subject S
Deep Registration
Network (U-Net)
T
S
68 * 68 * 68
68 * 68 * 68
Deformation
Field
3*28*28*28
Spatial
Transformation Warped
subject
28*28*28
Discriminator
S'φ
J. Fan, X. Cao, Z. Xue, PT. Yap, D. Shen, “Adversarial Similarity Network for Evaluating Image Alignment in Deep Learning based Registration”
Generator
Visual Recognition for Medical Images
Yonsei University Severance Hospital CCIDS
On Unseen Testing Dataset
J. Fan, X. Cao, Z. Xue, PT. Yap, D. Shen, “Adversarial Similarity Network for Evaluating Image Alignment in Deep Learning based Registration”
Visual Recognition for Medical Images
Yonsei University Severance Hospital CCIDS
Registration Result
J. Fan, X. Cao, Z. Xue, PT. Yap, D. Shen, “Adversarial Similarity Network for Evaluating Image Alignment in Deep Learning based Registration”
Visual Recognition for Medical Images
Yonsei University Severance Hospital CCIDS
Image Synthesis
MRI CT
Can we synthesize CT image from
a single MRI Image?
CT images are highly desired for
- Specific diagnosis
- Dose planning
- PET attenuation correction
D. Nie et al,
“Medical Image Synthesis with Deep Convolutional Adversarial Networks”
Visual Recognition for Medical Images
Yonsei University Severance Hospital CCIDS
D. Nie et al,
“Medical Image Synthesis with Deep Convolutional Adversarial Networks”
Auto-context
Use context features
to iteratively refine
the training results
Visual Recognition for Medical Images
Yonsei University Severance Hospital CCIDS
D. Nie et al,
“Medical Image Synthesis with Deep Convolutional Adversarial Networks”
Auto-context Refinements
Difference maps
Visual Recognition for Medical Images
Yonsei University Severance Hospital CCIDS
Q&A
Q. Whole Image가 아니라 Patch를 사용하는 이유?
A. 데이터 수가 딥러닝을 학습하기에는 부족하기 때문이다. Patch를 사용하면 훨씬 많은 학습 데이터를
확보할 수 있기 때문에 patch를 사용한다.
물론 데이터 수가 충분히 많다면 patch가 아니라 whole image를 사용하는 것이 더욱 좋다.
Yonsei University Severance Hospital CCIDS
Image and Video Synthesis:
How, Why and What if
https://sites.google.com/berkeley.edu/iccv-2019-image-and-video-syn
Image and Video Synthesis: How, Why and What if?
Yonsei University Severance Hospital CCIDS
Seeing What
a GAN Cannot Generate
David Bau
MIT
http://ganseeing.csail.mit.edu//
Seeing What a GAN Cannot Generate
Yonsei University Severance Hospital CCIDS
https://software.intel.com/en-us/blogs/2017/08/21/mode-collapse-in-gans
Model Collapse in GAN is serious Problem
※ Model Collapse : A problem when all the generator outputs are identical
(all of them or most of the samples are equal)
Seeing What a GAN Cannot Generate
Yonsei University Severance Hospital CCIDS
Fake Images
Real Images
http://ganseeing.csail.mit.edu//
Seeing What a GAN Cannot Generate
Yonsei University Severance Hospital CCIDS
Fake Images
Real Images
Inception
Inception
Fake Inception
Feature Space
Real Inception
Feature Space
http://ganseeing.csail.mit.edu//
FID (Frechet Inception Distance)
Measuring
GAN Quality
Seeing What a GAN Cannot Generate
Yonsei University Severance Hospital CCIDS
Fake Images
Real Images
http://ganseeing.csail.mit.edu//
1. What is actually missing
in the distribution?
2. What is actually missing
in each image?
Seeing What a GAN Cannot Generate
Yonsei University Severance Hospital CCIDS
http://ganseeing.csail.mit.edu//
1. Understanding Omissions in the Distribution
Generated Image Semantic segmentation
Real Image Semantic segmentation
Seeing What a GAN Cannot Generate
Yonsei University Severance Hospital CCIDS
http://ganseeing.csail.mit.edu//
1. Understanding Omissions in the Distribution
Generated Image Semantic segmentation
Real Image Semantic segmentation
Seeing What a GAN Cannot Generate
Yonsei University Severance Hospital CCIDS
http://ganseeing.csail.mit.edu//
2. Understanding Omissions in Individual Images
Real Image x Synthesized Image G(z)
Seeing What a GAN Cannot Generate
Yonsei University Severance Hospital CCIDS
http://ganseeing.csail.mit.edu//
2. Understanding Omissions in Individual Images
Real Image x Synthesized Image G(z)
Pairs (x, G(z*)) reveals omissions
Objective : z* = argminz Loss(x, G(z))
Seeing What a GAN Cannot Generate
Yonsei University Severance Hospital CCIDS
http://ganseeing.csail.mit.edu//
Two steps to invert a large generator
Seeing What a GAN Cannot Generate
Yonsei University Severance Hospital CCIDS
http://ganseeing.csail.mit.edu//
x = G(z) x = G(z*) x = real x = G(z*)
Generated Reconstruction Real Photo Reconstruction
When G generates x,
reconstruction is precise
When reconstruction is imperfect
we know G cannot generate x
Seeing What a GAN Cannot Generate
Yonsei University Severance Hospital CCIDS
http://ganseeing.csail.mit.edu//
GANs don’t like people
Seeing What a GAN Cannot Generate
Yonsei University Severance Hospital CCIDS
http://ganseeing.csail.mit.edu//
The Cheese Hypothesis
Original Image
Seeing What a GAN Cannot Generate
Yonsei University Severance Hospital CCIDS
http://ganseeing.csail.mit.edu//
The Cheese Hypothesis
Original Image
Optimized z
Seeing What a GAN Cannot Generate
Yonsei University Severance Hospital CCIDS
http://ganseeing.csail.mit.edu//
The Cheese Hypothesis
Original Image
Adapted Cheese
Seeing What a GAN Cannot Generate
Yonsei University Severance Hospital CCIDS
Real Image x
Seeing What a GAN Cannot Generate
Yonsei University Severance Hospital CCIDS
Optimized
vector
z*
G
Real Image x Reconstructed Image G(z*)
z* = argminz Loss(x, G(z))
Seeing What a GAN Cannot Generate
Yonsei University Severance Hospital CCIDS
Optimized
vector
z*
G
Real Image x Reconstructed Image G(z*, θ*)
θ*
z*, θ* = argminz,θ Loss(x, G(z)) + R(θ)
Regularizer
Inspired by Deep Image Prior [Ulyanove et al, 2018]
Yonsei University Severance Hospital CCIDS
Interpretating and Explaining
Visual AI Models
http://xai.unist.ac.kr/workshop/2019/
Interpretating and Explaining Visual AI Models
Yonsei University Severance Hospital CCIDS
Recent progress towards XAI
at UC Berkeley
Trevor Darrel
UC Berkeley
Recent progress towards XAI at UC Berkeley
Yonsei University Severance Hospital CCIDS
Recent progress towards XAI at UC Berkeley
Yonsei University Severance Hospital CCIDS
Explainable NN
Recent progress towards XAI at UC Berkeley
Yonsei University Severance Hospital CCIDS
Salience for Introspection
Image LIME GradCAM RISE
V Petsiuk et al. RISE: Randomized Input Sampling for Explanation of Black-box Models
Recent progress towards XAI at UC Berkeley
Yonsei University Severance Hospital CCIDS
Overview of RISE
V Petsiuk et al. RISE: Randomized Input Sampling for Explanation of Black-box Models
Recent progress towards XAI at UC Berkeley
Yonsei University Severance Hospital CCIDS
Pros of RISE
V Petsiuk et al. RISE: Randomized Input Sampling for Explanation of Black-box Models
Cons of RISE
A more general framework
as the importance map is obtained with
access to only the input and
output of the base model
Time and resource complexity
Recent progress towards XAI at UC Berkeley
Yonsei University Severance Hospital CCIDS
LRP : Layer-wise Relevance Propagation
Explaining Decisions of Neural Networks by LRP. Alexander Binder
@ Deep Learning: Theory, Algorithms, and Applications. Berlin, June 2017
Recent progress towards XAI at UC Berkeley
Yonsei University Severance Hospital CCIDS
LRP : Layer-wise Relevance Propagation
Explaining Decisions of Neural Networks by LRP. Alexander Binder
@ Deep Learning: Theory, Algorithms, and Applications. Berlin, June 2017
Recent progress towards XAI at UC Berkeley
Yonsei University Severance Hospital CCIDS
LRP : Layer-wise Relevance Propagation
Explaining Decisions of Neural Networks by LRP. Alexander Binder
@ Deep Learning: Theory, Algorithms, and Applications. Berlin, June 2017
Recent progress towards XAI at UC Berkeley
Yonsei University Severance Hospital CCIDS
Unmasking Clever Hans Predictors
S Lapuschkin et al. Unmasking Clever Hans Predictors and Assessing What Machines Really Learn
Images with a copyright watermark (a, c) are classified to “horse” class,
but images without an added copyright watermark (b, d) aren’t.
Recent progress towards XAI at UC Berkeley
Yonsei University Severance Hospital CCIDS
Understanding Learning Behaviour
S Lapuschkin et al. Unmasking Clever Hans Predictors and Assessing What Machines Really Learn
Model learns
1. Track the ball
2. Focus on paddle
3. Focus in the tunnel
Yonsei University Severance Hospital CCIDS
Learning a Generative Model
from a Single Natural Image
https://sites.google.com/berkeley.edu/iccv-2019-image-and-video-syn
SinGAN
Tamar Rott Shaham
Google AI
ICCV 2019 Best Paper
SinGAN
Yonsei University Severance Hospital CCIDS
SinGAN
Yonsei University Severance Hospital CCIDS
SinGAN
Yonsei University Severance Hospital CCIDS
SinGAN
Yonsei University Severance Hospital CCIDS
▶ Same model
▶ No extra training
SinGAN for single image animation
https://www.youtube.com/watch?v=xk8bWLZk4DU&feature=youtu.be
SinGAN
Yonsei University Severance Hospital CCIDS
Model
SinGAN
Yonsei University Severance Hospital CCIDS
Summary
- SinGAN : Single Image GAN
- Controllable Generation
- Image Manipulation Applications
SinGAN
Yonsei University Severance Hospital CCIDS
MRI Toy Example
Training Image
SinGAN
Yonsei University Severance Hospital CCIDS
MRI Toy Example
Training Image Generated High-Resolution Image
Thank You

Weitere ähnliche Inhalte

Ähnlich wie ICCV 2019 REVIEW [CDM]

A survey of deep learning approaches to medical applications
A survey of deep learning approaches to medical applicationsA survey of deep learning approaches to medical applications
A survey of deep learning approaches to medical applicationsJoseph Paul Cohen PhD
 
Humanizing bioinformatics
Humanizing bioinformaticsHumanizing bioinformatics
Humanizing bioinformaticsJan Aerts
 
(20180728) kosaim workshop vuno - kyuhwan jung
(20180728) kosaim workshop   vuno - kyuhwan jung(20180728) kosaim workshop   vuno - kyuhwan jung
(20180728) kosaim workshop vuno - kyuhwan jungKyuhwan Jung
 
Generative Adversarial Networks and Their Applications in Medical Imaging
Generative Adversarial Networks  and Their Applications in Medical ImagingGenerative Adversarial Networks  and Their Applications in Medical Imaging
Generative Adversarial Networks and Their Applications in Medical ImagingSanghoon Hong
 
Deep learning applications - for fun and profit
Deep learning applications -  for fun and profitDeep learning applications -  for fun and profit
Deep learning applications - for fun and profitRoberto Pereira Silveira
 
영상기반 딥러닝 의료 분야 응용 (KIST 김영준) - 2017 대한의료영상학회 발표
영상기반 딥러닝 의료 분야 응용 (KIST 김영준) - 2017 대한의료영상학회 발표영상기반 딥러닝 의료 분야 응용 (KIST 김영준) - 2017 대한의료영상학회 발표
영상기반 딥러닝 의료 분야 응용 (KIST 김영준) - 2017 대한의료영상학회 발표Youngjun Kim
 
Application of generative adversarial networks (GAN) for ophthalmology image ...
Application of generative adversarial networks (GAN) for ophthalmology image ...Application of generative adversarial networks (GAN) for ophthalmology image ...
Application of generative adversarial networks (GAN) for ophthalmology image ...Daniel983829
 
(2017/06)Practical points of deep learning for medical imaging
(2017/06)Practical points of deep learning for medical imaging(2017/06)Practical points of deep learning for medical imaging
(2017/06)Practical points of deep learning for medical imagingKyuhwan Jung
 
Recent advances in diagnosis and treatment planning1
Recent advances in diagnosis and treatment  planning1Recent advances in diagnosis and treatment  planning1
Recent advances in diagnosis and treatment planning1Indian dental academy
 
ZERNIKE-ENTROPY IMAGE SIMILARITY MEASURE BASED ON JOINT HISTOGRAM FOR FACE RE...
ZERNIKE-ENTROPY IMAGE SIMILARITY MEASURE BASED ON JOINT HISTOGRAM FOR FACE RE...ZERNIKE-ENTROPY IMAGE SIMILARITY MEASURE BASED ON JOINT HISTOGRAM FOR FACE RE...
ZERNIKE-ENTROPY IMAGE SIMILARITY MEASURE BASED ON JOINT HISTOGRAM FOR FACE RE...AM Publications
 
Ph.D. Defense Presentation Slides (Changhee Han) カリスの東大博論審査会(公聴会)発表スライド Patho...
Ph.D. Defense Presentation Slides (Changhee Han) カリスの東大博論審査会(公聴会)発表スライド Patho...Ph.D. Defense Presentation Slides (Changhee Han) カリスの東大博論審査会(公聴会)発表スライド Patho...
Ph.D. Defense Presentation Slides (Changhee Han) カリスの東大博論審査会(公聴会)発表スライド Patho...カリス 東大AI博士
 
Deep learning for biomedicine
Deep learning for biomedicineDeep learning for biomedicine
Deep learning for biomedicineDeakin University
 
Scio12 sem web_final
Scio12 sem web_finalScio12 sem web_final
Scio12 sem web_finalKristi Holmes
 
Fundamentals and Innovations in medical imaging.pptx
Fundamentals and Innovations in medical imaging.pptxFundamentals and Innovations in medical imaging.pptx
Fundamentals and Innovations in medical imaging.pptxHemant Rakesh
 
Matera Summer School AI&XR 2023.pdf
Matera Summer School AI&XR 2023.pdfMatera Summer School AI&XR 2023.pdf
Matera Summer School AI&XR 2023.pdfJoaquim Jorge
 
Pneumonia Detection System using AI
Pneumonia Detection System using AIPneumonia Detection System using AI
Pneumonia Detection System using AIIRJET Journal
 
Generative Adversarial Networksの基礎と応用について
Generative Adversarial Networksの基礎と応用についてGenerative Adversarial Networksの基礎と応用について
Generative Adversarial Networksの基礎と応用についてSo Hasegawa
 

Ähnlich wie ICCV 2019 REVIEW [CDM] (20)

Introduction to Programming
Introduction to ProgrammingIntroduction to Programming
Introduction to Programming
 
A survey of deep learning approaches to medical applications
A survey of deep learning approaches to medical applicationsA survey of deep learning approaches to medical applications
A survey of deep learning approaches to medical applications
 
Humanizing bioinformatics
Humanizing bioinformaticsHumanizing bioinformatics
Humanizing bioinformatics
 
(20180728) kosaim workshop vuno - kyuhwan jung
(20180728) kosaim workshop   vuno - kyuhwan jung(20180728) kosaim workshop   vuno - kyuhwan jung
(20180728) kosaim workshop vuno - kyuhwan jung
 
Generative Adversarial Networks and Their Applications in Medical Imaging
Generative Adversarial Networks  and Their Applications in Medical ImagingGenerative Adversarial Networks  and Their Applications in Medical Imaging
Generative Adversarial Networks and Their Applications in Medical Imaging
 
Deep learning applications - for fun and profit
Deep learning applications -  for fun and profitDeep learning applications -  for fun and profit
Deep learning applications - for fun and profit
 
영상기반 딥러닝 의료 분야 응용 (KIST 김영준) - 2017 대한의료영상학회 발표
영상기반 딥러닝 의료 분야 응용 (KIST 김영준) - 2017 대한의료영상학회 발표영상기반 딥러닝 의료 분야 응용 (KIST 김영준) - 2017 대한의료영상학회 발표
영상기반 딥러닝 의료 분야 응용 (KIST 김영준) - 2017 대한의료영상학회 발표
 
Application of generative adversarial networks (GAN) for ophthalmology image ...
Application of generative adversarial networks (GAN) for ophthalmology image ...Application of generative adversarial networks (GAN) for ophthalmology image ...
Application of generative adversarial networks (GAN) for ophthalmology image ...
 
(2017/06)Practical points of deep learning for medical imaging
(2017/06)Practical points of deep learning for medical imaging(2017/06)Practical points of deep learning for medical imaging
(2017/06)Practical points of deep learning for medical imaging
 
Recent advances in diagnosis and treatment planning1
Recent advances in diagnosis and treatment  planning1Recent advances in diagnosis and treatment  planning1
Recent advances in diagnosis and treatment planning1
 
resume
resumeresume
resume
 
ZERNIKE-ENTROPY IMAGE SIMILARITY MEASURE BASED ON JOINT HISTOGRAM FOR FACE RE...
ZERNIKE-ENTROPY IMAGE SIMILARITY MEASURE BASED ON JOINT HISTOGRAM FOR FACE RE...ZERNIKE-ENTROPY IMAGE SIMILARITY MEASURE BASED ON JOINT HISTOGRAM FOR FACE RE...
ZERNIKE-ENTROPY IMAGE SIMILARITY MEASURE BASED ON JOINT HISTOGRAM FOR FACE RE...
 
Ph.D. Defense Presentation Slides (Changhee Han) カリスの東大博論審査会(公聴会)発表スライド Patho...
Ph.D. Defense Presentation Slides (Changhee Han) カリスの東大博論審査会(公聴会)発表スライド Patho...Ph.D. Defense Presentation Slides (Changhee Han) カリスの東大博論審査会(公聴会)発表スライド Patho...
Ph.D. Defense Presentation Slides (Changhee Han) カリスの東大博論審査会(公聴会)発表スライド Patho...
 
Deep learning for biomedicine
Deep learning for biomedicineDeep learning for biomedicine
Deep learning for biomedicine
 
SHUBHAM _IET
SHUBHAM _IETSHUBHAM _IET
SHUBHAM _IET
 
Scio12 sem web_final
Scio12 sem web_finalScio12 sem web_final
Scio12 sem web_final
 
Fundamentals and Innovations in medical imaging.pptx
Fundamentals and Innovations in medical imaging.pptxFundamentals and Innovations in medical imaging.pptx
Fundamentals and Innovations in medical imaging.pptx
 
Matera Summer School AI&XR 2023.pdf
Matera Summer School AI&XR 2023.pdfMatera Summer School AI&XR 2023.pdf
Matera Summer School AI&XR 2023.pdf
 
Pneumonia Detection System using AI
Pneumonia Detection System using AIPneumonia Detection System using AI
Pneumonia Detection System using AI
 
Generative Adversarial Networksの基礎と応用について
Generative Adversarial Networksの基礎と応用についてGenerative Adversarial Networksの基礎と応用について
Generative Adversarial Networksの基礎と応用について
 

Mehr von Dongmin Choi

[Review] BoxInst: High-Performance Instance Segmentation with Box Annotations...
[Review] BoxInst: High-Performance Instance Segmentation with Box Annotations...[Review] BoxInst: High-Performance Instance Segmentation with Box Annotations...
[Review] BoxInst: High-Performance Instance Segmentation with Box Annotations...Dongmin Choi
 
Review: Incremental Few-shot Instance Segmentation [CDM]
Review: Incremental Few-shot Instance Segmentation [CDM]Review: Incremental Few-shot Instance Segmentation [CDM]
Review: Incremental Few-shot Instance Segmentation [CDM]Dongmin Choi
 
Review: You Only Look One-level Feature
Review: You Only Look One-level FeatureReview: You Only Look One-level Feature
Review: You Only Look One-level FeatureDongmin Choi
 
Transformer in Computer Vision
Transformer in Computer VisionTransformer in Computer Vision
Transformer in Computer VisionDongmin Choi
 
Review : Adaptive Consistency Regularization for Semi-Supervised Transfer Lea...
Review : Adaptive Consistency Regularization for Semi-Supervised Transfer Lea...Review : Adaptive Consistency Regularization for Semi-Supervised Transfer Lea...
Review : Adaptive Consistency Regularization for Semi-Supervised Transfer Lea...Dongmin Choi
 
YolactEdge Review [cdm]
YolactEdge Review [cdm]YolactEdge Review [cdm]
YolactEdge Review [cdm]Dongmin Choi
 
Review : Inter-slice Context Residual Learning for 3D Medical Image Segmentation
Review : Inter-slice Context Residual Learning for 3D Medical Image SegmentationReview : Inter-slice Context Residual Learning for 3D Medical Image Segmentation
Review : Inter-slice Context Residual Learning for 3D Medical Image SegmentationDongmin Choi
 
Deformable DETR Review [CDM]
Deformable DETR Review [CDM]Deformable DETR Review [CDM]
Deformable DETR Review [CDM]Dongmin Choi
 
ViT (Vision Transformer) Review [CDM]
ViT (Vision Transformer) Review [CDM]ViT (Vision Transformer) Review [CDM]
ViT (Vision Transformer) Review [CDM]Dongmin Choi
 
Review : Prototype Mixture Models for Few-shot Semantic Segmentation
Review : Prototype Mixture Models for Few-shot Semantic SegmentationReview : Prototype Mixture Models for Few-shot Semantic Segmentation
Review : Prototype Mixture Models for Few-shot Semantic SegmentationDongmin Choi
 
Review : PolarMask: Single Shot Instance Segmentation with Polar Representati...
Review : PolarMask: Single Shot Instance Segmentation with Polar Representati...Review : PolarMask: Single Shot Instance Segmentation with Polar Representati...
Review : PolarMask: Single Shot Instance Segmentation with Polar Representati...Dongmin Choi
 
Review : Multi-Domain Image Completion for Random Missing Input Data [cdm]
Review : Multi-Domain Image Completion for Random Missing Input Data [cdm]Review : Multi-Domain Image Completion for Random Missing Input Data [cdm]
Review : Multi-Domain Image Completion for Random Missing Input Data [cdm]Dongmin Choi
 
Review : Rethinking Pre-training and Self-training
Review : Rethinking Pre-training and Self-trainingReview : Rethinking Pre-training and Self-training
Review : Rethinking Pre-training and Self-trainingDongmin Choi
 
Review : Structure Boundary Preserving Segmentation
for Medical Image with Am...
Review : Structure Boundary Preserving Segmentation
for Medical Image with Am...Review : Structure Boundary Preserving Segmentation
for Medical Image with Am...
Review : Structure Boundary Preserving Segmentation
for Medical Image with Am...Dongmin Choi
 
Pyradiomics Customization [CDM]
Pyradiomics Customization [CDM]Pyradiomics Customization [CDM]
Pyradiomics Customization [CDM]Dongmin Choi
 
Neural network pruning with residual connections and limited-data review [cdm]
Neural network pruning with residual connections and limited-data review [cdm]Neural network pruning with residual connections and limited-data review [cdm]
Neural network pruning with residual connections and limited-data review [cdm]Dongmin Choi
 
Network Deconvolution review [cdm]
Network Deconvolution review [cdm]Network Deconvolution review [cdm]
Network Deconvolution review [cdm]Dongmin Choi
 
How much position information do convolutional neural networks encode? review...
How much position information do convolutional neural networks encode? review...How much position information do convolutional neural networks encode? review...
How much position information do convolutional neural networks encode? review...Dongmin Choi
 
Objects as points (CenterNet) review [CDM]
Objects as points (CenterNet) review [CDM]Objects as points (CenterNet) review [CDM]
Objects as points (CenterNet) review [CDM]Dongmin Choi
 
Augmix review [cdm]
Augmix review [cdm]Augmix review [cdm]
Augmix review [cdm]Dongmin Choi
 

Mehr von Dongmin Choi (20)

[Review] BoxInst: High-Performance Instance Segmentation with Box Annotations...
[Review] BoxInst: High-Performance Instance Segmentation with Box Annotations...[Review] BoxInst: High-Performance Instance Segmentation with Box Annotations...
[Review] BoxInst: High-Performance Instance Segmentation with Box Annotations...
 
Review: Incremental Few-shot Instance Segmentation [CDM]
Review: Incremental Few-shot Instance Segmentation [CDM]Review: Incremental Few-shot Instance Segmentation [CDM]
Review: Incremental Few-shot Instance Segmentation [CDM]
 
Review: You Only Look One-level Feature
Review: You Only Look One-level FeatureReview: You Only Look One-level Feature
Review: You Only Look One-level Feature
 
Transformer in Computer Vision
Transformer in Computer VisionTransformer in Computer Vision
Transformer in Computer Vision
 
Review : Adaptive Consistency Regularization for Semi-Supervised Transfer Lea...
Review : Adaptive Consistency Regularization for Semi-Supervised Transfer Lea...Review : Adaptive Consistency Regularization for Semi-Supervised Transfer Lea...
Review : Adaptive Consistency Regularization for Semi-Supervised Transfer Lea...
 
YolactEdge Review [cdm]
YolactEdge Review [cdm]YolactEdge Review [cdm]
YolactEdge Review [cdm]
 
Review : Inter-slice Context Residual Learning for 3D Medical Image Segmentation
Review : Inter-slice Context Residual Learning for 3D Medical Image SegmentationReview : Inter-slice Context Residual Learning for 3D Medical Image Segmentation
Review : Inter-slice Context Residual Learning for 3D Medical Image Segmentation
 
Deformable DETR Review [CDM]
Deformable DETR Review [CDM]Deformable DETR Review [CDM]
Deformable DETR Review [CDM]
 
ViT (Vision Transformer) Review [CDM]
ViT (Vision Transformer) Review [CDM]ViT (Vision Transformer) Review [CDM]
ViT (Vision Transformer) Review [CDM]
 
Review : Prototype Mixture Models for Few-shot Semantic Segmentation
Review : Prototype Mixture Models for Few-shot Semantic SegmentationReview : Prototype Mixture Models for Few-shot Semantic Segmentation
Review : Prototype Mixture Models for Few-shot Semantic Segmentation
 
Review : PolarMask: Single Shot Instance Segmentation with Polar Representati...
Review : PolarMask: Single Shot Instance Segmentation with Polar Representati...Review : PolarMask: Single Shot Instance Segmentation with Polar Representati...
Review : PolarMask: Single Shot Instance Segmentation with Polar Representati...
 
Review : Multi-Domain Image Completion for Random Missing Input Data [cdm]
Review : Multi-Domain Image Completion for Random Missing Input Data [cdm]Review : Multi-Domain Image Completion for Random Missing Input Data [cdm]
Review : Multi-Domain Image Completion for Random Missing Input Data [cdm]
 
Review : Rethinking Pre-training and Self-training
Review : Rethinking Pre-training and Self-trainingReview : Rethinking Pre-training and Self-training
Review : Rethinking Pre-training and Self-training
 
Review : Structure Boundary Preserving Segmentation
for Medical Image with Am...
Review : Structure Boundary Preserving Segmentation
for Medical Image with Am...Review : Structure Boundary Preserving Segmentation
for Medical Image with Am...
Review : Structure Boundary Preserving Segmentation
for Medical Image with Am...
 
Pyradiomics Customization [CDM]
Pyradiomics Customization [CDM]Pyradiomics Customization [CDM]
Pyradiomics Customization [CDM]
 
Neural network pruning with residual connections and limited-data review [cdm]
Neural network pruning with residual connections and limited-data review [cdm]Neural network pruning with residual connections and limited-data review [cdm]
Neural network pruning with residual connections and limited-data review [cdm]
 
Network Deconvolution review [cdm]
Network Deconvolution review [cdm]Network Deconvolution review [cdm]
Network Deconvolution review [cdm]
 
How much position information do convolutional neural networks encode? review...
How much position information do convolutional neural networks encode? review...How much position information do convolutional neural networks encode? review...
How much position information do convolutional neural networks encode? review...
 
Objects as points (CenterNet) review [CDM]
Objects as points (CenterNet) review [CDM]Objects as points (CenterNet) review [CDM]
Objects as points (CenterNet) review [CDM]
 
Augmix review [cdm]
Augmix review [cdm]Augmix review [cdm]
Augmix review [cdm]
 

Kürzlich hochgeladen

#StandardsGoals for 2024: What’s new for BISAC - Tech Forum 2024
#StandardsGoals for 2024: What’s new for BISAC - Tech Forum 2024#StandardsGoals for 2024: What’s new for BISAC - Tech Forum 2024
#StandardsGoals for 2024: What’s new for BISAC - Tech Forum 2024BookNet Canada
 
Presentation on how to chat with PDF using ChatGPT code interpreter
Presentation on how to chat with PDF using ChatGPT code interpreterPresentation on how to chat with PDF using ChatGPT code interpreter
Presentation on how to chat with PDF using ChatGPT code interpreternaman860154
 
The Codex of Business Writing Software for Real-World Solutions 2.pptx
The Codex of Business Writing Software for Real-World Solutions 2.pptxThe Codex of Business Writing Software for Real-World Solutions 2.pptx
The Codex of Business Writing Software for Real-World Solutions 2.pptxMalak Abu Hammad
 
08448380779 Call Girls In Diplomatic Enclave Women Seeking Men
08448380779 Call Girls In Diplomatic Enclave Women Seeking Men08448380779 Call Girls In Diplomatic Enclave Women Seeking Men
08448380779 Call Girls In Diplomatic Enclave Women Seeking MenDelhi Call girls
 
My Hashitalk Indonesia April 2024 Presentation
My Hashitalk Indonesia April 2024 PresentationMy Hashitalk Indonesia April 2024 Presentation
My Hashitalk Indonesia April 2024 PresentationRidwan Fadjar
 
Transcript: #StandardsGoals for 2024: What’s new for BISAC - Tech Forum 2024
Transcript: #StandardsGoals for 2024: What’s new for BISAC - Tech Forum 2024Transcript: #StandardsGoals for 2024: What’s new for BISAC - Tech Forum 2024
Transcript: #StandardsGoals for 2024: What’s new for BISAC - Tech Forum 2024BookNet Canada
 
How to Troubleshoot Apps for the Modern Connected Worker
How to Troubleshoot Apps for the Modern Connected WorkerHow to Troubleshoot Apps for the Modern Connected Worker
How to Troubleshoot Apps for the Modern Connected WorkerThousandEyes
 
[2024]Digital Global Overview Report 2024 Meltwater.pdf
[2024]Digital Global Overview Report 2024 Meltwater.pdf[2024]Digital Global Overview Report 2024 Meltwater.pdf
[2024]Digital Global Overview Report 2024 Meltwater.pdfhans926745
 
A Domino Admins Adventures (Engage 2024)
A Domino Admins Adventures (Engage 2024)A Domino Admins Adventures (Engage 2024)
A Domino Admins Adventures (Engage 2024)Gabriella Davis
 
08448380779 Call Girls In Greater Kailash - I Women Seeking Men
08448380779 Call Girls In Greater Kailash - I Women Seeking Men08448380779 Call Girls In Greater Kailash - I Women Seeking Men
08448380779 Call Girls In Greater Kailash - I Women Seeking MenDelhi Call girls
 
SQL Database Design For Developers at php[tek] 2024
SQL Database Design For Developers at php[tek] 2024SQL Database Design For Developers at php[tek] 2024
SQL Database Design For Developers at php[tek] 2024Scott Keck-Warren
 
How to Remove Document Management Hurdles with X-Docs?
How to Remove Document Management Hurdles with X-Docs?How to Remove Document Management Hurdles with X-Docs?
How to Remove Document Management Hurdles with X-Docs?XfilesPro
 
WhatsApp 9892124323 ✓Call Girls In Kalyan ( Mumbai ) secure service
WhatsApp 9892124323 ✓Call Girls In Kalyan ( Mumbai ) secure serviceWhatsApp 9892124323 ✓Call Girls In Kalyan ( Mumbai ) secure service
WhatsApp 9892124323 ✓Call Girls In Kalyan ( Mumbai ) secure servicePooja Nehwal
 
Factors to Consider When Choosing Accounts Payable Services Providers.pptx
Factors to Consider When Choosing Accounts Payable Services Providers.pptxFactors to Consider When Choosing Accounts Payable Services Providers.pptx
Factors to Consider When Choosing Accounts Payable Services Providers.pptxKatpro Technologies
 
Injustice - Developers Among Us (SciFiDevCon 2024)
Injustice - Developers Among Us (SciFiDevCon 2024)Injustice - Developers Among Us (SciFiDevCon 2024)
Injustice - Developers Among Us (SciFiDevCon 2024)Allon Mureinik
 
Human Factors of XR: Using Human Factors to Design XR Systems
Human Factors of XR: Using Human Factors to Design XR SystemsHuman Factors of XR: Using Human Factors to Design XR Systems
Human Factors of XR: Using Human Factors to Design XR SystemsMark Billinghurst
 
08448380779 Call Girls In Friends Colony Women Seeking Men
08448380779 Call Girls In Friends Colony Women Seeking Men08448380779 Call Girls In Friends Colony Women Seeking Men
08448380779 Call Girls In Friends Colony Women Seeking MenDelhi Call girls
 
Enhancing Worker Digital Experience: A Hands-on Workshop for Partners
Enhancing Worker Digital Experience: A Hands-on Workshop for PartnersEnhancing Worker Digital Experience: A Hands-on Workshop for Partners
Enhancing Worker Digital Experience: A Hands-on Workshop for PartnersThousandEyes
 
From Event to Action: Accelerate Your Decision Making with Real-Time Automation
From Event to Action: Accelerate Your Decision Making with Real-Time AutomationFrom Event to Action: Accelerate Your Decision Making with Real-Time Automation
From Event to Action: Accelerate Your Decision Making with Real-Time AutomationSafe Software
 
Understanding the Laravel MVC Architecture
Understanding the Laravel MVC ArchitectureUnderstanding the Laravel MVC Architecture
Understanding the Laravel MVC ArchitecturePixlogix Infotech
 

Kürzlich hochgeladen (20)

#StandardsGoals for 2024: What’s new for BISAC - Tech Forum 2024
#StandardsGoals for 2024: What’s new for BISAC - Tech Forum 2024#StandardsGoals for 2024: What’s new for BISAC - Tech Forum 2024
#StandardsGoals for 2024: What’s new for BISAC - Tech Forum 2024
 
Presentation on how to chat with PDF using ChatGPT code interpreter
Presentation on how to chat with PDF using ChatGPT code interpreterPresentation on how to chat with PDF using ChatGPT code interpreter
Presentation on how to chat with PDF using ChatGPT code interpreter
 
The Codex of Business Writing Software for Real-World Solutions 2.pptx
The Codex of Business Writing Software for Real-World Solutions 2.pptxThe Codex of Business Writing Software for Real-World Solutions 2.pptx
The Codex of Business Writing Software for Real-World Solutions 2.pptx
 
08448380779 Call Girls In Diplomatic Enclave Women Seeking Men
08448380779 Call Girls In Diplomatic Enclave Women Seeking Men08448380779 Call Girls In Diplomatic Enclave Women Seeking Men
08448380779 Call Girls In Diplomatic Enclave Women Seeking Men
 
My Hashitalk Indonesia April 2024 Presentation
My Hashitalk Indonesia April 2024 PresentationMy Hashitalk Indonesia April 2024 Presentation
My Hashitalk Indonesia April 2024 Presentation
 
Transcript: #StandardsGoals for 2024: What’s new for BISAC - Tech Forum 2024
Transcript: #StandardsGoals for 2024: What’s new for BISAC - Tech Forum 2024Transcript: #StandardsGoals for 2024: What’s new for BISAC - Tech Forum 2024
Transcript: #StandardsGoals for 2024: What’s new for BISAC - Tech Forum 2024
 
How to Troubleshoot Apps for the Modern Connected Worker
How to Troubleshoot Apps for the Modern Connected WorkerHow to Troubleshoot Apps for the Modern Connected Worker
How to Troubleshoot Apps for the Modern Connected Worker
 
[2024]Digital Global Overview Report 2024 Meltwater.pdf
[2024]Digital Global Overview Report 2024 Meltwater.pdf[2024]Digital Global Overview Report 2024 Meltwater.pdf
[2024]Digital Global Overview Report 2024 Meltwater.pdf
 
A Domino Admins Adventures (Engage 2024)
A Domino Admins Adventures (Engage 2024)A Domino Admins Adventures (Engage 2024)
A Domino Admins Adventures (Engage 2024)
 
08448380779 Call Girls In Greater Kailash - I Women Seeking Men
08448380779 Call Girls In Greater Kailash - I Women Seeking Men08448380779 Call Girls In Greater Kailash - I Women Seeking Men
08448380779 Call Girls In Greater Kailash - I Women Seeking Men
 
SQL Database Design For Developers at php[tek] 2024
SQL Database Design For Developers at php[tek] 2024SQL Database Design For Developers at php[tek] 2024
SQL Database Design For Developers at php[tek] 2024
 
How to Remove Document Management Hurdles with X-Docs?
How to Remove Document Management Hurdles with X-Docs?How to Remove Document Management Hurdles with X-Docs?
How to Remove Document Management Hurdles with X-Docs?
 
WhatsApp 9892124323 ✓Call Girls In Kalyan ( Mumbai ) secure service
WhatsApp 9892124323 ✓Call Girls In Kalyan ( Mumbai ) secure serviceWhatsApp 9892124323 ✓Call Girls In Kalyan ( Mumbai ) secure service
WhatsApp 9892124323 ✓Call Girls In Kalyan ( Mumbai ) secure service
 
Factors to Consider When Choosing Accounts Payable Services Providers.pptx
Factors to Consider When Choosing Accounts Payable Services Providers.pptxFactors to Consider When Choosing Accounts Payable Services Providers.pptx
Factors to Consider When Choosing Accounts Payable Services Providers.pptx
 
Injustice - Developers Among Us (SciFiDevCon 2024)
Injustice - Developers Among Us (SciFiDevCon 2024)Injustice - Developers Among Us (SciFiDevCon 2024)
Injustice - Developers Among Us (SciFiDevCon 2024)
 
Human Factors of XR: Using Human Factors to Design XR Systems
Human Factors of XR: Using Human Factors to Design XR SystemsHuman Factors of XR: Using Human Factors to Design XR Systems
Human Factors of XR: Using Human Factors to Design XR Systems
 
08448380779 Call Girls In Friends Colony Women Seeking Men
08448380779 Call Girls In Friends Colony Women Seeking Men08448380779 Call Girls In Friends Colony Women Seeking Men
08448380779 Call Girls In Friends Colony Women Seeking Men
 
Enhancing Worker Digital Experience: A Hands-on Workshop for Partners
Enhancing Worker Digital Experience: A Hands-on Workshop for PartnersEnhancing Worker Digital Experience: A Hands-on Workshop for Partners
Enhancing Worker Digital Experience: A Hands-on Workshop for Partners
 
From Event to Action: Accelerate Your Decision Making with Real-Time Automation
From Event to Action: Accelerate Your Decision Making with Real-Time AutomationFrom Event to Action: Accelerate Your Decision Making with Real-Time Automation
From Event to Action: Accelerate Your Decision Making with Real-Time Automation
 
Understanding the Laravel MVC Architecture
Understanding the Laravel MVC ArchitectureUnderstanding the Laravel MVC Architecture
Understanding the Laravel MVC Architecture
 

ICCV 2019 REVIEW [CDM]

  • 1. Yonsei University Severance Hospital CCIDS ICCV 2019 Review Dongmin Choi
  • 2. ICCV - ICCV : International Conference on Computer Vision - 2019년 국내에서 첫 개최 : 10.27 ~ 11.2, COEX - 2019 ICCV : 약 7,500명 참가 총 1,077편의 논문 Yonsei University Severance Hospital CCIDS
  • 3. ICCV ICCV 2019 Summary 1. Visual Recognition for Medical Images 2. Image and Video Synthesis: How, Why and What if? 3. Interpretating and Explaining Visual AI Models 4. SinGAN: Learning a Generative Model from a Single Natural Images (Best Paper) Yonsei University Severance Hospital CCIDS
  • 4. Yonsei University Severance Hospital CCIDS Visual Recognition for Medical Images https://sites.google.com/view/iccv19-vrmi
  • 5. Visual Recognition for Medical Images Yonsei University Severance Hospital CCIDS Deep Learning in Neuroimaging and Radiotherapy Dinggang Shen University of North Carolina at Chapel Hill
  • 6. Visual Recognition for Medical Images Yonsei University Severance Hospital CCIDS Image Registration Template T Subject S Deep-learning based Registration J. Fan, X. Cao, Z. Xue, PT. Yap, D. Shen, “Adversarial Similarity Network for Evaluating Image Alignment in Deep Learning based Registration”
  • 7. Visual Recognition for Medical Images Yonsei University Severance Hospital CCIDS Train the registration network by Similarity Metric Template T Subject S Deep Registration Network (U-Net) T S 68 * 68 * 68 68 * 68 * 68 Deformation Field 3*28*28*28 Spatial Transformation Warped subject 28*28*28 Dissimilarity Loss LD(T, S') S'φ J. Fan, X. Cao, Z. Xue, PT. Yap, D. Shen, “Adversarial Similarity Network for Evaluating Image Alignment in Deep Learning based Registration”
  • 8. Visual Recognition for Medical Images Yonsei University Severance Hospital CCIDS Train the registration network by Similarity Metric (inspired by GAN) Template T Subject S Deep Registration Network (U-Net) T S 68 * 68 * 68 68 * 68 * 68 Deformation Field 3*28*28*28 Spatial Transformation Warped subject 28*28*28 Discriminator S'φ J. Fan, X. Cao, Z. Xue, PT. Yap, D. Shen, “Adversarial Similarity Network for Evaluating Image Alignment in Deep Learning based Registration” Generator
  • 9. Visual Recognition for Medical Images Yonsei University Severance Hospital CCIDS On Unseen Testing Dataset J. Fan, X. Cao, Z. Xue, PT. Yap, D. Shen, “Adversarial Similarity Network for Evaluating Image Alignment in Deep Learning based Registration”
  • 10. Visual Recognition for Medical Images Yonsei University Severance Hospital CCIDS Registration Result J. Fan, X. Cao, Z. Xue, PT. Yap, D. Shen, “Adversarial Similarity Network for Evaluating Image Alignment in Deep Learning based Registration”
  • 11. Visual Recognition for Medical Images Yonsei University Severance Hospital CCIDS Image Synthesis MRI CT Can we synthesize CT image from a single MRI Image? CT images are highly desired for - Specific diagnosis - Dose planning - PET attenuation correction D. Nie et al, “Medical Image Synthesis with Deep Convolutional Adversarial Networks”
  • 12. Visual Recognition for Medical Images Yonsei University Severance Hospital CCIDS D. Nie et al, “Medical Image Synthesis with Deep Convolutional Adversarial Networks” Auto-context Use context features to iteratively refine the training results
  • 13. Visual Recognition for Medical Images Yonsei University Severance Hospital CCIDS D. Nie et al, “Medical Image Synthesis with Deep Convolutional Adversarial Networks” Auto-context Refinements Difference maps
  • 14. Visual Recognition for Medical Images Yonsei University Severance Hospital CCIDS Q&A Q. Whole Image가 아니라 Patch를 사용하는 이유? A. 데이터 수가 딥러닝을 학습하기에는 부족하기 때문이다. Patch를 사용하면 훨씬 많은 학습 데이터를 확보할 수 있기 때문에 patch를 사용한다. 물론 데이터 수가 충분히 많다면 patch가 아니라 whole image를 사용하는 것이 더욱 좋다.
  • 15. Yonsei University Severance Hospital CCIDS Image and Video Synthesis: How, Why and What if https://sites.google.com/berkeley.edu/iccv-2019-image-and-video-syn
  • 16. Image and Video Synthesis: How, Why and What if? Yonsei University Severance Hospital CCIDS Seeing What a GAN Cannot Generate David Bau MIT http://ganseeing.csail.mit.edu//
  • 17. Seeing What a GAN Cannot Generate Yonsei University Severance Hospital CCIDS https://software.intel.com/en-us/blogs/2017/08/21/mode-collapse-in-gans Model Collapse in GAN is serious Problem ※ Model Collapse : A problem when all the generator outputs are identical (all of them or most of the samples are equal)
  • 18. Seeing What a GAN Cannot Generate Yonsei University Severance Hospital CCIDS Fake Images Real Images http://ganseeing.csail.mit.edu//
  • 19. Seeing What a GAN Cannot Generate Yonsei University Severance Hospital CCIDS Fake Images Real Images Inception Inception Fake Inception Feature Space Real Inception Feature Space http://ganseeing.csail.mit.edu// FID (Frechet Inception Distance) Measuring GAN Quality
  • 20. Seeing What a GAN Cannot Generate Yonsei University Severance Hospital CCIDS Fake Images Real Images http://ganseeing.csail.mit.edu// 1. What is actually missing in the distribution? 2. What is actually missing in each image?
  • 21. Seeing What a GAN Cannot Generate Yonsei University Severance Hospital CCIDS http://ganseeing.csail.mit.edu// 1. Understanding Omissions in the Distribution Generated Image Semantic segmentation Real Image Semantic segmentation
  • 22. Seeing What a GAN Cannot Generate Yonsei University Severance Hospital CCIDS http://ganseeing.csail.mit.edu// 1. Understanding Omissions in the Distribution Generated Image Semantic segmentation Real Image Semantic segmentation
  • 23. Seeing What a GAN Cannot Generate Yonsei University Severance Hospital CCIDS http://ganseeing.csail.mit.edu// 2. Understanding Omissions in Individual Images Real Image x Synthesized Image G(z)
  • 24. Seeing What a GAN Cannot Generate Yonsei University Severance Hospital CCIDS http://ganseeing.csail.mit.edu// 2. Understanding Omissions in Individual Images Real Image x Synthesized Image G(z) Pairs (x, G(z*)) reveals omissions Objective : z* = argminz Loss(x, G(z))
  • 25. Seeing What a GAN Cannot Generate Yonsei University Severance Hospital CCIDS http://ganseeing.csail.mit.edu// Two steps to invert a large generator
  • 26. Seeing What a GAN Cannot Generate Yonsei University Severance Hospital CCIDS http://ganseeing.csail.mit.edu// x = G(z) x = G(z*) x = real x = G(z*) Generated Reconstruction Real Photo Reconstruction When G generates x, reconstruction is precise When reconstruction is imperfect we know G cannot generate x
  • 27. Seeing What a GAN Cannot Generate Yonsei University Severance Hospital CCIDS http://ganseeing.csail.mit.edu// GANs don’t like people
  • 28. Seeing What a GAN Cannot Generate Yonsei University Severance Hospital CCIDS http://ganseeing.csail.mit.edu// The Cheese Hypothesis Original Image
  • 29. Seeing What a GAN Cannot Generate Yonsei University Severance Hospital CCIDS http://ganseeing.csail.mit.edu// The Cheese Hypothesis Original Image Optimized z
  • 30. Seeing What a GAN Cannot Generate Yonsei University Severance Hospital CCIDS http://ganseeing.csail.mit.edu// The Cheese Hypothesis Original Image Adapted Cheese
  • 31. Seeing What a GAN Cannot Generate Yonsei University Severance Hospital CCIDS Real Image x
  • 32. Seeing What a GAN Cannot Generate Yonsei University Severance Hospital CCIDS Optimized vector z* G Real Image x Reconstructed Image G(z*) z* = argminz Loss(x, G(z))
  • 33. Seeing What a GAN Cannot Generate Yonsei University Severance Hospital CCIDS Optimized vector z* G Real Image x Reconstructed Image G(z*, θ*) θ* z*, θ* = argminz,θ Loss(x, G(z)) + R(θ) Regularizer Inspired by Deep Image Prior [Ulyanove et al, 2018]
  • 34. Yonsei University Severance Hospital CCIDS Interpretating and Explaining Visual AI Models http://xai.unist.ac.kr/workshop/2019/
  • 35. Interpretating and Explaining Visual AI Models Yonsei University Severance Hospital CCIDS Recent progress towards XAI at UC Berkeley Trevor Darrel UC Berkeley
  • 36. Recent progress towards XAI at UC Berkeley Yonsei University Severance Hospital CCIDS
  • 37. Recent progress towards XAI at UC Berkeley Yonsei University Severance Hospital CCIDS Explainable NN
  • 38. Recent progress towards XAI at UC Berkeley Yonsei University Severance Hospital CCIDS Salience for Introspection Image LIME GradCAM RISE V Petsiuk et al. RISE: Randomized Input Sampling for Explanation of Black-box Models
  • 39. Recent progress towards XAI at UC Berkeley Yonsei University Severance Hospital CCIDS Overview of RISE V Petsiuk et al. RISE: Randomized Input Sampling for Explanation of Black-box Models
  • 40. Recent progress towards XAI at UC Berkeley Yonsei University Severance Hospital CCIDS Pros of RISE V Petsiuk et al. RISE: Randomized Input Sampling for Explanation of Black-box Models Cons of RISE A more general framework as the importance map is obtained with access to only the input and output of the base model Time and resource complexity
  • 41. Recent progress towards XAI at UC Berkeley Yonsei University Severance Hospital CCIDS LRP : Layer-wise Relevance Propagation Explaining Decisions of Neural Networks by LRP. Alexander Binder @ Deep Learning: Theory, Algorithms, and Applications. Berlin, June 2017
  • 42. Recent progress towards XAI at UC Berkeley Yonsei University Severance Hospital CCIDS LRP : Layer-wise Relevance Propagation Explaining Decisions of Neural Networks by LRP. Alexander Binder @ Deep Learning: Theory, Algorithms, and Applications. Berlin, June 2017
  • 43. Recent progress towards XAI at UC Berkeley Yonsei University Severance Hospital CCIDS LRP : Layer-wise Relevance Propagation Explaining Decisions of Neural Networks by LRP. Alexander Binder @ Deep Learning: Theory, Algorithms, and Applications. Berlin, June 2017
  • 44. Recent progress towards XAI at UC Berkeley Yonsei University Severance Hospital CCIDS Unmasking Clever Hans Predictors S Lapuschkin et al. Unmasking Clever Hans Predictors and Assessing What Machines Really Learn Images with a copyright watermark (a, c) are classified to “horse” class, but images without an added copyright watermark (b, d) aren’t.
  • 45. Recent progress towards XAI at UC Berkeley Yonsei University Severance Hospital CCIDS Understanding Learning Behaviour S Lapuschkin et al. Unmasking Clever Hans Predictors and Assessing What Machines Really Learn Model learns 1. Track the ball 2. Focus on paddle 3. Focus in the tunnel
  • 46. Yonsei University Severance Hospital CCIDS Learning a Generative Model from a Single Natural Image https://sites.google.com/berkeley.edu/iccv-2019-image-and-video-syn SinGAN Tamar Rott Shaham Google AI ICCV 2019 Best Paper
  • 50. SinGAN Yonsei University Severance Hospital CCIDS ▶ Same model ▶ No extra training SinGAN for single image animation https://www.youtube.com/watch?v=xk8bWLZk4DU&feature=youtu.be
  • 51. SinGAN Yonsei University Severance Hospital CCIDS Model
  • 52. SinGAN Yonsei University Severance Hospital CCIDS Summary - SinGAN : Single Image GAN - Controllable Generation - Image Manipulation Applications
  • 53. SinGAN Yonsei University Severance Hospital CCIDS MRI Toy Example Training Image
  • 54. SinGAN Yonsei University Severance Hospital CCIDS MRI Toy Example Training Image Generated High-Resolution Image