Review : A Probabilistic U-Net for Segmentation of Ambiguous Images - by Seunghyun Hwang (Yonsei University, Severance Hospital, Center for Clinical Data Science)

1. A Probabilistic U-Net for Segmentation of
Ambiguous Images
Hwang seung hyun
Yonsei University Severance Hospital CCIDS
DeepMind, Division of Medical Image Computing, German
Cancer Research Center, Germany | NIPS 2018
2020.04.19

2. Introduction Related Work Methods and
Experiments
01 02 03
Conclusion
04
Yonsei Unversity Severance Hospital CCIDS
Contents

3. Probabilistic Unet
Introduction – Limitations of prior methods
• There exist ambiguities in segmentation task, especially in medical imaging applications
• A lesion might be clearly visible, but ground truth labels can vary depending on
radiologists.
• Most existing segmentation algorithms either provide only consistent hypothesis of a
pixel-wise probability(e.g. “each pixel is 50% cat, 50% dog)
• Pixel wise probabilities ignores all co-variances between the pixels.
• Existing methods are Ensemble Unet, dropout Unet, M heads model, etc.
Introduction / Related Work / Methods and Experiments / Conclusion

4. Probabilistic Unet
Introduction – Probabilistic Unet Architecture
• Probabilistic Unet provides multiple segmentation hypotheses for ambiguous images.
• Combines conditional variational auto encoder(CVAE), and U-Net
• First extract latent space and encodes the possible segmentation variants
• Random sample from the space is injected into the Unet to produce segmentation map.
Introduction / Related Work / Methods and Experiments / Conclusion

5. Probabilistic Unet
Introduction – Contributions
• Provides consistent segmentation maps instead of pixel-wise probabilities,
providing joint likelihood of modes.
• Able to learn calibrated probabilities of segmentation modes.
• Can produce diverse outputs for single image
Introduction / Related Work / Methods and Experiments / Conclusion

6. Related Work
CVAE (Conditional Variational Auto Encoder)
Introduction / Related Work / Methods and Experiments / Conclusion
• Encoder를 통해 도출된 latent coding Z를 가우시
안 분포로 나타내기 위해 분산과 평균을 이용함
• Label 정보를 추가로 넣어준다

7. Related Work
U-Net
Introduction / Related Work / Methods and Experiments / Conclusion
• Encoding Phase

8. Methods and Experiments
Network Architecture
Introduction / Related Work / Methods and Experiments / Conclusion
• Sampling Process • Training Process

9. Methods and Experiments
Sampling Process
Introduction / Related Work / Methods and Experiments / Conclusion
• Prior Net (Unet’s encoding phase + global average
pooling) produces Latent Space
• Each position in this space encodes a
segmentation variant
• Broadcast the sample to feature map with the
same shape as the segmentation map, and
concatenate this map to the las activation map of
U-Net
* P : prior probability distribution
* fcomb = three subsequent 1x1 convolutions
* S: segmentation map corresponding to point z in latent space

10. Methods and Experiments
Training Process
Introduction / Related Work / Methods and Experiments / Conclusion
• Introduce Posterior Net that learns to recognize a
useful segmentation variant
• Posterior Net and Prior Net are updated through the
standard training procedure for CVAE, by minimizing
variational lower bound
(Kullback-Leibler divergence)
• Cross-entropy loss penalizes differences between S
and Y
• KL loss pulls the posterior distribution and prior
distribution towards each other
• Eventually covers the space of all useful segmentation
variants for input image
21

11. Methods and Experiments
Sampling Process
Introduction / Related Work / Methods and Experiments / Conclusion
Output Samples
Visualization of the Latent Space

12. Methods and Experiments
Introduction / Related Work / Methods and Experiments / Conclusion
Performance Measures
• Generalized Energy Distance Matrix
• Not only compare deterministic prediction, but also compares
distributions of segmentations
* d: distance measure
* Y, Y’ : Independent samples from the ground truth distribution
* S, S’: independent samples from the predicted distribution
* d(x,y) = 1 - IOU(x,y)

13. Methods and Experiments
Introduction / Related Work / Methods and Experiments / Conclusion
Results

14. Methods and Experiments
Introduction / Related Work / Methods and Experiments / Conclusion
Results
• Energy Distance decreases as more samples are drawn indicating an improved
matching of the GT distribution, as well as enhanced sample diversity.

15. Conclusion
Introduction / Related Work / Methods and Experiments / Conclusion
• Each sample produced by probabilistic Unet is consistent segmentation
result that closely match the multi-modal GT distributions
• Employed energy distance matrix measures whether the model’s
individual samples are both coherent, and whether they are produced
with expected frequencies.
• Can be used to assess annotations with model
• Probabilistic U Net can replace the currently applied deterministic U
Nets in large field of studies, especially in the medical domain
• Guide steps to resolve ambiguities