1. MICCAI 17 CVPR 18 MICCAI 18
Pancreas Segmentation
Kun Zhan
UTS Building 11
15 July, 2018
Kun Zhan Pancreas Segmentation
2. MICCAI 17 CVPR 18 MICCAI 18
Content I
1 MICCAI 17
A Fixed-Point Model for Pancreas Segmentation in
Abdominal CT Scans
2 CVPR 18
Recurrent Saliency Transformation Network: Incorporating
Multi-Stage Visual Cues for Small Organ Segmentation
3 MICCAI 18
Bridging the Gap Between 2D and 3D Organ Segmentation
with Volumetric Fusion Net
Kun Zhan Pancreas Segmentation
3. MICCAI 17 CVPR 18 MICCAI 18
Pancreatic Cancer
1 Pancreatic cancer is a major killer of human being
12 out of 100,000 people have pancreatic cancer
330,000 new cases globally in the year of 2012 (7th most)
2 Even in all types of cancer, it belongs to the most dangerous
and fatal ones
Extremely difficult to diagnose in its early stage
In the time of diagnosis, the cancer has often spread to other
organs
Most often, nothing can be done to cure the patients (5-year
survival rate: 7.7%)
The rate of diagnosis is almost the same as the rate of death
Kun Zhan Pancreas Segmentation
4. MICCAI 17 CVPR 18 MICCAI 18
The NIH Dataset
82 CT samples collected from healthy people
A volume is of size 512 × 512 × L, L ∈ [181, 466] varies from
sample to sample, and the slice thickness varies between
1.5mm and 2.5mm
Slice-by-slice (in the axial view) annotated by a medical
student, and verified (modified) by an experienced radiologist
Kun Zhan Pancreas Segmentation
5. MICCAI 17 CVPR 18 MICCAI 18
Occupy a very small fraction (e.g., < 0.5%) of the CT volume.
Kun Zhan Pancreas Segmentation
6. MICCAI 17 CVPR 18 MICCAI 18 A Fixed-Point Model for Pancreas Segmentation in Abdominal CT
Outline
1 MICCAI 17
A Fixed-Point Model for Pancreas Segmentation in
Abdominal CT Scans
2 CVPR 18
Recurrent Saliency Transformation Network: Incorporating
Multi-Stage Visual Cues for Small Organ Segmentation
3 MICCAI 18
Bridging the Gap Between 2D and 3D Organ Segmentation
with Volumetric Fusion Net
Kun Zhan Pancreas Segmentation
7. MICCAI 17 CVPR 18 MICCAI 18 A Fixed-Point Model for Pancreas Segmentation in Abdominal CT
Outline
1 MICCAI 17
A Fixed-Point Model for Pancreas Segmentation in
Abdominal CT Scans
2 CVPR 18
Recurrent Saliency Transformation Network: Incorporating
Multi-Stage Visual Cues for Small Organ Segmentation
3 MICCAI 18
Bridging the Gap Between 2D and 3D Organ Segmentation
with Volumetric Fusion Net
Kun Zhan Pancreas Segmentation
8. MICCAI 17 CVPR 18 MICCAI 18 A Fixed-Point Model for Pancreas Segmentation in Abdominal CT
Model. (pre-trained “fcn8s-heavy-pascal.caffemodel”)
VGG-16 FCN-8s
Kun Zhan Pancreas Segmentation
9. MICCAI 17 CVPR 18 MICCAI 18 A Fixed-Point Model for Pancreas Segmentation in Abdominal CT
Loss Function
DSC Loss Function:
Dice Sφrensen Coeffcient = 1 −
2× i ziyi
i zi+ i yi
Loss Function:
L = 1 −
2× i ziyi+
i zi+ i yi+
The gradient computation is straightforward:
∂L
∂zj
= −
2yj( i zi+ i yi+ )−2 i ziyi−
( i zi+ i yi+ )2
Kun Zhan Pancreas Segmentation
10. MICCAI 17 CVPR 18 MICCAI 18 A Fixed-Point Model for Pancreas Segmentation in Abdominal CT
Segmentation results with different input regions
Red, green, and yellow indicate the prediction, ground-truth, and
overlapped pixels, respectively.
Kun Zhan Pancreas Segmentation
11. MICCAI 17 CVPR 18 MICCAI 18 A Fixed-Point Model for Pancreas Segmentation in Abdominal CT
Scheme
Illustration of the testing process. Only one iteration is shown
here. In practice, there are at most 10 iterations.
Kun Zhan Pancreas Segmentation
12. MICCAI 17 CVPR 18 MICCAI 18 A Fixed-Point Model for Pancreas Segmentation in Abdominal CT
The floodfill algorithm is used to detect the connecting components.
Kun Zhan Pancreas Segmentation
13. MICCAI 17 CVPR 18 MICCAI 18 A Fixed-Point Model for Pancreas Segmentation in Abdominal CT
Results
Segmentation accuracy reported by different approaches.
Kun Zhan Pancreas Segmentation
14. MICCAI 17 CVPR 18 MICCAI 18 Recurrent Saliency Transformation Network: Incorporating Multi-
Outline
1 MICCAI 17
A Fixed-Point Model for Pancreas Segmentation in
Abdominal CT Scans
2 CVPR 18
Recurrent Saliency Transformation Network: Incorporating
Multi-Stage Visual Cues for Small Organ Segmentation
3 MICCAI 18
Bridging the Gap Between 2D and 3D Organ Segmentation
with Volumetric Fusion Net
Kun Zhan Pancreas Segmentation
15. MICCAI 17 CVPR 18 MICCAI 18 Recurrent Saliency Transformation Network: Incorporating Multi-
Outline
1 MICCAI 17
A Fixed-Point Model for Pancreas Segmentation in
Abdominal CT Scans
2 CVPR 18
Recurrent Saliency Transformation Network: Incorporating
Multi-Stage Visual Cues for Small Organ Segmentation
3 MICCAI 18
Bridging the Gap Between 2D and 3D Organ Segmentation
with Volumetric Fusion Net
Kun Zhan Pancreas Segmentation
16. MICCAI 17 CVPR 18 MICCAI 18 Recurrent Saliency Transformation Network: Incorporating Multi-
Motivation
Since “MICCAI 2017” is inconsistency between its training and
testing strategies, we propose a Recurrent Saliency Transformation
Network with two advantages:
1 In the training phase, the coarse-scaled and fine-scaled
networks are optimized jointly, so that the segmentation
ability of each of them gets improved.
2 In the testing phase, the segmentation mask of each iteration
is preserved and propagated throughout iterations, enabling
multi-stage visual cues to be incorporated towards more
accurate segmentation.
‘To the best of our knowledge, this idea was not studied in the
computer vision community, as it requires making use of some
special properties of CT scans.’
Kun Zhan Pancreas Segmentation
17. MICCAI 17 CVPR 18 MICCAI 18 Recurrent Saliency Transformation Network: Incorporating Multi-
A failure case of “MICCAI 2017”
Kun Zhan Pancreas Segmentation
21. MICCAI 17 CVPR 18 MICCAI 18 Bridging the Gap Between 2D and 3D Organ Segmentation with
Outline
1 MICCAI 17
A Fixed-Point Model for Pancreas Segmentation in
Abdominal CT Scans
2 CVPR 18
Recurrent Saliency Transformation Network: Incorporating
Multi-Stage Visual Cues for Small Organ Segmentation
3 MICCAI 18
Bridging the Gap Between 2D and 3D Organ Segmentation
with Volumetric Fusion Net
Kun Zhan Pancreas Segmentation
22. MICCAI 17 CVPR 18 MICCAI 18 Bridging the Gap Between 2D and 3D Organ Segmentation with
Outline
1 MICCAI 17
A Fixed-Point Model for Pancreas Segmentation in
Abdominal CT Scans
2 CVPR 18
Recurrent Saliency Transformation Network: Incorporating
Multi-Stage Visual Cues for Small Organ Segmentation
3 MICCAI 18
Bridging the Gap Between 2D and 3D Organ Segmentation
with Volumetric Fusion Net
Kun Zhan Pancreas Segmentation
23. MICCAI 17 CVPR 18 MICCAI 18 Bridging the Gap Between 2D and 3D Organ Segmentation with
Highlights
1 We present an alternative framework, which trains 2D
networks on different view points for segmentation, and builds
a 3D Volumetric Fusion Net (VFN) to fuse the 2D
segmentation results.
2 We train and test the segmentation and fusion modules
individually, and propose a novel strategy, named
cross-cross-augmentation, to make full use of the limited
training data.
cross-cross-augmentation: Suppose we split data into K folds
for cross-validation, and the k1-th fold is left for testing. For all
k2 = k1, we train 2D segmentation models on the folds in
{1, . . . , K} {k1, k2}, and test on the k2-th fold to generate
training data for the VFN.
Kun Zhan Pancreas Segmentation
24. MICCAI 17 CVPR 18 MICCAI 18 Bridging the Gap Between 2D and 3D Organ Segmentation with
VFN Scheme
It has three down-sampling stages and three up-sampling stages.
Each down-sampling stage is composed of two 3 × 3 × 3
convolutional layers and a 2 × 2 × 2 max-pooling layer with a stride
of 2, and each up-sampling stage is implemented by a single
4 × 4 × 4 deconvolutional layer with a stride of 2.
Kun Zhan Pancreas Segmentation
25. MICCAI 17 CVPR 18 MICCAI 18 Bridging the Gap Between 2D and 3D Organ Segmentation with
Results
Comparison of segmentation accuracy
Kun Zhan Pancreas Segmentation
