Methods that move towards less supervised scenarios are key for image segmentation, as dense labels demand significant human intervention. Generally, the annotation burden is mitigated by labeling datasets with weaker forms of supervision, e.g. image-level labels or bounding boxes. Another option are semi-supervised settings, that commonly leverage a few strong annotations and a huge number of unlabeled/weakly-labeled data. In this paper, we revisit semi-supervised segmentation schemes and narrow down significantly the annotation budget (in terms of total labeling time of the training set) compared to previous approaches. With a very simple pipeline, we demonstrate that at low annotation budgets, semi-supervised methods outperform by a wide margin weakly-supervised ones for both semantic and instance segmentation. Our approach also outperforms previous semi-supervised works at a much reduced labeling cost. We present results for the Pascal VOC benchmark and unify weakly and semi-supervised approaches by considering the total annotation budget, thus allowing a fairer comparison between methods. http://openaccess.thecvf.com/content_CVPRW_2019/html/Deep_Vision_Workshop/Bellver_Budget-aware_Semi-Supervised_Semantic_and_Instance_Segmentation_CVPRW_2019_paper.html

1. Budget-aware Semi-Supervised
Semantic and Instance Segmentation
Miriam Bellver, Amaia Salvador, Jordi Torres, Xavier Giro-i-Nieto
Women In Computer Vision - CVPR 2019

2. Motivation
Budget-aware Semi-Supervised Semantic and Instance Segmentation: Miriam Bellver, Amaia Salvador, Jordi Torres, Xavier Giro-i-Nieto
Semantic segmentation Instance segmentation
Pixel-level
annotations are
expensive!

3. Motivation
Budget-aware Semi-Supervised Semantic and Instance Segmentation: Miriam Bellver, Amaia Salvador, Jordi Torres, Xavier Giro-i-Nieto
Semantic segmentation Instance segmentation
Solution: Weakly and
Semi-supervised methods!

4. Motivation
Budget-aware Semi-Supervised Semantic and Instance Segmentation: Miriam Bellver, Amaia Salvador, Jordi Torres, Xavier Giro-i-Nieto
Semantic segmentation Instance segmentation
Solution: Weakly and
Semi-supervised methods!

5. Contributions
Budget-aware Semi-Supervised Semantic and Instance Segmentation: Miriam Bellver, Amaia Salvador, Jordi Torres, Xavier Giro-i-Nieto
1. We unify the segmentation benchmarks regardless the training setting and the supervision signals
comparing them in terms of the total annotation cost.

6. Contributions
Budget-aware Semi-Supervised Semantic and Instance Segmentation: Miriam Bellver, Amaia Salvador, Jordi Torres, Xavier Giro-i-Nieto
1. We unify the segmentation benchmarks regardless the training setting and the supervision signals
comparing them in terms of the total annotation cost.
2. We experiment with a semi-supervised pipeline and test it in Pascal VOC for semantic and instance
segmentation, outperforming previous works at low annotated budgets.

7. Contributions
Budget-aware Semi-Supervised Semantic and Instance Segmentation: Miriam Bellver, Amaia Salvador, Jordi Torres, Xavier Giro-i-Nieto
1. We unify the segmentation benchmarks regardless the training setting and the supervision signals
comparing them in terms of the total annotation cost.
2. We experiment with a semi-supervised pipeline and test it in Pascal VOC for semantic and instance
segmentation, outperforming previous works at low annotated budgets.
3. We show that for low annotation budgets, it’s more convenient having fewer but stronger-labeled data
over having larger weakly-annotated sets.

8. Budget-aware Semi-Supervised Semantic and Instance Segmentation: Miriam Bellver, Amaia Salvador, Jordi Torres, Xavier Giro-i-Nieto
Semi-Supervised PipelineSemi-Supervised Pipeline
Annotation
network

9. Budget-aware Semi-Supervised Semantic and Instance Segmentation: Miriam Bellver, Amaia Salvador, Jordi Torres, Xavier Giro-i-Nieto
Semi-Supervised PipelineSemi-Supervised Pipeline
Annotation
network
Segmentation
network

10. Experimental validation for Pascal VOC
Budget-aware Semi-Supervised Semantic and Instance Segmentation: Miriam Bellver, Amaia Salvador, Jordi Torres, Xavier Giro-i-Nieto
Semantic segmentation
Annotation cost
Segmentationquality
We used DeepLab v3+ for both the
annotation and segmentation network

11. Experimental validation for Pascal VOC
Budget-aware Semi-Supervised Semantic and Instance Segmentation: Miriam Bellver, Amaia Salvador, Jordi Torres, Xavier Giro-i-Nieto
Instance segmentation
Annotation cost
Segmentationquality
We used RSIS for both the annotation
and segmentation network

12. Comparison to other works
Budget-aware Semi-Supervised Semantic and Instance Segmentation: Miriam Bellver, Amaia Salvador, Jordi Torres, Xavier Giro-i-Nieto
Semantic segmentation Instance segmentation
Annotation cost
Segmentationquality

13. Visualization for semantic segmentation
Budget-aware Semi-Supervised Semantic and Instance Segmentation: Miriam Bellver, Amaia Salvador, Jordi Torres, Xavier Giro-i-Nieto

14. Visualization for instance segmentation
Budget-aware Semi-Supervised Semantic and Instance Segmentation: Miriam Bellver, Amaia Salvador, Jordi Torres, Xavier Giro-i-Nieto

15. Thanks for
your attention!