1. ICG
CVPR 2010 Tutorial
Semi-Supervised Learning in
Vision
A, Saffari, Ch. Leistner, H. Bischof
Inst. for Computer Graphics and Vision
Graz University of Technology
http://www.icg.tugraz.at/Members/Saffari/ssl-cvpr2010
Horst Bischof
Professor Horst Cerjak, 19.12.2005 SSL CVPR Tutorial
1

2. ICG
Typical Vision Tasks/Trends
Internet: Image Search/Classification Categorization
Huge Amounts of data, Huge labeling effort
Horst Bischof
Professor Horst Cerjak, 19.12.2005 SSL CVPR Tutorial
2

3. ICG
Typical Vision Tasks/Trends
Internet: Various Video and Image Databases
Huge Amounts of data, Partially/weakly labeled data
Horst Bischof
Professor Horst Cerjak, 19.12.2005 SSL CVPR Tutorial
3

4. ICG
Typical Vision Tasks/Trends
Object Categorization
Huge Amounts of data, Huge labeling effort
Horst Bischof
Professor Horst Cerjak, 19.12.2005 SSL CVPR Tutorial
4

5. ICG
Perona 2009 Visipedia
Horst Bischof
Professor Horst Cerjak, 19.12.2005 SSL CVPR Tutorial
5

6. ICG
Typical Vision Tasks/Trends
Surveillance: On-line data/Detection-Tracking
Huge Amounts of data, On-line processing, Scene
adaptation
Horst Bischof
Professor Horst Cerjak, 19.12.2005 SSL CVPR Tutorial
6

7. ICG
Typical Vision Tasks/Trends
Many other tasks:
• Tracking: On-line adaptation to object
• Interactive segmentation: Changing model on
the fly
• Interactive labeling: Suggestions as you label
• …..
Horst Bischof
Professor Horst Cerjak, 19.12.2005 SSL CVPR Tutorial
7

8. ICG
Requirements
Huge
datasets
Changing Unreliable
environme (partial)
nts labels
Horst Bischof
Professor Horst Cerjak, 19.12.2005 SSL CVPR Tutorial
8

9. ICG
Requirements on Learner
On-line
learning
Semi-
supervised Robustness
Learning
Horst Bischof
Professor Horst Cerjak, 19.12.2005 SSL CVPR Tutorial
9

10. ICG
Theory/Methods Covered
1. Semi-Supervised Learning
2. Self-Training
3. Generative Models
4. Margin Assumption
5. Cluster and Manifold Assumption
6. Multi-View Learning
7. Large-Scale, Multi-Class SSL and Online Learning
8. Transfer Learning, Domain Adaptation, and Weakly
Related Data
9. Multiple-Instance Learning
Horst Bischof
Professor Horst Cerjak, 19.12.2005 SSL CVPR Tutorial
10

11. ICG
Applications Covered
1. Object Detection
2. Categorization
3. Tracking
4. Activity Recognition
5. Segmentation
Using:
Boosting
Random Forests
Horst Bischof
Professor Horst Cerjak, 19.12.2005 SSL CVPR Tutorial
11

12. ICG
Learning Tasks
• Unsupervised learning
– Density estimation, Clustering, Dimensionality reduction.
• Semi-Supervised Clustering
– Clustering with pair-wise constraints: must-link, cannot-link.
• Semi-Supervised Classification and
Regression
• Supervised Learning
– Classification, Regression.
Horst Bischof
Professor Horst Cerjak, 19.12.2005 SSL CVPR Tutorial
12

13. ICG
Labels
How do we get labels?
Horst Bischof
Professor Horst Cerjak, 19.12.2005 SSL CVPR Tutorial
13

14. ICG
Semi-Supervised Learning
SSL is Supervised Learning...
Goal: Estimate P(y|x) from Labeled Data
Dl={ (xi,yi) }
p( x | y ) p( y )
p( y | x) =
P( x)
But: Additional Source tells about P(x)
(e.g., Unlabeled Data Du={xj})
The Interesting Case:
Horst Bischof
Professor Horst Cerjak, 19.12.2005 SSL CVPR Tutorial
14

15. ICG
Supervised learning
+ -
+ -
Maximum margin
Horst Bischof
Professor Horst Cerjak, 19.12.2005 SSL CVPR Tutorial
15

16. ICG
Can Unlabeled Data Help?
? ? ?
? ? ? ? ?
? ? ? ?
?
? +? ? -
? ? ? ?
? ?
? ? ?
? ? + ? ?- ?
? ? ?
?
low density ? ? ? ?
around
? ?
decision
boundary
? ?
Horst Bischof
Professor Horst Cerjak, 19.12.2005 SSL CVPR Tutorial
16

17. ICG
SSL is biologically plausible
• Co-training by infants [Bahrick et.al. 2002]
• Human change model once they see unlabeled data
[Zahki et.al. 2007,Zhu et.al. 2007,Vandist et.al. 2007]
• Humans do On-line SSL [Zhu ICML 2010]
Horst Bischof
Professor Horst Cerjak, 19.12.2005 SSL CVPR Tutorial
17

18. ICG
Why?
1. Unlabelled data is cheap/free
2. Labeled data is hard to get
• human annotation is boring
• labels may require experts
• labels may require special devices
• your graduate student is on vacation
Horst Bischof
Professor Horst Cerjak, 19.12.2005 SSL CVPR Tutorial
18

19. ICG
∞Computation
Online Learning in Perspective
Batch/Off-line
Online Incremental
0
0 ∞ Memory
Horst Bischof
Professor Horst Cerjak, 19.12.2005 SSL CVPR Tutorial
21

20. ICG
Why on-line learning?
Too much training data to fit in memory
– Internet!!!
Sample generation process
– Tracking, Co-Training
Changing processes
– Changing Environment
Horst Bischof
Professor Horst Cerjak, 19.12.2005 SSL CVPR Tutorial
22

21. ICG
Why on-line learning?
Specializing
– Forget irrelevant information
– Specialize to current scene
Interactive Applications
– Data labeling
– Classifier Training
– Specializing (Human in the loop)
– Interactive Training (Segmentation)
Horst Bischof
Professor Horst Cerjak, 19.12.2005 SSL CVPR Tutorial
23

22. ICG
Robustness in Learning
Noisy input data
Label noise
• Semi-supervised learning
• Weakly-labeled data
• Co-training
• Self-learning
• On-line learning
Horst Bischof
Professor Horst Cerjak, 19.12.2005 SSL CVPR Tutorial
24

23. ICG
Amir Saffari
Theory
Christian Leistner
Algorithms/Applications
Horst Bischof
Professor Horst Cerjak, 19.12.2005 SSL CVPR Tutorial
25