call girls in Kamla Market (DELHI) 🔝 >༒9953330565🔝 genuine Escort Service 🔝✔️✔️
CVPR2010: Semi-supervised Learning in Vision: Part 3: Algorithms and Applications
1. Semi-Supervised Learning in Computer Vision
Part II
Amir Saffari,Christian Leistner,Horst Bischof
Institute for Computer Graphics and Vision, Graz University of Technology
June 18th, 2010
2. SemiBoost & Visual Similarity Learning On-line Semi-supervised Boosting Semi-Supervised Random Forests On-line Manifo
Outline
1 SemiBoost & Visual Similarity Learning
2 On-line Semi-supervised Boosting
Tracking
3 Semi-Supervised Random Forests
MILForests
On-line Random Forests
4 On-line Manifold Regularization
5 Conclusion & Outlook
Graz University of Technology
Amir Saffari,Christian Leistner,Horst Bischof Semi-Supervised Learning in Computer Vision Part II
3. SemiBoost & Visual Similarity Learning On-line Semi-supervised Boosting Semi-Supervised Random Forests On-line Manifo
SemiBoost
[Mallapragada et al.,PAMI’09] [Leistner et al.,CVPR’08]
Loss function
(x,y)∈XL e −yF (x) +
F (x)
λu s(x, x ) cosh(F (x) − F (x )) + λl s(x, x )e −2y
x∈XU x ∈XU (x ,y )∈XL
Optimization Problem
arg min = s(x, x )e −2y(F (x )+αf (x ))
f (x),α x ∈XU (x,y)∈XL
+λu s(x, x )e ((F (x )−F (x)) e α(f (x)−f (x ))
x ∈XU Graz University of Technology
Amir Saffari,Christian Leistner,Horst Bischof Semi-Supervised Learning in Computer Vision Part II
4. SemiBoost & Visual Similarity Learning On-line Semi-supervised Boosting Semi-Supervised Random Forests On-line Manifo
SemiBoost
λu
px = λl I(y = 1)s(x, x )e −2F (x ) + s(x, x )e F (x )−F (x)
2
(x ,y )∈XL x∈XU
and
λu
qx = λl I(y = −1)s(x, x )e −2F (x ) + s(x, x )e F (x)−F (x )
2
(x ,y )∈XL x∈XU
Graz University of Technology
Amir Saffari,Christian Leistner,Horst Bischof Semi-Supervised Learning in Computer Vision Part II
5. SemiBoost & Visual Similarity Learning On-line Semi-supervised Boosting Semi-Supervised Random Forests On-line Manifo
SemiBoost
Pseudo Labels and Weights
ˆ
yx = sign(px − qx )
wx = |px − qx |
Optimal α
1 x∈XU pi I(f (x) = 1) + qi I(f (x) = −1)
α= ln
4 x∈XU pi I(f (x) = −1) + qi I(f (x) = 1)
Graz University of Technology
Amir Saffari,Christian Leistner,Horst Bischof Semi-Supervised Learning in Computer Vision Part II
6. SemiBoost & Visual Similarity Learning On-line Semi-supervised Boosting Semi-Supervised Random Forests On-line Manifo
SemiBoost
labeled training data (x, y) ∈ XL and unlabeled data x ∈ XU
Similarity measure s(x, x )
Weak learners fi
weight parameters λu , λl
max iterations T
1 For t = 1, 2, . . . , T
2 Compute pi and qi for every given sample
3 ˆ
yx = sign(px − qx )
4 wx = |px − qx |
5 Train weak classifier ft (x)
6 Compute αt
7 F (x) ← F (x) + αt ft (x)
8 EndFor Graz University of Technology
Amir Saffari,Christian Leistner,Horst Bischof Semi-Supervised Learning in Computer Vision Part II
7. SemiBoost & Visual Similarity Learning On-line Semi-supervised Boosting Semi-Supervised Random Forests On-line Manifo
SemiBoost with learned Similarities
[Hertz et al.,CVPR’04]
Radial Basis Function [Zhu et al.,ICML’03]
d(x,x )2
−
σ2
s(x, x ) = e
d(x, x ) . . . distance between points
Graz University of Technology
Amir Saffari,Christian Leistner,Horst Bischof Semi-Supervised Learning in Computer Vision Part II
8. SemiBoost & Visual Similarity Learning On-line Semi-supervised Boosting Semi-Supervised Random Forests On-line Manifo
Learning Distance Functions
Idea
Learn distance or metric function on labeled data which then
can discriminatively support task-specific classification.
Distance Function
F d : X × X → Y = [−1 1]
Training Pairs of “same” or “different” [Hertz et al.,CVPR’04]
Dd = {(x, x , +1)|y = y , x, x ∈ DL } ∪
∪{(x, x , −1)|y y , x, x ∈ DL }
Graz University of Technology
Amir Saffari,Christian Leistner,Horst Bischof Semi-Supervised Learning in Computer Vision Part II
9. SemiBoost & Visual Similarity Learning On-line Semi-supervised Boosting Semi-Supervised Random Forests On-line Manifo
SemiBoost with learned Distance Functions
Number of Training Pairs (Symmetric case)
n·(n−1)
2
?
+- ? +
?
+ - SemiBoost
?
+ - ?
Graz University of Technology
Amir Saffari,Christian Leistner,Horst Bischof Semi-Supervised Learning in Computer Vision Part II
10. SemiBoost & Visual Similarity Learning On-line Semi-supervised Boosting Semi-Supervised Random Forests On-line Manifo
Using Arbitrary Classifiers
Approximate pair-wise classifier
|F (x, x )| ≈ |F (x) − F (x )|
+
+ ?
?
-
Graz University of Technology
Amir Saffari,Christian Leistner,Horst Bischof Semi-Supervised Learning in Computer Vision Part II
11. SemiBoost & Visual Similarity Learning On-line Semi-supervised Boosting Semi-Supervised Random Forests On-line Manifo
Reusing Prior Classifiers
[Schapire et al,ML’02]
Classifier Combination
F C (x) = α0 F P (x) + F (x)
?
? +
SemiBoost ?
?
- ?
Graz University of Technology
Amir Saffari,Christian Leistner,Horst Bischof Semi-Supervised Learning in Computer Vision Part II
12. SemiBoost & Visual Similarity Learning On-line Semi-supervised Boosting Semi-Supervised Random Forests On-line Manifo
SemiBoost Applications
Car Detection
Graz University of Technology
Amir Saffari,Christian Leistner,Horst Bischof Semi-Supervised Learning in Computer Vision Part II
13. SemiBoost & Visual Similarity Learning On-line Semi-supervised Boosting Semi-Supervised Random Forests On-line Manifo
Similarity Performance
Accuracy depending on the number of samples
Graz University of Technology
Amir Saffari,Christian Leistner,Horst Bischof Semi-Supervised Learning in Computer Vision Part II
14. SemiBoost & Visual Similarity Learning On-line Semi-supervised Boosting Semi-Supervised Random Forests On-line Manifo
SemiBoost Applications
Car Detection
Graz University of Technology
Amir Saffari,Christian Leistner,Horst Bischof Semi-Supervised Learning in Computer Vision Part II
15. SemiBoost & Visual Similarity Learning On-line Semi-supervised Boosting Semi-Supervised Random Forests On-line Manifo
SemiBoost Applications
Face Detection
(a) prior (b) trained (c) combined
Graz University of Technology
Amir Saffari,Christian Leistner,Horst Bischof Semi-Supervised Learning in Computer Vision Part II
16. SemiBoost & Visual Similarity Learning On-line Semi-supervised Boosting Semi-Supervised Random Forests On-line Manifo
Simple Data mining method
[Levin et al.,ICCV’03][Rosenberg et al.,2005]
1 Labeled training data (x, y) ∈ XL
2 Train cascaded detector F P (x) on XL using [Viola & Jones,2001]
3 Use a web image search engine in order to collect huge
amounts of possibly useful images XU ; pass phrases that are
much likely related to your target object
4 Apply F P (x) in a sliding window manner on XU and copy all
∗
detections to XU
∗
5 Train a SemiBoost classifier F (x) on XL and XU using F P (x)
as prior
6 Output the final classifier F (x)
Graz University of Technology
Amir Saffari,Christian Leistner,Horst Bischof Semi-Supervised Learning in Computer Vision Part II
17. SemiBoost & Visual Similarity Learning On-line Semi-supervised Boosting Semi-Supervised Random Forests On-line Manifo
SemiBoost Applications
Transfer Learning
Graz University of Technology
Amir Saffari,Christian Leistner,Horst Bischof Semi-Supervised Learning in Computer Vision Part II
18. SemiBoost & Visual Similarity Learning On-line Semi-supervised Boosting Semi-Supervised Random Forests On-line Manifo
SemiBoost Applications
Transfer Learning
Graz University of Technology
Amir Saffari,Christian Leistner,Horst Bischof Semi-Supervised Learning in Computer Vision Part II
19. SemiBoost & Visual Similarity Learning On-line Semi-supervised Boosting Semi-Supervised Random Forests On-line Manifo
Outline
1 SemiBoost & Visual Similarity Learning
2 On-line Semi-supervised Boosting
Tracking
3 Semi-Supervised Random Forests
MILForests
On-line Random Forests
4 On-line Manifold Regularization
5 Conclusion & Outlook
Graz University of Technology
Amir Saffari,Christian Leistner,Horst Bischof Semi-Supervised Learning in Computer Vision Part II
20. SemiBoost & Visual Similarity Learning On-line Semi-supervised Boosting Semi-Supervised Random Forests On-line Manifo
On-line Boosting
Graz University of Technology
Amir Saffari,Christian Leistner,Horst Bischof Semi-Supervised Learning in Computer Vision Part II
21. SemiBoost & Visual Similarity Learning On-line Semi-supervised Boosting Semi-Supervised Random Forests On-line Manifo
Tracking
[Oza,PhD-Thesis’01], [Grabner & Bischof,CVPR’06]
Graz University of Technology
Amir Saffari,Christian Leistner,Horst Bischof Semi-Supervised Learning in Computer Vision Part II
22. SemiBoost & Visual Similarity Learning On-line Semi-supervised Boosting Semi-Supervised Random Forests On-line Manifo
Tracking
Graz University of Technology
Amir Saffari,Christian Leistner,Horst Bischof Semi-Supervised Learning in Computer Vision Part II
23. SemiBoost & Visual Similarity Learning On-line Semi-supervised Boosting Semi-Supervised Random Forests On-line Manifo
Tracking
Graz University of Technology
Amir Saffari,Christian Leistner,Horst Bischof Semi-Supervised Learning in Computer Vision Part II
24. SemiBoost & Visual Similarity Learning On-line Semi-supervised Boosting Semi-Supervised Random Forests On-line Manifo
Tracking
Graz University of Technology
Amir Saffari,Christian Leistner,Horst Bischof Semi-Supervised Learning in Computer Vision Part II
25. SemiBoost & Visual Similarity Learning On-line Semi-supervised Boosting Semi-Supervised Random Forests On-line Manifo
Tracking
Tracking is an One-Shot Semi-supervised Learning Problem
Graz University of Technology
Amir Saffari,Christian Leistner,Horst Bischof Semi-Supervised Learning in Computer Vision Part II
26. SemiBoost & Visual Similarity Learning On-line Semi-supervised Boosting Semi-Supervised Random Forests On-line Manifo
On-line SemiBoost
P
−Fn−1 (x) −Fn−1 (x) + e −Fn−1 (x) e F (x)
˜
px ≈ e S(x, xi ) ≈ e F (x) ≈ F P (x) P
xi ∈X+
e + e −F (x)
P
e Fn−1 (x) e −F (x)
qx ≈ e Fn−1 (x)
˜ S(x, xi ) ≈ e Fn−1 (x) F − (x) ≈ P P
xi ∈X−
e F (x) + e −F (x)
sinh(F P (x) − Fn−1 )
˜ ˜
pn (x)−qn (x) = = tanh(F P (x))−tanh(Fn−1 (x))
cosh(F P (x))
Graz University of Technology
Amir Saffari,Christian Leistner,Horst Bischof Semi-Supervised Learning in Computer Vision Part II
27. SemiBoost & Visual Similarity Learning On-line Semi-supervised Boosting Semi-Supervised Random Forests On-line Manifo
Tracking
Graz University of Technology
Amir Saffari,Christian Leistner,Horst Bischof Semi-Supervised Learning in Computer Vision Part II
28. SemiBoost & Visual Similarity Learning On-line Semi-supervised Boosting Semi-Supervised Random Forests On-line Manifo
Tracking
Graz University of Technology
Amir Saffari,Christian Leistner,Horst Bischof Semi-Supervised Learning in Computer Vision Part II
29. SemiBoost & Visual Similarity Learning On-line Semi-supervised Boosting Semi-Supervised Random Forests On-line Manifo
Tracking
Problem: Rapid Appearance Changes
Graz University of Technology
Amir Saffari,Christian Leistner,Horst Bischof Semi-Supervised Learning in Computer Vision Part II
30. SemiBoost & Visual Similarity Learning On-line Semi-supervised Boosting Semi-Supervised Random Forests On-line Manifo
Tracking
Graz University of Technology
Amir Saffari,Christian Leistner,Horst Bischof Semi-Supervised Learning in Computer Vision Part II
31. SemiBoost & Visual Similarity Learning On-line Semi-supervised Boosting Semi-Supervised Random Forests On-line Manifo
Exploration-Exploitation Dilemma
Convex Trade-off
(F (x)) = (1 − α) l (F (x)) + α u (F (x))
We need more Robustness when minimizing the labeled loss!
Graz University of Technology
Amir Saffari,Christian Leistner,Horst Bischof Semi-Supervised Learning in Computer Vision Part II
32. SemiBoost & Visual Similarity Learning On-line Semi-supervised Boosting Semi-Supervised Random Forests On-line Manifo
Loss Functions
Random classification noise defeats all convex potential boosters
[Long and Servidio,ICML’08]
Graz University of Technology
Amir Saffari,Christian Leistner,Horst Bischof Semi-Supervised Learning in Computer Vision Part II
33. SemiBoost & Visual Similarity Learning On-line Semi-supervised Boosting Semi-Supervised Random Forests On-line Manifo
On-line Gradient Boost
Gradient Descent Functional Gradient Descent
GradientBoost [Friedman et al.,Annals of Statistics’01]
ft (x) = arg max − LT f (x)
f (x)
Graz University of Technology
Amir Saffari,Christian Leistner,Horst Bischof Semi-Supervised Learning in Computer Vision Part II
34. SemiBoost & Visual Similarity Learning On-line Semi-supervised Boosting Semi-Supervised Random Forests On-line Manifo
On-line Gradient Boost
A training sample: (xn , yn ), A differentiable loss function (·)
Number of selectors M , Number of weak learners per selector K
1 Set F0 (xn ) = 0.
2 Set the initial weight wn = − (0).
3 For m = 1 to M
4 For k = 1 to K
5 Train k th weak learner fm (x) with sample (xn , yn ) and weight wn .
k
k k k
6 em ← em + wn I(sign(fm (xn )) yn ) //Compute the error
7 EndFor
8 Find the best weak learner with the least total weighted error:
k
j = arg min em .
k
j
9 Set fm (xn ) = fm (xn ).
10 Set Fm (xn ) = Fm−1 (xn ) + fm (xn ).
11 Set the weight wn = − (yn Fm (xn )).
12 EndFor Graz University of Technology
13
Amir Saffari,Christian Leistner,Horst Bischof Semi-Supervised Learning in Computer Vision Part II
35. SemiBoost & Visual Similarity Learning On-line Semi-supervised Boosting Semi-Supervised Random Forests On-line Manifo
Weight Updates
Graz University of Technology
Amir Saffari,Christian Leistner,Horst Bischof Semi-Supervised Learning in Computer Vision Part II
36. SemiBoost & Visual Similarity Learning On-line Semi-supervised Boosting Semi-Supervised Random Forests On-line Manifo
Co-Training of Pedestrian Detectors
Exponential Loss Logit Loss
Graz University of Technology
Amir Saffari,Christian Leistner,Horst Bischof Semi-Supervised Learning in Computer Vision Part II
37. SemiBoost & Visual Similarity Learning On-line Semi-supervised Boosting Semi-Supervised Random Forests On-line Manifo
SERBoost
Expectation Regularization [Mann and MacCallum,ICML’07]
Penalize model predictions on unlabeled data that deviate from
certain expectation.
SERBoost [Saffari et al.,ECCV’08]
L(H (x), X) = Ll (H (x), Xl ) + βLu (H (x), Xu )
L(H (x), X) = e −yH (x) + e −yp H (x) cosh(H (x))
x∈XL x∈XU
Pseudo Label
+
yp = 2Pp (x) − 1
Graz University of Technology
Amir Saffari,Christian Leistner,Horst Bischof Semi-Supervised Learning in Computer Vision Part II
38. SemiBoost & Visual Similarity Learning On-line Semi-supervised Boosting Semi-Supervised Random Forests On-line Manifo
On-line SERBoost with logistic loss
Supervised Loss
Ll (XL ) = log 1 + e −2yF (x)
(x,y)∈Xl
= log e −yF (x) (e yF (x) + e −yF (x) )
(x,y)∈XL
= −yF (x) + log e F (x) + e −F (x) .
(x,y)∈XL
Graz University of Technology
Amir Saffari,Christian Leistner,Horst Bischof Semi-Supervised Learning in Computer Vision Part II
39. SemiBoost & Visual Similarity Learning On-line Semi-supervised Boosting Semi-Supervised Random Forests On-line Manifo
On-line SERBoost with logistic loss
Minimize the cross entropy
H (Pp , P) = − Pp (y = z|x) log P(y = z|x)
z∈{−1,1}
= − 2Pp (y = 1|x) − 1 F (x) + log e F (x) + e −F (x)
yp (x)
Graz University of Technology
Amir Saffari,Christian Leistner,Horst Bischof Semi-Supervised Learning in Computer Vision Part II
40. SemiBoost & Visual Similarity Learning On-line Semi-supervised Boosting Semi-Supervised Random Forests On-line Manifo
On-line SERBoost with logistic loss
Unsupervised Loss
Lu (XU ) = ˆ
H (Pp , P) = −yp (x)F (x) + log e F (x) + e −F (x)
x∈XU x∈XU
Graz University of Technology
Amir Saffari,Christian Leistner,Horst Bischof Semi-Supervised Learning in Computer Vision Part II
41. SemiBoost & Visual Similarity Learning On-line Semi-supervised Boosting Semi-Supervised Random Forests On-line Manifo
On-line SERBoost with logistic loss
Unsupervised Loss
Lu (XU ) = ˆ
H (Pp , P) = −yp (x)F (x) + log e F (x) + e −F (x)
x∈XU x∈XU
Unlabeled Update
∀ x ∈ XU :wx = yp (x) − tanh(F (x))
ˆ
yx = sign yp (x) − tanh(F (x))
Graz University of Technology
Amir Saffari,Christian Leistner,Horst Bischof Semi-Supervised Learning in Computer Vision Part II
42. SemiBoost & Visual Similarity Learning On-line Semi-supervised Boosting Semi-Supervised Random Forests On-line Manifo
OSER Tracking
λ = 0.5
Graz University of Technology
Amir Saffari,Christian Leistner,Horst Bischof Semi-Supervised Learning in Computer Vision Part II
43. SemiBoost & Visual Similarity Learning On-line Semi-supervised Boosting Semi-Supervised Random Forests On-line Manifo
Influence of convex combination
Graz University of Technology
Amir Saffari,Christian Leistner,Horst Bischof Semi-Supervised Learning in Computer Vision Part II
44. SemiBoost & Visual Similarity Learning On-line Semi-supervised Boosting Semi-Supervised Random Forests On-line Manifo
Multiple Instance Boosting
[Viola et al.,NIPS’05][Babenko et al.,CVPR’09]
Graz University of Technology
Amir Saffari,Christian Leistner,Horst Bischof Semi-Supervised Learning in Computer Vision Part II
45. SemiBoost & Visual Similarity Learning On-line Semi-supervised Boosting Semi-Supervised Random Forests On-line Manifo
Multiple Instance Boosting
[Viola et al.,NIPS’05][Babenko et al.,CVPR’09]
Bags
{(B1 , y1 ), . . . , (Bn , yn )}
Bi = {xi1 , xi2 , . . . , xini }
Minimize binary log-likelihood
log L= (yi log p(yi ) + (1 − yi ) log p(yi ))
i
Graz University of Technology
Amir Saffari,Christian Leistner,Horst Bischof Semi-Supervised Learning in Computer Vision Part II
46. SemiBoost & Visual Similarity Learning On-line Semi-supervised Boosting Semi-Supervised Random Forests On-line Manifo
Semi-Supervised Multiple Instance Boosting
[Zeisl et al.,CVPR’10]
Combine benefits of MIL and SSL
Graz University of Technology
Amir Saffari,Christian Leistner,Horst Bischof Semi-Supervised Learning in Computer Vision Part II
47. SemiBoost & Visual Similarity Learning On-line Semi-supervised Boosting Semi-Supervised Random Forests On-line Manifo
Semi-Supervised Multiple Instance Boosting
[Zeisl et al.,CVPR’10]
Unlabeled Loss of the Bags
Nu
Lu (XB ) = −
u Pp (z|Bu ) log(P(z|Bu ))
i i
i=1 z∈Y
Approximate max with geometric mean
NBi
1/NBi
P(y = 1|Bi ) = 1 − 1 − P(y = 1|xij )
j=1
Graz University of Technology
Amir Saffari,Christian Leistner,Horst Bischof Semi-Supervised Learning in Computer Vision Part II
48. SemiBoost & Visual Similarity Learning On-line Semi-supervised Boosting Semi-Supervised Random Forests On-line Manifo
Semi-Supervised Multiple Instance Boosting
[Zeisl et al.,CVPR’10]
Gradient for NOR and geometric mean
2 z − P(y = 1|Bi )
aij (z) = P(y = 1|xij )
NBi P(y = 1|Bi )
Pseudo Labels and Weights
wij =β Pp (z|Bu )aij (z)
i
z∈Y
yij =I β Pp (z|Bu )aij (z) > 0
i
z∈Y
Graz University of Technology
Amir Saffari,Christian Leistner,Horst Bischof Semi-Supervised Learning in Computer Vision Part II
49. SemiBoost & Visual Similarity Learning On-line Semi-supervised Boosting Semi-Supervised Random Forests On-line Manifo
Semi-Supervised Multiple Instance Boosting
[Zeisl et al.,CVPR’10]
Experimental Results
Sequence MILSER MIL OSB OAB
sylv 0.64 0.61 0.46 0.50
david 0.71 0.54 0.31 0.32
faceocc2 0.78 0.65 0.63 0.64
coke11 0.18 0.29 0.12 0.20
tiger1 0.60 0.51 0.17 0.27
tiger2 0.46 0.50 0.08 0.25
faceocc1 0.68 0.63 0.71 0.47
girl 0.64 0.53 0.69 0.38
Graz University of Technology
Amir Saffari,Christian Leistner,Horst Bischof Semi-Supervised Learning in Computer Vision Part II
50. SemiBoost & Visual Similarity Learning On-line Semi-supervised Boosting Semi-Supervised Random Forests On-line Manifo
On-line Co-Training
[Liu et al.,ICCV’09][Saffari et al.,ECCV’10]
Performance measured in average location center errors in pixels
Approach sylv david faceocc2 tiger1 tiger2 coke faceocc1 girl
MV-GPBoost 17 20 10 15 16 20 12 15
CoBoost 15 33 11 22 19 14 13 17
SemiBoost 22 59 43 46 53 85 41 52
MILBoost 11 23 20 15 17 21 27 32
Graz University of Technology
Amir Saffari,Christian Leistner,Horst Bischof Semi-Supervised Learning in Computer Vision Part II
51. SemiBoost & Visual Similarity Learning On-line Semi-supervised Boosting Semi-Supervised Random Forests On-line Manifo
End Part I
Graz University of Technology
Amir Saffari,Christian Leistner,Horst Bischof Semi-Supervised Learning in Computer Vision Part II
52. SemiBoost & Visual Similarity Learning On-line Semi-supervised Boosting Semi-Supervised Random Forests On-line Manifo
Random Forests
[Breiman,ML’01]
Ensemble of n decision trees
N
F (x) = n=1 f (x)
Information Gain
|Il | |Ir |
∆H = − |I |+|Ir | H (Il ) − |Il |+|Ir | H (Ir )
l
Graz University of Technology
Amir Saffari,Christian Leistner,Horst Bischof Semi-Supervised Learning in Computer Vision Part II
53. SemiBoost & Visual Similarity Learning On-line Semi-supervised Boosting Semi-Supervised Random Forests On-line Manifo
Random Forests
Advantages:
speed
parallelism
noise robust
inherently multi-class
Graz University of Technology
Amir Saffari,Christian Leistner,Horst Bischof Semi-Supervised Learning in Computer Vision Part II
54. SemiBoost & Visual Similarity Learning On-line Semi-supervised Boosting Semi-Supervised Random Forests On-line Manifo
Random Forests
Advantages:
speed
parallelism
noise robust
inherently multi-class
Applications:
Object Detection, Semantic Segmentation, Categorization,
Tracking, etc.
Graz University of Technology
Amir Saffari,Christian Leistner,Horst Bischof Semi-Supervised Learning in Computer Vision Part II
55. SemiBoost & Visual Similarity Learning On-line Semi-supervised Boosting Semi-Supervised Random Forests On-line Manifo
Random Forests
Advantages:
speed
parallelism
noise robust
inherently multi-class
Applications:
Object Detection, Semantic Segmentation, Categorization,
Tracking, etc.
Disadvantage:
RFs demand a huge amount of data in order to leverage their
full potential [Caruana et al.,ICML’08]
Graz University of Technology
Amir Saffari,Christian Leistner,Horst Bischof Semi-Supervised Learning in Computer Vision Part II
56. SemiBoost & Visual Similarity Learning On-line Semi-supervised Boosting Semi-Supervised Random Forests On-line Manifo
Semi-Supervised Random Forests
Random Forests maximize the margin
ml (x, y) = p(y|x) − max p(k|x)
k∈Y
k y
Unlabeled Margin
mu (xu ) = max fi (xu )
i∈Y
Semi-supervised Loss
1 λ
L(f) = (fy (x)) + (mu (x))
|Xl | |Xu |
(x,y)∈Xl x∈Xu
Graz University of Technology
Amir Saffari,Christian Leistner,Horst Bischof Semi-Supervised Learning in Computer Vision Part II
57. SemiBoost & Visual Similarity Learning On-line Semi-supervised Boosting Semi-Supervised Random Forests On-line Manifo
Optimization
Incorporate labels for the unlabeled data as additional
optimization variables!
Graz University of Technology
Amir Saffari,Christian Leistner,Horst Bischof Semi-Supervised Learning in Computer Vision Part II
58. SemiBoost & Visual Similarity Learning On-line Semi-supervised Boosting Semi-Supervised Random Forests On-line Manifo
Optimization
Incorporate labels for the unlabeled data as additional
optimization variables!
Deterministic Annealing [Rose,IJCNN’98]
p ∗ = arg minEp (F(y)) − T H(p)
p∈P
T0 > T1 > . . . > T∞ = 0
p ∗ . . . distributions over the label predictions
Graz University of Technology
Amir Saffari,Christian Leistner,Horst Bischof Semi-Supervised Learning in Computer Vision Part II
59. SemiBoost & Visual Similarity Learning On-line Semi-supervised Boosting Semi-Supervised Random Forests On-line Manifo
Optimization
DA-Loss for Semi-supervised Random Forests
1
LDA (f, p) =
ˆ (fy (x))+
|Xl |
(x,y)∈Xl
K
α
+ ˆ
p(i|x) (fi (x))+
|Xu |
x∈Xu i=1
K
T
+ ˆ
H (p)
|Xu |
x∈Xu i=1
Graz University of Technology
Amir Saffari,Christian Leistner,Horst Bischof Semi-Supervised Learning in Computer Vision Part II
60. SemiBoost & Visual Similarity Learning On-line Semi-supervised Boosting Semi-Supervised Random Forests On-line Manifo
Two Step Optimization
First Stage
1
f∗ = arg min
n (fy (x))+
f |Xl |
(x,y)∈Xl
α
+ (fyu (x))
ˆ
|Xu |
x∈Xu
Graz University of Technology
Amir Saffari,Christian Leistner,Horst Bischof Semi-Supervised Learning in Computer Vision Part II
61. SemiBoost & Visual Similarity Learning On-line Semi-supervised Boosting Semi-Supervised Random Forests On-line Manifo
Two Step Optimization
Second Stage
K
α
p∗ =arg min
ˆ ˆ
p(i|x) (fi (x))+
ˆ
p |Xu |
x∈Xu i=1
K
T
+ ˆ ˆ
p(i|x) log(p(i|x))
|Xu |
x∈Xu i=1
p ∗ (i|x) = exp(− α
ˆ (fi (x))+T
T )/Z (x)
Graz University of Technology
Amir Saffari,Christian Leistner,Horst Bischof Semi-Supervised Learning in Computer Vision Part II
62. SemiBoost & Visual Similarity Learning On-line Semi-supervised Boosting Semi-Supervised Random Forests On-line Manifo
Finding the optimal Distributions
Take the derivate w.r.t. each class
ˆ ˆ ˆ
hi (p, x) = p(i|x)(α (gi (x)) + T log(p(i|x))) (1)
dhi
ˆ
= α (gi (x)) + T log(p(i|x)) + T (2)
ˆ
d pi
Optimal Distribution
p ∗ (i|x) = exp(− α
ˆ (fi (x))+T
T )/Z (x)
K
Z (x) = ˆ∗
i=1 p (i|x) is the partition function
Graz University of Technology
Amir Saffari,Christian Leistner,Horst Bischof Semi-Supervised Learning in Computer Vision Part II
63. SemiBoost & Visual Similarity Learning On-line Semi-supervised Boosting Semi-Supervised Random Forests On-line Manifo
Experiments
Classification Accuracy in %
Method SVM TSVM SER RMSB RF DAS-RF
g50c 91.7 93.1 91.9 94.2 89.1 93.3
Letter 70.3 65.9 76.5 79.9 76.4 79.7
SensIt 80.2 79.9 81.9 83.7 76.5 84.3
Train and Test time in Seconds
Method SVM TSVM SER RMSB RF DAS-RF GPU
Letter 25 74 3124 125 35 72 29
SensIt 195 687 1158 514 125 410 137
Graz University of Technology
Amir Saffari,Christian Leistner,Horst Bischof Semi-Supervised Learning in Computer Vision Part II
64. SemiBoost & Visual Similarity Learning On-line Semi-supervised Boosting Semi-Supervised Random Forests On-line Manifo
Caltech-101
binary classification error
Class RF DAS-RF Improvement
C4 0.0081 0.0033 58%
C5 0.0078 0.002 65%
C20 0.011 0.0013 87.5%
C33 0.007 0.003 52%
C81 0.0027 0.001 62.5%
classification error over different numbers of labeled samples
Algorithm l = 15 l = 30
RF 0.72 0.64
DAS-RF 0.70 0.60
LinSVM 0.74 0.65
Graz University of Technology
improvement 2% 4%
Amir Saffari,Christian Leistner,Horst Bischof Semi-Supervised Learning in Computer Vision Part II
65. SemiBoost & Visual Similarity Learning On-line Semi-supervised Boosting Semi-Supervised Random Forests On-line Manifo
Prior Regularization
Potential Information Gain
|Il | |Ir |
∆H = − |I |+|Ir | H (Il ) − |Il |+|Ir | H (Ir )
l
Kullback-Leibler Divergence
DKL (q p) = H (q, p) − H (q)
1
DSKL (q p) = 2 (DKL (q p) + DKL (p q))
Prior-regularized node score
∆H ∗ = ∆H + λ∆DSKL (q p)
ˆ
Graz University of Technology
Amir Saffari,Christian Leistner,Horst Bischof Semi-Supervised Learning in Computer Vision Part II
66. SemiBoost & Visual Similarity Learning On-line Semi-supervised Boosting Semi-Supervised Random Forests On-line Manifo
Airbag
m−1 m
OOBE : eF − eF
Graz University of Technology
Amir Saffari,Christian Leistner,Horst Bischof Semi-Supervised Learning in Computer Vision Part II
67. SemiBoost & Visual Similarity Learning On-line Semi-supervised Boosting Semi-Supervised Random Forests On-line Manifo
Airbag
m−1 m
OOBE : eF − eF
Graz University of Technology
Amir Saffari,Christian Leistner,Horst Bischof Semi-Supervised Learning in Computer Vision Part II
68. SemiBoost & Visual Similarity Learning On-line Semi-supervised Boosting Semi-Supervised Random Forests On-line Manifo
Outline
1 SemiBoost & Visual Similarity Learning
2 On-line Semi-supervised Boosting
Tracking
3 Semi-Supervised Random Forests
MILForests
On-line Random Forests
4 On-line Manifold Regularization
5 Conclusion & Outlook
Graz University of Technology
Amir Saffari,Christian Leistner,Horst Bischof Semi-Supervised Learning in Computer Vision Part II
69. SemiBoost & Visual Similarity Learning On-line Semi-supervised Boosting Semi-Supervised Random Forests On-line Manifo
Multiple Instance Forests
[Leistner et al.,ECCV’10]
-
-
- -
+ -
-
+ -
- -
-
+
-
[Dietterich,AI’97]
Content-based Image Retrieval
Object Detection and Categorization
Tracking
Action Recognition
Graz University of Technology
Amir Saffari,Christian Leistner,Horst Bischof Semi-Supervised Learning in Computer Vision Part II
70. SemiBoost & Visual Similarity Learning On-line Semi-supervised Boosting Semi-Supervised Random Forests On-line Manifo
Multiple Instance Forests
Multiple Instance Learning is a special case of semi-supervised
Learning!
Graz University of Technology
Amir Saffari,Christian Leistner,Horst Bischof Semi-Supervised Learning in Computer Vision Part II
71. SemiBoost & Visual Similarity Learning On-line Semi-supervised Boosting Semi-Supervised Random Forests On-line Manifo
Multiple Instance Forests
Multi-class Instance Classifier
F (x) : X → Y = {1, . . . , K }
{(B1 , y1 ), . . . , (Bn , yn )}, where yi ∈ {1, . . . , K }
Objective Function
n ni
j j
({yi }∗ , F ∗ ) =arg min (Fy j (xi ))
j i
{yi },F (·) i=1 j=1
ni
j
s.t. ∀i : I(yi = arg max Fk (xi )) 1.
j=1 k∈Y
Graz University of Technology
Amir Saffari,Christian Leistner,Horst Bischof Semi-Supervised Learning in Computer Vision Part II
72. SemiBoost & Visual Similarity Learning On-line Semi-supervised Boosting Semi-Supervised Random Forests On-line Manifo
Multiple Instance Forests
DA Loss Function
n ni K n
j j
LDA (F , p) =
ˆ ˆ
p(k|xi ) (Fk (xi )) +T ˆ
H (pi )
i=1 j=1 k=1 i=1
Entropy of the distribution inside a bag
ni K
j j
ˆ
H (pi ) = − ˆ ˆ
p(k|xi ) log(p(k|xi ))
j=1 k=1
Graz University of Technology
Amir Saffari,Christian Leistner,Horst Bischof Semi-Supervised Learning in Computer Vision Part II
74. SemiBoost & Visual Similarity Learning On-line Semi-supervised Boosting Semi-Supervised Random Forests On-line Manifo
Corel Data Set
Graz University of Technology
Amir Saffari,Christian Leistner,Horst Bischof Semi-Supervised Learning in Computer Vision Part II
75. SemiBoost & Visual Similarity Learning On-line Semi-supervised Boosting Semi-Supervised Random Forests On-line Manifo
Corel Data Set
Results for the COREL image categorization benchmark
Method Corel-1000 Corel-2000 Testing[sec.] Training[sec.]
MILForest 59 66 4.6 22.0
MILES 58 67 180 960
Graz University of Technology
Amir Saffari,Christian Leistner,Horst Bischof Semi-Supervised Learning in Computer Vision Part II
76. SemiBoost & Visual Similarity Learning On-line Semi-supervised Boosting Semi-Supervised Random Forests On-line Manifo
Semantic Segmentation
[Vezhnevets & Buhmann,CVPR’10]
Graz University of Technology
Amir Saffari,Christian Leistner,Horst Bischof Semi-Supervised Learning in Computer Vision Part II
77. SemiBoost & Visual Similarity Learning On-line Semi-supervised Boosting Semi-Supervised Random Forests On-line Manifo
Outline
1 SemiBoost & Visual Similarity Learning
2 On-line Semi-supervised Boosting
Tracking
3 Semi-Supervised Random Forests
MILForests
On-line Random Forests
4 On-line Manifold Regularization
5 Conclusion & Outlook
Graz University of Technology
Amir Saffari,Christian Leistner,Horst Bischof Semi-Supervised Learning in Computer Vision Part II
78. SemiBoost & Visual Similarity Learning On-line Semi-supervised Boosting Semi-Supervised Random Forests On-line Manifo
On-line Random Forests
On-line Bagging [Oza,PhD-Thesis’01] → Poisson(λ)
On-line recursive splitting is hard → Tree Growing
Info Gain
|Rjls | |Rjrs |
∆L(Rj , s) = L(Rj ) − L(Rjls ) − L(Rjrs )
|Rj | |Rj |
Splitting Rules
|Rj | > α and ∃s ∈ S : ∆L(Rj , s) > β
On-line DA → Annealing Schedule for each sample xi
Graz University of Technology
Amir Saffari,Christian Leistner,Horst Bischof Semi-Supervised Learning in Computer Vision Part II
79. SemiBoost & Visual Similarity Learning On-line Semi-supervised Boosting Semi-Supervised Random Forests On-line Manifo
On-line Random Forests
Graz University of Technology
Amir Saffari,Christian Leistner,Horst Bischof Semi-Supervised Learning in Computer Vision Part II
80. SemiBoost & Visual Similarity Learning On-line Semi-supervised Boosting Semi-Supervised Random Forests On-line Manifo
Interactive Segmentation
Graz University of Technology
Amir Saffari,Christian Leistner,Horst Bischof Semi-Supervised Learning in Computer Vision Part II
81. SemiBoost & Visual Similarity Learning On-line Semi-supervised Boosting Semi-Supervised Random Forests On-line Manifo
Tracking with On-line RF
Graz University of Technology
Amir Saffari,Christian Leistner,Horst Bischof Semi-Supervised Learning in Computer Vision Part II
82. SemiBoost & Visual Similarity Learning On-line Semi-supervised Boosting Semi-Supervised Random Forests On-line Manifo
Tracking
Graz University of Technology
Amir Saffari,Christian Leistner,Horst Bischof Semi-Supervised Learning in Computer Vision Part II
84. SemiBoost & Visual Similarity Learning On-line Semi-supervised Boosting Semi-Supervised Random Forests On-line Manifo
On-line Manifold Regularization
[Goldberg et al.,ECML’08]
Based on Convex Programming in kernel space using
stochastic gradient descent
Random Projection Trees [Dasgupta & Freund, TR, 2007]
Graz University of Technology
Amir Saffari,Christian Leistner,Horst Bischof Semi-Supervised Learning in Computer Vision Part II
85. SemiBoost & Visual Similarity Learning On-line Semi-supervised Boosting Semi-Supervised Random Forests On-line Manifo
On-line Manifold Regularization
[Goldberg et al.,ECML’08]
Graz University of Technology
Amir Saffari,Christian Leistner,Horst Bischof Semi-Supervised Learning in Computer Vision Part II
86. SemiBoost & Visual Similarity Learning On-line Semi-supervised Boosting Semi-Supervised Random Forests On-line Manifo
On-line Graph-based SSL
[Kveton et al.,OLCV’10]
Harmonic Function Solution
Graz University of Technology
Amir Saffari,Christian Leistner,Horst Bischof Semi-Supervised Learning in Computer Vision Part II
87. SemiBoost & Visual Similarity Learning On-line Semi-supervised Boosting Semi-Supervised Random Forests On-line Manifo
On-line Graph-based SSL
[Kveton et al.,OLCV’10]
Merge the two most similar vertices and add the new vertex
Graz University of Technology
Amir Saffari,Christian Leistner,Horst Bischof Semi-Supervised Learning in Computer Vision Part II
88. SemiBoost & Visual Similarity Learning On-line Semi-supervised Boosting Semi-Supervised Random Forests On-line Manifo
On-line Graph-based SSL
[Kveton et al.,OLCV’10]
Face recognition of 8 people
Graz University of Technology
Amir Saffari,Christian Leistner,Horst Bischof Semi-Supervised Learning in Computer Vision Part II
89. SemiBoost & Visual Similarity Learning On-line Semi-supervised Boosting Semi-Supervised Random Forests On-line Manifo
Conclusion
Semi-supervised Learning is a powerful learning paradigm with
many potential applications in Computer Vision
Graz University of Technology
Amir Saffari,Christian Leistner,Horst Bischof Semi-Supervised Learning in Computer Vision Part II
90. SemiBoost & Visual Similarity Learning On-line Semi-supervised Boosting Semi-Supervised Random Forests On-line Manifo
Conclusion
Semi-supervised Learning is a powerful learning paradigm with
many potential applications in Computer Vision
It is often also the way how learning is done in nature
It can be applied virtually everywhere where classifiers are
applied
On-line SSL can be used in order to make
tracking-by-detection systems more robust
Graz University of Technology
Amir Saffari,Christian Leistner,Horst Bischof Semi-Supervised Learning in Computer Vision Part II
91. SemiBoost & Visual Similarity Learning On-line Semi-supervised Boosting Semi-Supervised Random Forests On-line Manifo
Outlook
We need to increase the robustness of SSL algorithms in order to
leverage more applications
Graz University of Technology
Amir Saffari,Christian Leistner,Horst Bischof Semi-Supervised Learning in Computer Vision Part II
92. SemiBoost & Visual Similarity Learning On-line Semi-supervised Boosting Semi-Supervised Random Forests On-line Manifo
Outlook
We need to increase the robustness of SSL algorithms in order to
leverage more applications
Demand for more on-line Semi-Supervised Methods
SSL from weakly-related unlabeled data
Graz University of Technology
Amir Saffari,Christian Leistner,Horst Bischof Semi-Supervised Learning in Computer Vision Part II
93. SemiBoost & Visual Similarity Learning On-line Semi-supervised Boosting Semi-Supervised Random Forests On-line Manifo
References
Books
O. Chapelle and B. Schoelkopf and A. Zien, “Semi-Supervised Learning”, The MIT Press, 2006
Xiaojin Zhu and Andrew B. Goldberg, “Introduction to Semi-Supervised Learning”, Morgan & Claypool, 2009
Papers and Articles
C. Leistner, A. Saffari and H. Bischof, “MILForests: Multiple-Instance Learning with Randomized
Trees”,ECCV’10
C. Leistner, A. Saffari, J. Santner and H. Bischof: “,Semi-Supervised Random Forests”,ICCV’09
C. Leistner, A. Saffari, P Roth and H. Bischof: “On Robustness of On-line Boosting – A Competitive
.M.
Study”,(ICCV) OLCV’09
H. Grabner, C. Leistner and H. Bischof: “On-line Semi-Supervised Boosting for Robust Tracking”,ECCV’08
B. Zeisl, C. Leistner, A. Saffari and H. Bischof: “On-line Semi-supervised Multiple-Instance Boosting”,CVPR’10
C. Leistner, “Semi-Supervised Ensemble Methods for Computer Vision”, PhD-Thesis, Graz University of
Technology, 2010
A. Saffari, C. Leistner, M. Godec, J. Santner and H. Bischof, “On-line Random Forests”, (ICCV) OLCV’09
A. Saffari, C. Leistner, M. Godec and H. Bischof, “Robust Multi-View Multi-Class Boosting with Priors”,ECCV’10
B. Kveton, M. Valko, M. Philipose and L. Huang, “Online Semi-Supervised Perception: Real-Time Learning
without Explicit Feedback”, (CVPR) OLCV’10
A. Saffari, C. Leistner and H. Bischof, “Regularized Multi-Class Semi-Supervised Boosting”,CVPR’09
C. Leistner, H. Grabner and H. Bischof, “Semi-Supervised Boosting using Visual Similarity Learning”,CVPR’08
A. Saffari, C. Leistner and H. Bischof, “Regularized Multi-Class Semi-Supervised Boosting”,CVPR’09
A. Saffari, H. Grabner and H. Bischof, “SERBoost: Semi-supervised Boosting with Expectation Graz University of Technology
Regularization”,ECCV’08
Amir Saffari,Christian Leistner,Horst Bischof Semi-Supervised Learning in Computer Vision Part II