Yifan Sun

1. 1. S. Bai et al., Scalable Person Re-identification on Supervised Smoothed Manifold.
CVPR2017 spotlight
2. Y. Sun et al., Deep Representation Learning: a Similarity Smoothing Perspective
3. Y. Shen et al., Person Re-identification with Deep Similarity-Guided Graph Neural
Network. ECCV 2018 accepted
Keywords: Smooth Similarity, Smoothed Manifold, Graph Neural Network
孙奕帆
ReID中的相似性平滑约束

2. Scalable Re-ID on Supervised Smoothed Manifold.
Reference:
1. D. Zhou et al., Learning with local and global consistency, NIPS2003
2. S. Bai et al., Scalable Person Re-identification on Supervised Smoothed Manifold. CVPR2017
何为相似性不平滑
PA: A1 A2
B2
假设某特征空间中存在以下关系：
A1 close to B1
A2 close to B2
现考察样本对内部的样本距离，假设：
A1 close to A2
B1 far from B2
PB : B1
PA close to PB
互斥于 PA close to PB
一个极端的例子：假设2组样本对PA=(A1,A2)及PB=(B1,B2)

3. 如何施加相似性平滑约束
k
i
l
j
Qki: 样本k，i间的相似性 Qlj: 样本l,j间的相似性
P(ki→lj) 样本对ki到样本对lj的状态转移概率
Lki： 样本k，i是否属于同一ID
正样对为1，负样对为0，L视为硬化的相似性，故可与Q进行加法运算
直观意义：k,i之间的相似性Qk i“吸收”所有其它样本的相似性
Ql j, l,j∈{1,2,…,N} ，
吸收强度由样本对{l,j}、{k,i}之间的状态转移概率P(ki→lj)决定
Scalable Re-ID on Supervised Smoothed Manifold.

4. 如何施加相似性平滑约束
k
i
l
j
Qki: 样本k，i间的相似性 Qlj: 样本l,j间的相似性
P(ki→lj) 样本对ki到样本对lj的状态转移概率
Lki： 样本k，i是否属于同一ID
正样对为1，负样对为0，L视为硬化的相似性，故可与Q进行加法运算
直观意义：k,i之间的相似性Qk i“吸收”所有其它样本的相似性
Ql j, l,j∈{1,2,…,N} ，
吸收强度由样本对{l,j}、{k,i}之间的状态转移概率P(ki→lj)决定
Scalable Re-ID on Supervised Smoothed Manifold.
真的是
所有吗？

5. 相似性吸收强度
2
W expij
dij

  
      
一个常用定义：
控制半径
越小，越关注局部：欧氏距离近的样本对，其W绝对占优，导致吸收仅在较小局部发生
越大，越关注全局：欧氏距离较远的样本对，其强度仍然能被吸收
Scalable Re-ID on Supervised Smoothed Manifold.

6. SSM的一些问题：
采用后处理方式，增加测试阶段时间。在整个样本空间
（train+test）进行相似性传递，复杂度接近O(N4) （N为样本数），
难以在大训练集上学习。且利用了gallery信息

7. Deep Representation Learning:
a Similarity Smoothing Perspective
Yifan Sun, Liang Zheng, Qin Xu, Zhongdao Wang, Shengjin Wang

8. Motivation----smooth similarity
• Has been valued in semi-supervised learning or transductive inference
• Has not been explored in deep representation learning under fully
supervision
A1
A2
B1
B2
Pair I Pair II
similar dissimilar
A1
A2
B1
B2
Pair I Pair II
A1
A2
B1
B2
Pair I Pair II
(a) smooth (b) smooth (c) unsmooth

9. Our Contribution
• We introduce the smooth similarity constraint, which is traditionally utilized in
semi-supervised learning, to deep representation learning under the fully
supervised manner
• We define an evaluation protocol to measure similarity smoothness and
transform it to a Similarity Smoothing Regularizer (SSR).
• We demonstrate though extensive experiments on four fine-grained retrieval
datasets, that similarity smoothing is beneficial towards more discriminative
representation.

10. Proposed Method
• A revisit to Smooth Similarity Constraint
• Similarity Smoothness Indicator
• Similarity Smoothing Regularizer (SSR)
• The similarity measure W
• A light edition of SSR for efficient training

11. Proposed Method
• A revisit to Smooth Similarity Constraint
Affinity value which is initialized with W

12. Proposed Method
• A revisit to Smooth Similarity Constraint
Affinity value which is initialized with W
Wij: the similarity value calculated with similarity measure
which may be heuristically defined

13. Proposed Method
• Similarity Smoothness Indicator

14. Proposed Method
• Similarity Smoothness Indicator
A weighted mean of the similarity variations
between sample pairs

15. Proposed Method
• Similarity Smoothing Regularizer (SSR)
• Takes the same formula as the Similarity Smoothness Indicator
• To be evaluated within the training mini-batch
• Essentially enforces the similarity not to change too much between nearby pairs
• May achieve a optimum Solution A, compromising the discriminative ability
• So, it is important to combine a metric loss

16. Proposed Method
• The similarity measure W
• Cosine similarity
• Gaussian similarity
RBF kernel width

17. Proposed Method
• The similarity measure W
• Gaussian similarity
RBF kernel width impacts on the optimization of SSR
Inappropriate settings will lead SSR to approximate
another optimum Solution B, decreasing the retrieval accuracy:

18. Proposed Method
• A light edition of SSR for efficient training
When adopting the N-pairs sampling strategy, the computational cost is reduced by V^4
（1/4096 when 8 instances for a same identity）
while bringing little impact on the retrieval accuracy
LSSR focuses on inter-class similarity smoothness

19. Task----Fine grained Retrieval
• Person Re-identification
• Birds & Cars Retrieval (王重道)
CUB-200 CARS196

20. Experiments-reID

21. Experiments-birds and cars retrieval
Baseline 4: contrastive loss alone

22. Experiments-Mechanism Study
Impact of Similarity Measure W
1）under cosine similarity
2) The effectiveness of using Gaussian Simlarity
Depends
3) A interesting observation:
Only when SSR construct a competing effect
with the cooperating metric loss, (Solution A),
the accuracy increases

23. Experiments-Mechanism Study
Visualization

25. SGGNN----propagating features within mini-batch
Element-wise subtraction
and square operation
FC+sigmoid (positive pair1
negative pair0)
Absorbing difference feature
“d” from other pairs.
The absorbing weight W is
determined by the transition
probability

26. SGGNN----propagating features within mini-batch
The message (or feature) to propagate
is a learnable t = 2FC+BN on “d”, rather
than “d” itself

27. SGGNN----propagating features within mini-batch
• Performance
• The similarity scores
are predicted by SGGNN
---time consuming

28. SGGNN----propagating features within mini-batch

29. Connection & Difference between SSR and SGGNN
• Both method employ smooth similarity constraint on the training dataset (instead
of on the training + testing)
A1
A2
B1
B2
Pair I Pair II
SGGNN: propagating
features within
triplet (special case
of quadruple)
SSR:
propagating
similarities
between any
sample pairs
(quadruple)
A1
A2
B2
Pair I Pair II