論文紹介 Probabilistic sfa for behavior analysis

1. Zafeiriou, Lazaros, et al.
Neural Networks and Learning Systems, IEEE Transactions on
Probabilistic Slow Feature Analysis
for Behavior Analysis
Presenter : S5lab. Shuuji Mihara

2. Abstract
1
 This Paper propose a number of extensions in both
deterministic and the probabilistic SFA optimization
framework. Particularly about EM-SFA.
 This paper shed further light on the relation of the two
sequence EM-SFA and CCA(Canonical Correlation
Analysis).
 The proposed EM-SFA with DTW(Dynamic Time
Warping) algorithms were applied for facial behavior
analysis, demonstrating their usefulness for this task.

3. Index 2
1. Introduction – What’s SFA?
2. Deterministic SFA
3. Probabilistic SFA
4. EM-SFA
5. EM-SFA with DTW
6. Experiments
7. Conclusion

4. Index 3
1. Introduction – What’s SFA?
2. Deterministic SFA
3. Probabilistic SFA
4. EM-SFA
5. EM-SFA with DTW
6. Experiments
7. Conclusion

5. Slow Feature Analysis
(2002 Wiskott)
Objective : Extract Slow Feature from Time series data .
4
transform
observation latent variable
Slow Feature Analysis
Slow Feature
1

6. Index 5
1. Introduction – What’s SFA?
2. Deterministic SFA
3. Probabilistic SFA
4. EM-SFA
5. EM-SFA with DTW
6. Experiments
7. Conclusion

7. Deterministic SFA(1) 6
transform
observation latent variable
Slow Feature Analysis
Slow Feature
𝑥1,1:𝑇
𝑥2,1:𝑇
𝑥 𝑀,1:𝑇
𝑦1,1:𝑇
𝑦2,1:𝑇
𝑦 𝑁,1:𝑇
𝑋 = [𝑥1,1:𝑇; 𝑥2,1:𝑇 … ; 𝑥 𝑀,1:𝑇] 𝑌 = [𝑦1,1:𝑇; 𝑦2,1:𝑇 … ; 𝑦 𝑁,1:𝑇]
𝑌 = 𝑉 𝑇 𝑋 (𝑉: 𝑀 × 𝑁 𝑚𝑎𝑡𝑟𝑖𝑥)

8. Deterministic SFA(2) 7
Determnistic SFA problem is formulated such optimization problem
min
𝑉
tr[ 𝐘 𝐘T] 𝑠. 𝑡. 𝐘𝟏 = 𝟎, 𝐘𝐘T = 𝐈
constraints: zero mean, unit variance
decorreration
𝑌: 1𝑠𝑡 order time difference

9. Index 8
1. Introduction – What’s SFA?
2. Deterministic SFA
3. Probabilistic SFA
4. EM-SFA
5. EM-SFA with DTW
6. Experiments
7. Conclusion

10. Probabilistic Slow Feature Analysis
(pSFA) 9
Observation model
𝒙 𝑡 = 𝑾−1 𝒚 𝑡 + 𝒘 𝑡
𝒚 𝑡: latent variable 𝒙 𝑡 ∶ observed data
𝝀 ∶ dependency of 𝒚 𝒕−𝟏
𝑾−𝟏: observation matrix
𝒗 𝑡, 𝒘 𝑡: noise(Gaussian)
𝒚 𝑡 = 𝝀𝒚 𝑡−1 + 𝒗 𝒕
System model
𝑣 𝑡~𝑁 0, Σ
𝑤𝑡~𝑁(0, 𝜎 𝑥
2 𝐼)
constraints
𝝀 𝒏
𝟐 + 𝝈 𝒏
𝟐 = 𝟏

11. Probabilistic Slow Feature Analysis
(pSFA) 10
Slow Feature
𝝀 𝒏
𝟐 + 𝝈 𝒏
𝟐 = 𝟏𝜆 𝑛 large
𝜎 𝑛 small
𝜆 𝑛 small
𝜎 𝑛 large
𝒚 𝑡 = 𝝀𝒚 𝑡−1 + 𝒗 𝒕

12. Index 11
1. Introduction – What’s SFA?
2. Deterministic SFA
3. Probabilistic SFA
4. EM-SFA
5. EM-SFA with DTW
6. Experiments
7. Conclusion

13. EM-SFA(1) 12
Previous method
Kalman smoother
ML
estimate Λ, 𝑊−1
estimate 𝒚
Proposed method
Kalman smoother
Learning Sufficient Statistics
✕ Can’t estimate 𝜎 𝑥
2
EM algorithm
estimate Λ, 𝑊−1
, 𝜎 𝑥
2
Update
Sufficient Statistics

14. Kalman Smoother (1) 13
State Space Model
𝒙 𝑘 = 𝐴 𝑘−1 𝒙 𝑘−1 + 𝜼
𝒚 𝑘 = 𝐻 𝑘 𝒙 𝑘 + 𝝐
⇔
𝒙 𝑘 ~ 𝑁(𝐴 𝑘−1 𝒙 𝑘−1, 𝑄 𝑘−1)
𝒚 𝑘 ~ 𝑁(𝑦 𝑘 𝒙 𝑘−1, Σ 𝑘)

15. Kalman Smoother (2) 14
Estimate by Kalman Smoother
14
t,1x
t,2x
t,3x
t,1y
t,2y
t,3y
noise 𝜎𝑥
2
ESTIMATION
Slow Feature

16. Probabilistic Slow Feature Analysis
(pSFA) 15
Observation model
𝒙 𝑡 = 𝑉𝒚 𝑡 + 𝒘 𝑡
𝒚 𝑡: latent variable 𝒙 𝑡 ∶ observed data
𝚲 ∶ dependency of 𝒚 𝒕−𝟏
𝑽 : observation matrix
𝒗 𝑡, 𝒘 𝑡: noise(Gaussian)
𝒚 𝑡 = 𝚲𝒚 𝑡−1 + 𝒗 𝒕
System model
𝑣 𝑡~𝑁 0, Σ
𝑤𝑡~𝑁(0, 𝜎 𝑥
2 𝐼)
constraints
𝝀 𝒏
𝟐 + 𝝈 𝒏
𝟐 = 𝟏

17. Inference In SFA 16
Parameter
Sufficient Statistics for EM
Kalman Smoother

18. EM-SFA Algorithm 17

19. Index 18
1. Introduction – What’s SFA?
2. Deterministic SFA
3. Probabilistic SFA
4. EM-SFA
5. EM-SFA with DTW
6. Experiments
7. Conclusion

20. Time Alignment 19

21. Dynamic Time Warping(DTW) 20
dynamic time warping (DTW) is an algorithm for
measuring similarity between two temporal sequences
which may vary in time or speed.

22. Canonical Correration Analysis 21
canonical-correlation analysis (CCA) is a way of making
sense of cross-covariance matrices.
𝑢 = 𝑎′ 𝑥 𝑣 = 𝑏′ 𝑦
𝑥 = [𝑥1, 𝑥2, … ] y = [𝑦1, 𝑦2, … ]
multivariate data
univariate
𝑎′ 𝑏′ = arg max
𝑎′,𝑏′
𝐶𝑜𝑟[𝑢, 𝑣]

23. EM-SFA with DTW 22
The proposed EM-SFA is more suitable for aligning time
series, since it incorporates temporal constraints (via the
first-order Markov prior), while CCA incorporates a fully
connected MRF prior over the latent space

24. EM-SFA for Two Sequences 23
The Complete joint likelihood distribution
log 𝑃 𝑋1, 𝑋2, 𝑌 𝜃)
= log 𝑃 𝑦1 0, Σ1) +
𝑡=2
𝑇
log 𝑃 𝑦𝑡 𝑦𝑡−1, Λ)
+
𝑡=1
𝑇
log 𝑃 𝑥 𝑡
1
𝑦𝑡, 𝑉1, 𝜎 𝑥,1
2
) +
𝑡=1
𝑇
log 𝑃 𝑥 𝑡
2
𝑦𝑡, 𝑉2, 𝜎 𝑥,2
2
)

25. EM-SFA with DTW 24

26. Index 25
1. Introduction – What’s SFA?
2. Deterministic SFA
3. Probabilistic SFA
4. EM-SFA
5. EM-SFA with DTW
6. Experiments
7. Conclusion

27. Experiment A –Synthetic Data 26

28. 27
Action Unit(AU)

29. Experiment B –Real data1
Unsupervised AU Temporal Phase Segmentation 28

30. Experiment B –Real data2
Temporal Alignment CTW VS EM-SFA with DTW 29

31. Experiment B –Real data3
Conflict Detection 30

32. Experiment B –Real data3
Conflict Detection 31

33. Index 32
1. Introduction – What’s SFA?
2. Deterministic SFA
3. Probabilistic SFA
4. EM-SFA
5. EM-SFA with DTW
6. Experiments
7. Conclusion

34. Conclusion
33
 This Paper propose a number of extensions in both
deterministic and the probabilistic SFA optimization
framework. Particularly about EM-SFA.
 This paper shed further light on the relation of the two
sequence EM-SFA and CCA(Canonical Correlation
Analysis).
 The proposed EM-SFA with DTW(Dynamic Time
Warping) algorithms were applied for facial behavior
analysis, demonstrating their usefulness for this task.

35. State Space Model(1) 34
State Space Model
𝑥2 𝑥 𝑇𝑥1
𝑦1 𝑦2 𝑦 𝑇
latent variable
observed variable
sys-eq
obs-eq