https://mcv-m6-video.github.io/deepvideo-2018/ Overview of deep learning solutions for video processing. Part of a series of slides covering topics like action recognition, action detection, object tracking, object detection, scene segmentation, language and learning from videos. Prepared for the Master in Computer Vision Barcelona: http://pagines.uab.cat/mcv/

1. @DocXavi
Xavier Giró-i-Nieto
[http://pagines.uab.cat/mcv/]
Module 6
Deep Learning for Video:
Action Detection
22nd March 2018

2. BSc
thesis
http://activity-net.org/ 2
Temporal Action Detection

3. BSc
thesis
Activity Classification
Longboarding
3
Temporal Action Detection

4. BSc
thesis
Activity Temporal Localization
Longboarding
4
Temporal Action Detection

5. 5
A. Montes, Salvador, A., Pascual-deLaPuente, S., and Giró-i-Nieto, X., “Temporal Activity Detection in Untrimmed
Videos with Recurrent Neural Networks”, NIPS Workshop 2016 (best poster award)

6. BSc
thesis
Neural Network
Activity
6
Temporal Action Detection

7. BSc
thesis
Activity
CNN RNN+
7
Temporal Action Detection

8. 8
Action Detection: C3D + LSTM
Alberto Montes, Amaia Salvador, Santiago Pascual, and Xavier Giro-i-Nieto. "Temporal Activity Detection in Untrimmed Videos
with Recurrent Neural Networks." NIPS Workshop 2016 (best poster award)

9. 9
Action Detection: C3D + LSTM
Alberto Montes, Amaia Salvador, Santiago Pascual, and Xavier Giro-i-Nieto. "Temporal Activity Detection in Untrimmed Videos
with Recurrent Neural Networks." NIPS Workshop 2016 (best poster award)
Background
Activity 1
Activity 2
Activity 200
Clip1
Clip2
Clip3
ClipN

10. 10
Action Detection: C3D + LSTM
Alberto Montes, Amaia Salvador, Santiago Pascual, and Xavier Giro-i-Nieto. "Temporal Activity Detection in Untrimmed Videos
with Recurrent Neural Networks." NIPS Workshop 2016 (best poster award)
Background
Activity 1
Activity 2
Activity 200
Clip1
Clip2
Clip3
ClipN
Average

11. 11
Action Detection: C3D + LSTM
Alberto Montes, Amaia Salvador, Santiago Pascual, and Xavier Giro-i-Nieto. "Temporal Activity Detection in Untrimmed Videos
with Recurrent Neural Networks." NIPS Workshop 2016 (best poster award)
Background
Activity 1
Activity 2
Activity 200
Clip1
Clip2
Clip3
ClipN
Average
Max Probability

12. 12
Action Detection: C3D + LSTM
Alberto Montes, Amaia Salvador, Santiago Pascual, and Xavier Giro-i-Nieto. "Temporal Activity Detection in Untrimmed Videos
with Recurrent Neural Networks." NIPS Workshop 2016 (best poster award)
Background
Activity 1
Activity 2
Activity 200
Clip1
Clip2
Clip3
ClipN
Average filter of k samplestime

13. 13
Action Detection: C3D + LSTM
Alberto Montes, Amaia Salvador, Santiago Pascual, and Xavier Giro-i-Nieto. "Temporal Activity Detection in Untrimmed Videos
with Recurrent Neural Networks." NIPS Workshop 2016 (best poster award)
Background
Activity
Clip1
Clip2
Clip3
ClipN
Ɣ

14. 14
Action Detection: C3D + LSTM
Alberto Montes, Amaia Salvador, Santiago Pascual, and Xavier Giro-i-Nieto. "Temporal Activity Detection in Untrimmed Videos
with Recurrent Neural Networks." NIPS Workshop 2016 (best poster award)
Ɣ

15. 15
What techniques are used for object detection in images ?

16. 16
Shou, Zheng, Dongang Wang, and Shih-Fu Chang. "Temporal action localization in untrimmed videos via multi-stage cnns."
CVPR 2016. (Slidecast and Slides by Alberto Montes)
Action Detection: Proposals

17. 17
Shou, Zheng, Dongang Wang, and Shih-Fu Chang. "Temporal action localization in untrimmed videos via multi-stage cnns."
CVPR 2016. (Slidecast and Slides by Alberto Montes)
Action Detection: Proposals

18. 18
Shou, Zheng, Dongang Wang, and Shih-Fu Chang. "Temporal action localization in untrimmed videos via multi-stage cnns."
CVPR 2016. (Slidecast and Slides by Alberto Montes)
Non-Maximum
Suppresion
Action Detection: Proposals

19. BSc
thesis
19
Action Detection: Two-stream CNN + RNN
Singh, Bharat, Tim K. Marks, Michael Jones, Oncel Tuzel, and Ming Shao. "A multi-stream
bi-directional recurrent neural network for fine-grained action detection." CVPR 2016
Motion
Appearance

20. BSc
thesis
20
Singh, Bharat, Tim K. Marks, Michael Jones, Oncel Tuzel, and Ming Shao. "A multi-stream
bi-directional recurrent neural network for fine-grained action detection." CVPR 2016
Action Detection: Two-stream CNN + RNN
Multi-Stream
Network

21. 21
Yeung, Serena, Olga Russakovsky, Greg Mori, and Li Fei-Fei. "End-to-end Learning of Action Detection
from Frame Glimpses in Videos." CVPR 2016
Action Detection: Reinforcement Learning

22. 22
Yeung, Serena, Olga Russakovsky, Greg Mori, and Li Fei-Fei. "End-to-end Learning of Action Detection
from Frame Glimpses in Videos." CVPR 2016
Action Detection: Reinforcement Learning

23. 23
Shou, Zheng, Junting Pan, Jonathan Chan, Kazuyuki Miyazawa, Hassan Mansour, Anthony Vetro, Xavier
Giro-i-Nieto, and Shih-Fu Chang. "Online Action Detection in Untrimmed, Streaming Videos-Modeling and
Evaluation." arXiv preprint arXiv:1802.06822 (2018).
Action Detection: Online

24. ● MSc course (2017)
● BSc course (2018)
24
Deep Learning online courses by UPC:
● 1st edition (2016)
● 2nd edition (2017)
● 3rd edition (2018)
● 1st edition (2017)
● 2nd edition (2018)
Next edition Autumn 2018 Next edition Winter/Spring 2019Summer School (late June 2018)