Slides by Xavier Giró-i-Nieto, from the Computer Vision Reading Group. UPC, ETSETB. (27/10/2015)

1. DeepFix: A Fully Convolutional
Neural Network for Predicting
Human Fixations
Srinivas S S Kruthiventi, Kumar Ayush, and R. Venkatesh Babu
(arXiv October 2015) [URL]
Slides by Xavier Giró-i-Nieto, from the Computer Vision Reading Group. (27/10/2015)
https://imatge.upc.edu/web/teaching/computer-vision-reading-group

2. Introduction
2

3. Introduction
3
Bottom-up attention
Automatic
Reflexive
Stimulus-driven

4. Introduction
4
Top-down attention
Subjective’s prior
knowledge
Expectations
Task oriented
Memory
Behavioral goals

5. Introduction
5
Visual Attentional
Mechanisms
Bottom-up
Automatic
Reflexive
Stimulus-driven
Top-down
Subjective’s prior
knowledge
Expectations
Task oriented
Memory
Behavioral goals

6. Introduction

7. Introduction
7
DeepFixClassic method

8. Introduction
8
mit300 benchmark [URL]

9. Introduction
9
cat200 benchmark [URL]

10. The ingredients
10

11. Very deep network
11
Simonyan, Karen, and Andrew Zisserman. "Very deep convolutional networks for
large-scale image recognition." arXiv preprint arXiv:1409.1556 (2014)
● Inspired by Oxford’s VGG net (19 layers).
● 20 layers
● Small kernel sizes.

12. Fully convolutional network (FCN)
12
● Fully connected layers at the end
are replaced by convolutional
layers with very large receptive
fields.
● They capture the global context of
the scene.
● End-to-end training
Long, J., Shelhamer, E., & Darrell, T. (2015). Fully Convolutional Networks for Semantic
Segmentation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition
(pp. 3431-3440)

13. 13
Inception layers
Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., ... & Rabinovich, A. (2015). Going
Deeper With Convolutions. In Proceedings of the IEEE Conference on Computer Vision and Pattern
Recognition (pp. 1-9)
● GoogLeNet
● Different kernel sizes
operating in parallel.

14. 14
Location Biased Convolutional (LBC) layer
● Centre-bias
●

15. The network
15

16. Architecture
16
Small convolutional filters of
3x3 with stride of 1 to allow a
large depth without increasing
the memory requirement

17. Architecture
17
Max pooling layers (in red)
reduce computation.

18. Architecture
18
Gradual increase in the
amount of channels to
progressively learn richer
semantic representations: 64,
128, 256, 512...

19. Architecture
19
Weights initialized from
VGG-16 net for stable and
effective learning

20. Architecture
20
Convolution kernel 3x3 with
hole size 2 have a
receptive field of 5x5.

21. Architecture
21
Capture multi-scale
semantic structure using
two inception style
convolutional modules

22. Architecture
22
Very large receptive fields
of 25x25 by introducing
holes of size 6 in kernels

23. Architecture
23
Location Biased
Convolutional (LBC) layers

24. Architecture
24
Location Biased
Convolutional (LBC) layers

25. Architecture
25
constant during training learnt during training
weights from c’th filter in
a convolutional layer
input blob

26. Architecture
26
Final output W/8xH/8 is
upsampled.

27. Experiments
27

28. Training
28
2nd stage
MIT 1003
CAT2000
Mouse clicks from Microsoft CoCo
Not mentioned how to go from eye
fixations to heat mapa !!

29. Training
29
● End to end (as JuntingNet)
● Caffeframework
● 1 day in K40 GOU!

30. Results
30

31. Results
31

32. Results
32

33. Results
33

34. Results
34