https://telecombcn-dl.github.io/2018-dlcv/ Deep learning technologies are at the core of the current revolution in artificial intelligence for multimedia data analysis. The convergence of large-scale annotated datasets and affordable GPU hardware has allowed the training of neural networks for data analysis tasks which were previously addressed with hand-crafted features. Architectures such as convolutional neural networks, recurrent neural networks and Q-nets for reinforcement learning have shaped a brand new scenario in signal processing. This course will cover the basic principles and applications of deep learning to computer vision problems, such as image classification, object detection or image captioning.

1. Xavier Giro-i-Nieto
xavier.giro@upc.edu
Associate Professor
Universitat Politecnica de Catalunya
Technical University of Catalonia
The Neural Network Zoo
Day 1 Lecture 2
#DLUPC
http://bit.ly/dlcv2018

2. bit.ly/DLCV2018
#DLUPC
2
Acknowledgements
Fjodor Van Veen,
“The Neural Network Zoo”
The Asimov Institute (2016)

3. bit.ly/DLCV2018
#DLUPC
3
The Perceptron
J. Alammar, “A visual and interactive guide to the Basics of Neural Networks” (2016)
F. Van Veen, “The Neural Network Zoo” (2016)

4. bit.ly/DLCV2018
#DLUPC
4
Neural Network = Multi-layer Perceptron
2 layers of perceptrons
F. Van Veen, “The Neural Network Zoo” (2016)

5. bit.ly/DLCV2018
#DLUPC
5
Deep Neural Network (DNN)
F. Van Veen, “The Neural Network Zoo” (2016)

6. bit.ly/DLCV2018
#DLUPC
6
Recurrent Neural Network (RNN)
Alex Graves, “Supervised Sequence Labelling with Recurrent Neural Networks”
The hidden layers and the
output depend from previous
states of the hidden layers

7. bit.ly/DLCV2018
#DLUPC
7
Recurrent Neural Network (RNN)
F. Van Veen, “The Neural Network Zoo” (2016)

8. bit.ly/DLCV2018
#DLUPC
8
Recurrent Neural Network (RNN)
F. Van Veen, “The Neural Network Zoo” (2016)

9. bit.ly/DLCV2018
#DLUPC
9
Recurrent Neural Network (RNN)
Deep Learning TV, “Recurrent Neural Networks - Ep. 9”

10. bit.ly/DLCV2018
#DLUPC
10
Convolutional Neural Network (CNN)
LeCun, Y., Bottou, L., Bengio, Y., & Haffner, P. (1998). Gradient-based learning applied to document
recognition. Proceedings of the IEEE, 86(11), 2278-2324.
LeNet-5

11. bit.ly/DLCV2018
#DLUPC
11
Convolutional Neural Network (CNN)
Orange
A Krizhevsky, I Sutskever, GE Hinton “Imagenet classification with deep convolutional neural networks”
NIPS 2012.
AlexNet

12. bit.ly/DLCV2018
#DLUPC
12
Convolutional Neural Network (CNN)
Deep Learning TV, “Convolutional Neural Networks - Ep. 8”

13. bit.ly/DLCV2018
#DLUPC
13
Deconvolutional Neural Network
Junting Pan, SalGAN (2017)
F. Van Veen, “The Neural Network Zoo” (2016)

14. bit.ly/DLCV2018
#DLUPC
14
Deep Residual Network
F. Van Veen, “The Neural Network Zoo” (2016)
He, Kaiming, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. "Deep Residual Learning for Image Recognition."
CVPR 2016 [slides]

15. bit.ly/DLCV2018
#DLUPC
15
Skip Connections
Ronneberger, Olaf, Philipp Fischer, and Thomas Brox. "U-net: Convolutional networks for biomedical image
segmentation." MICCAI 2015
U-Net

16. bit.ly/DLCV2018
#DLUPC
16
Skip Connections
Van Den Oord, Aaron, Sander Dieleman, Heiga Zen, Karen Simonyan, Oriol Vinyals, Alex Graves, Nal Kalchbrenner,
Andrew Senior, and Koray Kavukcuoglu. "Wavenet: A generative model for raw audio." ISCA 2016.

17. bit.ly/DLCV2018
#DLUPC
17
Dense Connections
Huang, Gao, Zhuang Liu, Kilian Q. Weinberger, and Laurens van der Maaten. "Densely connected
convolutional networks." CVPR 2017. [code]
Dense Block of
5-layers with a
growth rate of k=4
Connect every layer to every other layer of the same filter
size.

18. bit.ly/DLCV2018
#DLUPC
18
Dense Connections
Huang, Gao, Zhuang Liu, Kilian Q. Weinberger, and Laurens van der Maaten. "Densely connected
convolutional networks." CVPR 2017. [code]

19. bit.ly/DLCV2018
#DLUPC
19Huang, Gao, Zhuang Liu, Kilian Q. Weinberger, and Laurens van der Maaten. "Densely connected
convolutional networks." CVPR 2017. [code]
Dense Connections

20. bit.ly/DLCV2018
#DLUPC
20
Autoencoder (AE)
F. Van Veen, “The Neural Network Zoo” (2016)

21. bit.ly/DLCV2018
#DLUPC
21
Autoencoder (AE)
“Deep Learning Tutorial”, Dept. Computer Science, Stanford
Autoencoders:
● Predict at the output the
same input data.
● Do not need labels:

22. bit.ly/DLCV2018
#DLUPC
22
Autoencoder (AE)
“Deep Learning Tutorial”, Dept. Computer Science, Stanford
Dimensionality reduction:
● Use hidden layer as
a feature extractor of
any desired size.
Application #1

23. bit.ly/DLCV2018
#DLUPC
23
Autoencoder (AE)
Encoder
W1
Decoder
W2
hdata reconstruction
Loss
(reconstruction error)
Latent variables
(representation/features)
Figure: Kevin McGuinness (DLCV UPC 2017)
Pretraining:
1. Initialize a NN solving an autoencoding
problem.
Application #2

24. bit.ly/DLCV2018
#DLUPC
24
Autoencoder (AE)
Pretraining:
1. Initialize a NN solving an autoencoding
problem.
2. Train for final task with “few” labels.
Figure: Kevin McGuinness (DLCV UPC 2017)
Encoder
W1
hdata Classifier
WC
Latent variables
(representation/features)
prediction
y Loss
(cross entropy)
Application #2

25. bit.ly/DLCV2018
#DLUPC
25
Autoencoder (AE)
Deep Learning TV, “Autoencoders - Ep. 10”

26. bit.ly/DLCV2018
#DLUPC
26
Variational Autoencoder (AE)
F. Van Veen, “The Neural Network Zoo” (2016)

27. bit.ly/DLCV2018
#DLUPC
27
Adversarial Networks
Goodfellow, Ian, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua
Bengio. "Generative adversarial nets." NIPS 2014
Goodfellow, Ian. "NIPS 2016 Tutorial: Generative Adversarial Networks." arXiv preprint arXiv:1701.00160 (2016).
F. Van Veen, “The Neural Network Zoo” (2016)

28. bit.ly/DLCV2018
#DLUPC
28
Differentiable Neural Computers (DNC)
F. Van Veen, “The Neural Network Zoo” (2016)
Graves, Alex, Greg Wayne, and Ivo Danihelka. "Neural turing machines." arXiv preprint arXiv:1410.5401
(2014). [slides] [code]

29. bit.ly/DLCV2018
#DLUPC
29
Differentiable Neural Computers (DNC)
von Neumann architecture (1952) Neural Turing Machine (2014)
Graves, Alex, Greg Wayne, and Ivo Danihelka. "Neural turing machines." arXiv preprint arXiv:1410.5401
(2014). [slides] [code]
End-to-end
trainable

30. bit.ly/DLCV2018
#DLUPC
30
Differentiable Neural Computers (DNC)
[Siraj Raval on YouTube]

31. bit.ly/DLCV2018
#DLUPC
31
MAC Network
Hudson, Drew A., and Christopher D. Manning. "Compositional attention networks for machine
reasoning." ICLR 2018.

32. bit.ly/DLCV2018
#DLUPC
32
More architectures to come...
Yann LeCun, “A Path to AI”,
Beneficial AI 2017.
von Neumann architecture (1952)

33. bit.ly/DLCV2018
#DLUPC
33
The Full Story
F. Van Veen, “The Neural Network Zoo” (2016)

34. bit.ly/DLCV2018
#DLUPC
34
The Prequel
Fjodor Van Veen, “Neural Network Zoo Prequel: Cells and Layers”(2017)

35. bit.ly/DLCV2018
#DLUPCQuestions
35