[PR12] Capsule Networks - Jaejun Yoo
•
11 gefällt mir•3,624 views
The document discusses capsule networks, a type of neural network proposed by Geoff Hinton in 2017 as an alternative to convolutional neural networks (CNNs) for computer vision tasks. Capsule networks aim to address some limitations of CNNs, such as their inability to capture spatial relationships and pose information. The key concepts discussed include dynamic routing between capsules, which allows for parts-based representation, and equivariance, where capsules can learn transformation properties like position and orientation. The document provides an overview of a capsule network architecture and routing algorithm proposed in a 2017 paper by Sabour et al.
Empfohlen
Empfohlen
Weitere ähnliche Inhalte
Was ist angesagt?
Was ist angesagt? (20)
Ähnlich wie [PR12] Capsule Networks - Jaejun Yoo
Ähnlich wie [PR12] Capsule Networks - Jaejun Yoo (8)
Mehr von JaeJun Yoo
Mehr von JaeJun Yoo (13)
Kürzlich hochgeladen
Kürzlich hochgeladen (20)
[PR12] Capsule Networks - Jaejun Yoo
- 1. Capsule Networks PR12와 함께 이해하는 Jaejun Yoo Ph.D. Candidate @KAIST PR12 17th Dec, 2017
- 2. Today’s contents Dynamic Routing Between Capsules by Sara Sabour, Nicholas Frosst, Geoffrey E. Hinton Oct. 2017: https://arxiv.org/abs/1710.09829 NIPS 2017 Paper
- 3. Convolutional Neural Networks What is the problem with CNNs? Contents from https://hackernoon.com/what-is-a-capsnet-or-capsule-network-2bfbe48769cc 1) If the images have rotation, tilt or any other different orientation then CNNs have poor performance. 2) In CNN each layer understands an image at a much more granular level (slow increase in receptive field). DATA AUGMENTATION, MAX POOLING
- 4. Convolutional Neural Networks What is the problem with CNNs? Contents from https://hackernoon.com/what-is-a-capsnet-or-capsule-network-2bfbe48769cc “Pooling helps in creating the positional invariance. Otherwise This invariance also triggers false positive for images which have the components of a ship but not in the correct order.”
- 5. Convolutional Neural Networks What is the problem with CNNs? Contents from https://hackernoon.com/what-is-a-capsnet-or-capsule-network-2bfbe48769cc “Pooling helps in creating the positional invariance. Otherwise This invariance also triggers false positive for images which have the components of a ship but not in the correct order.” This was never the intention of pooling layer!
- 6. Convolutional Neural Networks What we need : EQUIVARIANCE (not invariance) Contents from https://hackernoon.com/what-is-a-capsnet-or-capsule-network-2bfbe48769cc “Equivariance makes a CNN understand the rotation or proportion change and adapt itself accordingly so that the spatial positioning inside an image is not lost.”
- 7. Capsules “A capsule is a group of neurons whose activity vector represents the instantiation parameters of a specific type of entity such as an object or an object part.” 8D capsule e.g. Hue, Position, Size, Orientation, deformation, texture, etc.
- 8. Contents from https://www.slideshare.net/aureliengeron/introduction-to-capsule-networks-capsnets 8D capsule e.g. Hue, Position, Size, Orientation, deformation, texture, etc. Capsules 8D vector Inverse Rendering
- 9. Contents from https://www.slideshare.net/aureliengeron/introduction-to-capsule-networks-capsnets 8D capsule e.g. Hue, Position, Size, Orientation, deformation, texture, etc. Capsules 8D vector
- 10. Contents from https://www.slideshare.net/aureliengeron/introduction-to-capsule-networks-capsnets 8D capsule e.g. Hue, Position, Size, Orientation, deformation, texture, etc. Capsules 8D vector
- 11. Contents from https://www.slideshare.net/aureliengeron/introduction-to-capsule-networks-capsnets 8D capsule e.g. Hue, Position, Size, Orientation, deformation, texture, etc. Capsules 8D vector Equivariance of Capsules
- 12. Contents from https://www.slideshare.net/aureliengeron/introduction-to-capsule-networks-capsnets 8D capsule e.g. Hue, Position, Size, Orientation, deformation, texture, etc. Capsules 8D vector Equivariance of Capsules
- 15. Aurélien Géron, 2017 Primary Capsules = = Primary Capsules
- 16. Aurélien Géron, 2017 Predict Next Layer’s Output = = Primary Capsules
- 17. Aurélien Géron, 2017 Predict Next Layer’s Output = = Primary Capsules
- 18. Aurélien Géron, 2017 Predict Next Layer’s Output = = One transformation matrix Wi,j per part/whole pair (i, j). ûj|i = Wi,j ui Primary Capsules
- 19. Aurélien Géron, 2017 Predict Next Layer’s Output = = Primary Capsules
- 20. Aurélien Géron, 2017 Predict Next Layer’s Output = = Primary Capsules
- 21. Aurélien Géron, 2017 Compute Next Layer’s Output = = Predicted Outputs Primary Capsules
- 22. Aurélien Géron, 2017 Routing by Agreement = = Predicted Outputs Primary Capsules Strong agreement!
- 23. Aurélien Géron, 2017 The rectangle and triangle capsules should be routed to the boat capsules. Routing by Agreement = = Predicted Outputs Primary Capsules Strong agreement!
- 24. Aurélien Géron, 2017 Routing Weights = = Predicted Outputs Primary Capsules bi,j=0 for all i, j
- 25. Aurélien Géron, 2017 Routing Weights = = Predicted Outputs Primary Capsules 0.5 0.5 0.5 0.5 bi,j=0 for all i, j ci = softmax(bi)
- 26. Aurélien Géron, 2017 Compute Next Layer’s Output = = Predicted Outputs sj = weighted sum Primary Capsules 0.5 0.5 0.5 0.5
- 27. Aurélien Géron, 2017 Compute Next Layer’s Output = = Predicted Outputs Primary Capsules 0.5 0.5 0.5 0.5 sj = weighted sum vj = squash(sj)
- 28. Aurélien Géron, 2017 Actual outputs of the next layer capsules (round #1) Compute Next Layer’s Output = = Predicted Outputs Primary Capsules 0.5 0.5 0.5 0.5 sj = weighted sum vj = squash(sj)
- 29. Aurélien Géron, 2017 Actual outputs of the next layer capsules (round #1) Update Routing Weights = = Predicted Outputs Primary Capsules Agreement
- 30. Aurélien Géron, 2017 Actual outputs of the next layer capsules (round #1) Update Routing Weights = = Predicted Outputs Primary Capsules Agreement bi,j += ûj|i . vj
- 31. Aurélien Géron, 2017 Actual outputs of the next layer capsules (round #1) Update Routing Weights = = Predicted Outputs Primary Capsules Agreement bi,j += ûj|i . vj Large
- 32. Aurélien Géron, 2017 Actual outputs of the next layer capsules (round #1) Update Routing Weights = = Predicted Outputs Primary Capsules Disagreement bi,j += ûj|i . vj Small
- 33. Aurélien Géron, 2017 Compute Next Layer’s Output = = Predicted Outputs Primary Capsules 0.2 0.1 0.8 0.9
- 34. Aurélien Géron, 2017 Compute Next Layer’s Output = = Predicted Outputs sj = weighted sum Primary Capsules 0.2 0.1 0.8 0.9
- 35. Aurélien Géron, 2017 Compute Next Layer’s Output = = Predicted Outputs Primary Capsules sj = weighted sum vj = squash(sj)0.2 0.1 0.8 0.9
- 36. Aurélien Géron, 2017 Actual outputs of the next layer capsules (round #2) Compute Next Layer’s Output = = Predicted Outputs Primary Capsules 0.2 0.1 0.8 0.9
- 37. Aurélien Géron, 2017 Handling Crowded Scenes = = = =
- 38. Aurélien Géron, 2017 Handling Crowded Scenes = = = = Is this an upside down house?
- 39. Aurélien Géron, 2017 Handling Crowded Scenes = = = = House Thanks to routing by agreement, the ambiguity is quickly resolved (explaining away). Boat
- 40. Aurélien Géron, 2017 Classification CapsNet || ℓ2 || Estimated Class Probability
- 41. Aurélien Géron, 2017 Training || ℓ2 || Estimated Class Probability To allow multiple classes, minimize margin loss: Lk = Tk max(0, m+ - ||vk||2) + λ (1 - Tk) max(0, ||vk||2 - m-) Tk = 1 iff class k is present In the paper: m- = 0.1 m+ = 0.9 λ = 0.5
- 42. Aurélien Géron, 2017 Training Translated to English: “If an object of class k is present, then ||vk||2 should be no less than 0.9. If not, then ||vk||2 should be no more than 0.1.” || ℓ2 || Estimated Class Probability To allow multiple classes, minimize margin loss: Lk = Tk max(0, m+ - ||vk||2) + λ (1 - Tk) max(0, ||vk||2 - m-) Tk = 1 iff class k is present In the paper: m- = 0.1 m+ = 0.9 λ = 0.5
- 43. Aurélien Géron, 2017 Regularization by Reconstruction || ℓ2 || Feedforward Neural Network Decoder Reconstruction
- 44. Aurélien Géron, 2017 Regularization by Reconstruction || ℓ2 || Feedforward Neural Network Decoder Reconstruction Loss = margin loss + α reconstruction loss The reconstruction loss is the squared difference between the reconstructed image and the input image. In the paper, α = 0.0005.
- 45. Aurélien Géron, 2017 A CapsNet for MNIST (Figure 1 from the paper)
- 46. Aurélien Géron, 2017 A CapsNet for MNIST – Decoder (Figure 2 from the paper)
- 47. Aurélien Géron, 2017 Interpretable Activation Vectors (Figure 4 from the paper)
- 48. Aurélien Géron, 2017 Pros ● Reaches high accuracy on MNIST, and promising on CIFAR10 ● Requires less training data ● Position and pose information are preserved (equivariance) ● This is promising for image segmentation and object detection ● Routing by agreement is great for overlapping objects (explaining away) ● Capsule activations nicely map the hierarchy of parts ● Offers robustness to affine transformations ● Activation vectors are easier to interpret (rotation, thickness, skew…) ● It’s Hinton! ;-)
- 49. Aurélien Géron, 2017 ● Not state of the art on CIFAR10 (but it’s a good start) ● Not tested yet on larger images (e.g., ImageNet): will it work well? ● Slow training, due to the inner loop (in the routing by agreement algorithm) ● A CapsNet cannot see two very close identical objects ○ This is called “crowding”, and it has been observed as well in human vision Cons
- 50. Results What the individual dimensions of a capsule represent
- 52. Questions Remained Does capsules really work as the real neurons do? perceptual illusions Thompson, P. (1980). Margaret Thatcher: a new illusion. Perception, 38, (6). 483-484.
- 53. • https://arxiv.org/abs/1710.09829 (paper) • https://jhui.github.io/2017/11/03/Dynamic-Routing-Between- Capsules/ • https://hackernoon.com/what-is-a-capsnet-or-capsule-network- 2bfbe48769cc • https://medium.com/ai%C2%B3-theory-practice- business/understanding-hintons-capsule-networks-part-i-intuition- b4b559d1159b • https://www.youtube.com/watch?v=pPN8d0E3900 (video) • https://www.slideshare.net/aureliengeron/introduction-to-capsule- networks-capsnets (video slides) References