Poincare embeddings for Learning Hierarchical Representations

3. CONFIDENTIAL Today’s Paper Paper Stats • Guys from FAIR • Sumitted to arXiv at May 26, 2017 https://arxiv.org/abs/1705.08039 • Sumitted to NIPS2017? Key Contributions • Introducing hyperbolic geometry (Poincaré disk model) into word/graph embeddings paradigm • Automatically capture hierarchical structure of data • Achieved incredible better results than previous works.

4. CONFIDENTIAL 1. Problems 2. Hyperbolic Geometry 3. Poincaré Embeddings (and Some Incredible Results) Agenda 4

5. CONFIDENTIAL Problems 5

6. CONFIDENTIAL Find good representation(embedding) of items such that underlying hierarchical relation structure are well reconstructed The Problem

7. CONFIDENTIAL Embed nouns in WordNets such that related nouns are close in embedded space Taxonomy Embedding 7 http://www.nltk.org/book_1ed/ch02.html

8. CONFIDENTIAL Embed nodes in given graph such that missing links are well-reconstructed Graph Link Prediction 8 http://ml.cs.tsinghua.edu.cn/~jiaming/publications/

9. CONFIDENTIAL Back Theory 9

10. CONFIDENTIAL • Geometry with negative curvature • Many models (realizations): - Poincaré half space model - Poincaré disk model … each is isometric Hyperbolic Geometry 10

11. CONFIDENTIAL • Defined on upper half space with metric • Distance btw points is Poincaré Half Space Model 11

12. CONFIDENTIAL 12 Tree representation in H https://arxiv.org/abs/1006.5169 • Tree structure is well represented in Poincaré half space

13. CONFIDENTIAL • A realization of hyperbolic geometry • Defined on equipped with metric of • Distance btw points is Poincaré Disk Model 13 M.C. Escher's Circle Limit III, 1959

14. CONFIDENTIAL (for simplicity: 2-dim, identify as ) Relation to Poincaré Half Space Model 14 https://arxiv.org/abs/1006.5169

15. CONFIDENTIAL • Euclidean space is too narrow to embed hierarchical (tree) structures Why not Euclidean Space? 15 Surface Area / # of leaf nodes Volume / # of nodes Euclidean Ball O(R^n) O(R^n) b-ary tree O(b^R) O(b^R) ※ R=radius of ball or depth of tree

16. CONFIDENTIAL • b-array tree can be interpreted as discrete analogue of Poincaré disk Why Hyperbolic Space? 16

17. CONFIDENTIAL • Hyperbolic space is far more appropriate than Euclidean space to represent hierarchical structure • Many equivalent models - Poincaré half space model - Poincaré disk model … Conclusion Here 17

18. CONFIDENTIAL • R. Kleinberg, “Geographic routing using hyperbolic spaces”, 2007 • M. Boguna et al., “Sustaining the internet with hyperbolic mapping”, 2010 • P. D. Hoff et al., “Latent space approaches to social network analysis”, 2016 • A. B. Adcock et al., “Tree-like structure in large social and information networks’, 2013 • D. Krioukov et al., “Hyperbolic geometry of complex networks”, 2010 Prior Works around hyperbolic geometry applications 18

19. CONFIDENTIAL Poincaré Embeddings 19

20. CONFIDENTIAL 1. Parametrize each item in Poincaré ball 2. Optimize them by Riemannian optimization under metric of Proposed Method

21. CONFIDENTIAL 1. Compute stochastic (Euclidean) gradient 2. Correct metric 3. Apply GD 4. Project onto space Riemannian SGD 21

22. CONFIDENTIAL Embed nouns in WordNets such that related nouns are close in embedded space Taxonomy Embedding 22 http://www.nltk.org/book_1ed/ch02.html

23. CONFIDENTIAL Maximize Reconstruction setting: - D is full relations Prediction setting - D is subset of full relations Objective Function 23 randomly chosen 10 negative samples

24. CONFIDENTIAL Result 24

25. CONFIDENTIAL 25

26. CONFIDENTIAL Embed nodes in given graph such that missing links are well-reconstructed Graph Link Prediction 26 http://ml.cs.tsinghua.edu.cn/~jiaming/publications/

27. CONFIDENTIAL Minimize the cross entropy of probability Objective Function 27

28. CONFIDENTIAL Result 28

29. CONFIDENTIAL • Poincaré embeddings automatically capture hierarchical structure from data • Riemannian SGD provides the way to optimize Poincaré embeddings • Achieved quite good results on word/graph embedding tasks Summary 29

Poincare embeddings for Learning Hierarchical Representations

Empfohlen

Empfohlen

Weitere ähnliche Inhalte

Was ist angesagt?

Was ist angesagt? (20)

Andere mochten auch

Andere mochten auch (15)

Ähnlich wie Poincare embeddings for Learning Hierarchical Representations

Ähnlich wie Poincare embeddings for Learning Hierarchical Representations (20)

Mehr von Tatsuya Shirakawa

Mehr von Tatsuya Shirakawa (12)

Kürzlich hochgeladen

Kürzlich hochgeladen (20)

Poincare embeddings for Learning Hierarchical Representations