Diese Präsentation wurde erfolgreich gemeldet.
Wir verwenden Ihre LinkedIn Profilangaben und Informationen zu Ihren Aktivitäten, um Anzeigen zu personalisieren und Ihnen relevantere Inhalte anzuzeigen. Sie können Ihre Anzeigeneinstellungen jederzeit ändern.
Nächste SlideShare
What to Upload to SlideShare
What to Upload to SlideShare
Wird geladen in …3
×
1 von 43

Xenia miscouridou wi mlds 4

0

Teilen

Herunterladen, um offline zu lesen

Xenia presentation, WiMLDS Limassol and Paris, Feb 2021

Ähnliche Bücher

Kostenlos mit einer 30-tägigen Testversion von Scribd

Alle anzeigen

Ähnliche Hörbücher

Kostenlos mit einer 30-tägigen Testversion von Scribd

Alle anzeigen

Xenia miscouridou wi mlds 4

  1. 1. PhD and research overview Deep dive in probabilistic modelling for interaction data Open Questions Appendices X Miscouridou 1 / 33
  2. 2. PhD and research overview Deep dive in probabilistic modelling for interaction data Open Questions Appendices Outline 1 Research overview From probabilistic modelling to deep learning 2 Deep dive in probabilistic modelling Understanding social interactions 3 Open questions Bridging the gap between statistics and computation X Miscouridou 2 / 33
  3. 3. PhD and research overview Deep dive in probabilistic modelling for interaction data Open Questions Appendices Outline 1 PhD and research overview From probabilistic modelling to deep learning 2 Deep dive in probabilistic modelling for interaction data 3 Open Questions X Miscouridou 3 / 33
  4. 4. PhD and research overview Deep dive in probabilistic modelling for interaction data Open Questions Appendices From probabilistic modelling to deep learning My research map Statistical Machine Learning Probabilistic Modelling Model a phenomenon using random variables and probability distributions Statistical methods Expressivity and explainability Accuracy Deep Learning Algorithms inspired by the structure of neural networks in the brain Computation Artificial neural networks Prediction Goals of the two areas may differ but combining their strengths is promising X Miscouridou 4 / 33
  5. 5. PhD and research overview Deep dive in probabilistic modelling for interaction data Open Questions Appendices Outline 1 PhD and research overview 2 Deep dive in probabilistic modelling for interaction data Setup Modelling Temporal interactions Inference and performance 3 Open Questions X Miscouridou 5 / 33
  6. 6. PhD and research overview Deep dive in probabilistic modelling for interaction data Open Questions Appendices Setup Modelling temporal interactions Model temporal interaction data in triplets of the form (i, j, t) corresponding to an action from a node i to node j at time t Which are the factors that underpin these interactions? Which properties are observed in the underlying network that i, j belong to? Arising in social networks biological data neural activity Aiming to uncover trends predict future links assist in decision making X Miscouridou 6 / 33
  7. 7. PhD and research overview Deep dive in probabilistic modelling for interaction data Open Questions Appendices Setup B X Miscouridou 7 / 33
  8. 8. PhD and research overview Deep dive in probabilistic modelling for interaction data Open Questions Appendices Setup A B 2.1 X Miscouridou 8 / 33
  9. 9. PhD and research overview Deep dive in probabilistic modelling for interaction data Open Questions Appendices Setup A B 2.1, 2.8 X Miscouridou 9 / 33
  10. 10. PhD and research overview Deep dive in probabilistic modelling for interaction data Open Questions Appendices Setup A B 2.1, 2.8 5.6 X Miscouridou 10 / 33
  11. 11. PhD and research overview Deep dive in probabilistic modelling for interaction data Open Questions Appendices Setup A B 2.1, 2.8, 7.5 5.6 X Miscouridou 11 / 33
  12. 12. PhD and research overview Deep dive in probabilistic modelling for interaction data Open Questions Appendices Setup A B 2.1, 2.8, 7.5 5.6, 7.6 X Miscouridou 12 / 33
  13. 13. PhD and research overview Deep dive in probabilistic modelling for interaction data Open Questions Appendices Setup A B 2.1, 2.8, 7.5 5.6, 7.6, 7.8 X Miscouridou 13 / 33
  14. 14. PhD and research overview Deep dive in probabilistic modelling for interaction data Open Questions Appendices Setup A B 2.1, 2.8, 7.5, 8.1 5.6, 7.6, 7.8 X Miscouridou 14 / 33
  15. 15. PhD and research overview Deep dive in probabilistic modelling for interaction data Open Questions Appendices Setup A B 2.1, 2.8, 7.5, 8.1, 8.4 5.6, 7.6, 7.8 X Miscouridou 15 / 33
  16. 16. PhD and research overview Deep dive in probabilistic modelling for interaction data Open Questions Appendices Modelling Temporal interactions Which is the key property and how is it modelled? Reciprocity: an action from A to B increases the chances of a similar action being returned in the near future Modelled by inhomogeneous point processes: N(t):Counting process representing the number of events up to t λ(t): Intensity function driving number of events at time t N(t) ∼ HP (λ (t)) , P (N (t + dt) − N (t) = 1|Ht) = λ (t) dt, where Ht is the subset of events up to time t X Miscouridou 16 / 33
  17. 17. PhD and research overview Deep dive in probabilistic modelling for interaction data Open Questions Appendices Modelling Temporal interactions Which is the key property and how is it modelled? Reciprocity: an action from A to B increases the chances of a similar action being returned in the near future Modelled by inhomogeneous point processes: N(t):Counting process representing the number of events up to t λ(t): Intensity function driving number of events at time t N(t) ∼ HP (λ (t)) , P (N (t + dt) − N (t) = 1|Ht) = λ (t) dt, where Ht is the subset of events up to time t X Miscouridou 16 / 33
  18. 18. PhD and research overview Deep dive in probabilistic modelling for interaction data Open Questions Appendices Modelling Temporal interactions Which is the key property and how is it modelled? Reciprocity: an action from A to B increases the chances of a similar action being returned in the near future Modelled by inhomogeneous point processes: N(t):Counting process representing the number of events up to t λ(t): Intensity function driving number of events at time t N(t) ∼ HP (λ (t)) , P (N (t + dt) − N (t) = 1|Ht) = λ (t) dt, where Ht is the subset of events up to time t X Miscouridou 16 / 33
  19. 19. PhD and research overview Deep dive in probabilistic modelling for interaction data Open Questions Appendices Modelling Temporal interactions Which is the key property and how is it modelled? Reciprocity: an action from A to B increases the chances of a similar action being returned in the near future Modelled by inhomogeneous point processes: N(t):Counting process representing the number of events up to t λ(t): Intensity function driving number of events at time t N(t) ∼ HP (λ (t)) , P (N (t + dt) − N (t) = 1|Ht) = λ (t) dt, where Ht is the subset of events up to time t X Miscouridou 16 / 33
  20. 20. PhD and research overview Deep dive in probabilistic modelling for interaction data Open Questions Appendices Modelling Temporal interactions Which is the key property and how is it modelled? Reciprocity: an action from A to B increases the chances of a similar action being returned in the near future Modelled by inhomogeneous point processes: N(t):Counting process representing the number of events up to t λ(t): Intensity function driving number of events at time t N(t) ∼ HP (λ (t)) , P (N (t + dt) − N (t) = 1|Ht) = λ (t) dt, where Ht is the subset of events up to time t X Miscouridou 16 / 33
  21. 21. PhD and research overview Deep dive in probabilistic modelling for interaction data Open Questions Appendices Modelling Temporal interactions Mutually exciting processes Model mutual excitation between a pair using two point processes (NAB, NBA) with intensities (λAB, λBA) NAB(t) = P events up to time t with sender A, receiver B NBA(t) = P events up to time t with sender B, receiver A Mathematically, λAB(t) = µ + Z t 0 gAB(t − u) dNBA(u) λBA(t) = µ + Z t 0 gBA(t − u) dNAB(u) where µ = λAB(0) = λBA(0) > 0 are symmetric and gAB, gBA are non-negative functions X Miscouridou 17 / 33
  22. 22. PhD and research overview Deep dive in probabilistic modelling for interaction data Open Questions Appendices Modelling Temporal interactions Mutually exciting processes Model mutual excitation between a pair using two point processes (NAB, NBA) with intensities (λAB, λBA) NAB(t) = P events up to time t with sender A, receiver B NBA(t) = P events up to time t with sender B, receiver A Mathematically, λAB(t) = µ + Z t 0 gAB(t − u) dNBA(u) λBA(t) = µ + Z t 0 gBA(t − u) dNAB(u) where µ = λAB(0) = λBA(0) > 0 are symmetric and gAB, gBA are non-negative functions X Miscouridou 17 / 33
  23. 23. PhD and research overview Deep dive in probabilistic modelling for interaction data Open Questions Appendices Modelling Temporal interactions Recall the (A,B) pair example A B 2.1, 2.8, 7.5, 8.1, 8.4 5.6, 7.6, 7.8 X Miscouridou 18 / 33
  24. 24. PhD and research overview Deep dive in probabilistic modelling for interaction data Open Questions Appendices Modelling Temporal interactions Intensity and counting process graphically A −→ B B −→ A X Miscouridou 19 / 33
  25. 25. PhD and research overview Deep dive in probabilistic modelling for interaction data Open Questions Appendices Modelling Temporal interactions Model all pairs in the network B A C D E 2.1, 2.8, 7.5, 8.1, 8.4 5.6, 7.6, 7.8 2 1.3 2 2.4, 5.6, 9.3 7.8 0.5, 1.5 1.2 1.0, 2.0, 2.3 X Miscouridou 20 / 33
  26. 26. PhD and research overview Deep dive in probabilistic modelling for interaction data Open Questions Appendices Modelling Temporal interactions Model all the pairs in a network For each directed pair (i, j) let Nij(t) be the counting process for the events from i towards j, with intensity process λij(t) Nij, Nji are mutually exciting via their exponential intensities λij(t) = µij + Z t 0 ηe−δ(t−u) dNji(u) λji(t) = µji + Z t 0 ηe−δ(t−u) dNij(u) µij = µji X Miscouridou 21 / 33
  27. 27. PhD and research overview Deep dive in probabilistic modelling for interaction data Open Questions Appendices Modelling Temporal interactions Global network structure Through the base intensities µij we model the network structure. Sparsity/Density Heterogeneous behaviour Communities a. Dense b. Slightly Sparse c. Sparse d. Very Sparse X Miscouridou 22 / 33
  28. 28. PhD and research overview Deep dive in probabilistic modelling for interaction data Open Questions Appendices Inference and performance How is this model family useful and novel? 1 Provides accurate prediction of number of future links 2 Can be used to measure effects of interventions 3 Allows for personalization 4 Provides a good trade-off in explainability - scalability X Miscouridou 23 / 33
  29. 29. PhD and research overview Deep dive in probabilistic modelling for interaction data Open Questions Appendices Inference and performance How is this model family useful and novel? 1 Provides accurate prediction of number of future links 2 Can be used to measure effects of interventions 3 Allows for personalization 4 Provides a good trade-off in explainability - scalability X Miscouridou 23 / 33
  30. 30. PhD and research overview Deep dive in probabilistic modelling for interaction data Open Questions Appendices Inference and performance How is this model family useful and novel? 1 Provides accurate prediction of number of future links 2 Can be used to measure effects of interventions 3 Allows for personalization 4 Provides a good trade-off in explainability - scalability X Miscouridou 23 / 33
  31. 31. PhD and research overview Deep dive in probabilistic modelling for interaction data Open Questions Appendices Inference and performance How is this model family useful and novel? 1 Provides accurate prediction of number of future links 2 Can be used to measure effects of interventions 3 Allows for personalization 4 Provides a good trade-off in explainability - scalability X Miscouridou 23 / 33
  32. 32. PhD and research overview Deep dive in probabilistic modelling for interaction data Open Questions Appendices Inference and performance References Miscouridou X., Caron F. and Teh W.Y. Modelling sparsity, heterogeneity, reciprocity and community structure in temporal interaction data. Neural Information Processing Systems, 2018 Miscouridou X., Caron F. and Teh Y.W. Code for Hawkes Processes on Dynamic Graphs https: // github. com/ OxCSML-BayesNP/ HawkesNetOC , 2018 Todeschini, A., Miscouridou X.,and Caron, F. Exchangeable random measures for sparse and modular graphs with overlapping communities. Journal of the Royal Statistical Society, 2020. Todeschini, A., Miscouridou X.,and Caron, F. NetOC : Matlab package for Sparse Stochastic Blockmodels https: // github. com/ OxCSML-BayesNP/ SNetOC , 2019. X Miscouridou 24 / 33
  33. 33. PhD and research overview Deep dive in probabilistic modelling for interaction data Open Questions Appendices Outline 1 PhD and research overview 2 Deep dive in probabilistic modelling for interaction data 3 Open Questions Bridging the gap between Statistics and Computation X Miscouridou 25 / 33
  34. 34. PhD and research overview Deep dive in probabilistic modelling for interaction data Open Questions Appendices Bridging the gap between Statistics and Computation Bridging the gap between Statistics & Computation Combining the strengths of both contributes to effective and richer AI Statistical methods Can we scale them up and make them efficient? Deep Learning Can we make it more probabilistically principled? X Miscouridou 26 / 33
  35. 35. PhD and research overview Deep dive in probabilistic modelling for interaction data Open Questions Appendices Bridging the gap between Statistics and Computation Bridging the gap between Statistics & Computation Combining the strengths of both contributes to effective and richer AI Statistical methods Can we scale them up and make them efficient? Deep Learning Can we make it more probabilistically principled? X Miscouridou 26 / 33
  36. 36. PhD and research overview Deep dive in probabilistic modelling for interaction data Open Questions Appendices Bridging the gap between Statistics and Computation Bridging the gap between Statistics & Computation Combining the strengths of both contributes to effective and richer AI Statistical methods Can we scale them up and make them efficient? Deep Learning Can we make it more probabilistically principled? X Miscouridou 26 / 33
  37. 37. PhD and research overview Deep dive in probabilistic modelling for interaction data Open Questions Appendices Bridging the gap between Statistics and Computation Recap 1 Research overview Probabilistic modelling to deep learning 2 Deep dive in probabilistic modelling A bayesian probabilistic model for social interactions 3 Open questions Bridging the gap between probabilistic and deep learning X Miscouridou 27 / 33
  38. 38. PhD and research overview Deep dive in probabilistic modelling for interaction data Open Questions Appendices Bridging the gap between Statistics and Computation Q&A Thank You X Miscouridou 28 / 33
  39. 39. PhD and research overview Deep dive in probabilistic modelling for interaction data Open Questions Appendices Bibliography I Lee J., Miscouridou X., Caron F. A unified construction for series representations and finite approximations of completely random measures submitted to Bernoulli 2019. Miscouridou X., Caron F. and Teh W.Y. (2018). modelling sparsity, reciprocity, heterogeneity and community structure in temporal interaction data. Neural Information Processing Systems 2018 . Miscouridou X., Perotte A., Elhadad N., Ranganath R. Deep Survival Analysis: Nonparametrics and Missingness Proceedings of Machine Learning Research. Todeschini, A., Miscouridou X.,and Caron, F. (2019). Exchangeable random measures for sparse and modular graphs with overlapping communities. Journal of the Royal Statistical Society, 2019. Willetts M., Miscouridou X., Teh Y.W. , Holmes C, Roberts S. Relaxed Responsibility Vector Quantization, Under Review . X Miscouridou 29 / 33
  40. 40. PhD and research overview Deep dive in probabilistic modelling for interaction data Open Questions Appendices Kernel parameters The kernel is responsible for interactions from i to j that respond to previous interactions from j to i. Assumptions: Among all nodes in the network there is global behavior on the reciprocation Hyperparameters η ≥ 0 : size of the excitation jump δ > 0 : constant rate of exponential decay The stationarity condition for the processes is η < δ X Miscouridou 30 / 33
  41. 41. PhD and research overview Deep dive in probabilistic modelling for interaction data Open Questions Appendices Base parameters The base intensity µij is responsible for the interactions from i to j that arise from similar affiliations and link to sparsity and heterogeneity Assumptions A set of positive latent parameters (wi1, . . . , wip) ∈ Rp +, ∀i wik: level of its affiliation to each latent community k = 1, . . . , p. The base rate is given by µij = µji = p X k=1 wikwjk (1) Assortativity: Nodes with high levels of affiliation to the same communities are more likely to interact X Miscouridou 31 / 33
  42. 42. PhD and research overview Deep dive in probabilistic modelling for interaction data Open Questions Appendices Datasets • Email : emails sent within a research institution. T = 803 days, N = 986, E = 24929, I = 332334 • College: private messages on an online social network T = 193 days, N = 1899, E = 20296, I = 59835 . • Math: stack exchange website Math Overflow T = 2350 days, N = 24818, E = 239978, I = 506550 . • The Ubuntu: stack exchange website Ask Ubuntu T = 2613 days, N = 159316, E = 596933, I = 964437 X Miscouridou 32 / 33
  43. 43. PhD and research overview Deep dive in probabilistic modelling for interaction data Open Questions Appendices Link prediction performance across different models MSE between true and predicted number of links Properties of the six different models email college math ubuntu Hawkes-CCRM 10.95 1.88 20.07 29.1 CCRM 12.08 2.90 89.0 36.5 Hawkes-IRM 14.2 3.56 96.9 59.5 Poisson-IRM 31.7 15.7 204.7 79.3 Hawkes 154.8 153.29 220.10 191.39 Poisson ∼ 103 ∼ 104 ∼ 104 ∼ 104 sparsity/ community reciprocity heterogeneity structure X X X X X X X X X X Miscouridou 33 / 33

×