Diese Präsentation wurde erfolgreich gemeldet.
Wir verwenden Ihre LinkedIn Profilangaben und Informationen zu Ihren Aktivitäten, um Anzeigen zu personalisieren und Ihnen relevantere Inhalte anzuzeigen. Sie können Ihre Anzeigeneinstellungen jederzeit ändern.

Tutorial on Deep learning and Applications

23.105 Aufrufe

Veröffentlicht am

In this presentation, I would like to review basis techniques, models, and applications in deep learning. Hope you find the slides are interesting. Further information about my research can be found at "https://sites.google.com/site/ihaiphan/."

NhatHai Phan
CIS Department,
University of Oregon, Eugene, OR

Veröffentlicht in: Daten & Analysen

Tutorial on Deep learning and Applications

  1. 1. Tutorial on Deep Learning and Applications Hai Phan AIM Lab, University of Oregon 1Includes slide material sourced from Yoshua Bengio, Geoff Hinton, Yann LeCun, Andrew Ng, Honglak Lee, and Marc’Aurelio Ranzato
  2. 2. Outline • Deep learning – Greedy level-wise training (supervised learning) – Restricted Boltzmann machine (RBM) – Deep belief networks – Stacked autoassociators – Deep Boltzmann machines • Applications – Human motion modeling – Vision – Language 2
  3. 3. Motivation: why go deep? • Deep Architectures can be representationally efficient • Deep representation might allow for a hierarchy of representations – Comprehensibility • Multiple levels of latent variables allow combinatorial sharing of statistical strength • Deep architectures work well (vision, audio, NLP, etc.)! 3
  4. 4. Motivation 4
  5. 5. 5
  6. 6. 6
  7. 7. 7
  8. 8. 8
  9. 9. 9
  10. 10. 10
  11. 11. Outline • Deep learning – Greedy level-wise training (supervised learning) – Restricted Boltzmann machine (RBM) – Deep belief networks – Stacked autoassociators – Deep Boltzmann machines • Applications – Human motion modeling – Vision – Language 11
  12. 12. Restricted Boltzmann Machine (RBM) • Binary visible units – Pixels • Binary hidden units – Feature detectors 12
  13. 13. Training RBM • Free energy – inspired from physics • 13
  14. 14. Different Types of Unit • Gaussian visible units • Gaussian visible and hidden units • Softmax units 14
  15. 15. Outline • Deep learning – Greedy level-wise training (supervised learning) – Restricted Boltzmann machine (RBM) – Deep belief networks – Stacked autoassociators – Deep Boltzmann machines • Applications – Human motion modeling – Vision – Language 15
  16. 16. 16
  17. 17. 17
  18. 18. 18
  19. 19. 19
  20. 20. 20
  21. 21. Outline • Deep learning – Greedy level-wise training (supervised learning) – Restricted Boltzmann machine (RBM) – Deep belief networks – Stacked autoassociators – Deep Boltzmann machines • Applications – Human motion modeling – Vision – Language 21
  22. 22. Stacked Autoassociators • Reconstruct x 22
  23. 23. Stacked Autoassociators 23
  24. 24. Benchmarks 24
  25. 25. Benchmarks • Consistent improvement over Neural Net 25
  26. 26. Outline • Deep learning – Greedy level-wise training (supervised learning) – Restricted Boltzmann machine (RBM) – Deep belief networks – Stacked autoassociators – Deep Boltzmann machines • Applications – Human motion modeling – Vision – Language 26
  27. 27. 27
  28. 28. 28
  29. 29. 29
  30. 30. Greedy Layer-wise Retraining of DBM’s 30
  31. 31. Discriminative Fine-tuning of DBM’s 31
  32. 32. 32
  33. 33. 33
  34. 34. 34
  35. 35. Outline • Deep learning – Greedy level-wise training (supervised learning) – Restricted Boltzmann machine (RBM) – Deep belief networks – Stacked autoassociators – Deep Boltzmann machines • Applications – Human motion modeling – Vision – Language 35
  36. 36. Conditional Restricted Boltzmann Machines (CRBM) Auto-regression 36
  37. 37. Conditional Restricted Boltzmann Machines (CRBM) • http://www.uoguelph.ca/~gwtaylor/thesis/4/ 37
  38. 38. Outline • Deep learning – Greedy level-wise training (supervised learning) – Restricted Boltzmann machine (RBM) – Deep belief networks – Stacked autoassociators – Deep Boltzmann machines • Applications – Human motion modeling – Vision – Language 38
  39. 39. 39
  40. 40. Convolutional Deep Belief Networks 40
  41. 41. 41
  42. 42. Outline • Deep learning – Greedy level-wise training (supervised learning) – Restricted Boltzmann machine (RBM) – Deep belief networks – Stacked autoassociators – Deep Boltzmann machines • Applications – Human motion modeling – Vision – Language 42
  43. 43. 43
  44. 44. 44
  45. 45. 45
  46. 46. 46
  47. 47. 47
  48. 48. Thank you! 48

×