6. 6/33
変分推論の課題 2. 近似分布の柔軟性/表現力
Ø Mean Field VI
Ø Amortized VI [Kingma+ ICLR14]
Ø VI with Normalizing Flows [Rezende+ ICML15]
分解仮定
パラメータ共有
真の事後分布を十分に近似できる柔軟性を持つのか
32. 32/33
参考文献
• [Bishop 06] Bishop, C. M. (2006). Pattern recognition and machine learning. springer.
• [Dieng+ NIPS17] Dieng, A. B., Tran, D., Ranganath, R., Paisley, J., & Blei, D. (2017).
Variational Inference via χ Upper Bound Minimization. In Advances in Neural
Information Processing Systems (pp. 2732-2741).
• [Ermon+ 18] Ermon S. & Grover A. (2018). Normalizing flow models (CS236 course
notes). https://deepgenerativemodels.github.io/notes/flow/.
• [Grover+ AAAI18] Grover, A., Dhar, M., & Ermon, S. (2018, April). Flow-GAN:
Combining maximum likelihood and adversarial learning in generative models. In Thirty-
Second AAAI Conference on Artificial Intelligence.
• [Kingma+ ICLR14] Kingma, D. P., & Welling, M. (2014). Auto-encoding variational
Bayes. arXiv preprint arXiv:1312.6114.
• [Li+ NIPS16] Li, Y., & Turner, R. E. (2016). Rényi divergence variational inference. In
Advances in Neural Information Processing Systems (pp. 1073-1081).
• [Rezende+ ICML15] Rezende, D. J., & Mohamed, S. (2015). Variational inference with
normalizing flows. arXiv preprint arXiv:1505.05770.
• [Rumbos 08] Rumbos A. J. (2008). Probability lecture notes.
https://pages.pomona.edu/~ajr04747/Spring2008/Math151/Math151NotesSpring08.pd
f.
33. 33/33
その他資料
• おすすめ (変分ベイズ/深層生成モデルのチュートリアル)
• Variational Bayes and beyond: Bayesian inference for big data (ICML2018),
http://www.tamarabroderick.com/tutorial_2018_icml.html
• Variational Inference: Foundations and Innovations,
http://www.cs.columbia.edu/~blei/talks/Blei_VI_tutorial.pdf
• Tutorial on Deep Generative Models (IJCAI-ECAI 2018),
https://drive.google.com/file/d/1uwvXkKfrOjYsRKLO7RK4KbvpWmu_YPN_/view?usp=shari
ng
• Tutorial on Deep Generative Models (UAI 2017),
https://www.shakirm.com/slides/DeepGenModelsTutorial.pdf
• その他
• Normalizing Flows Tutorial, Part 1: Distributions and Determinants,
https://blog.evjang.com/2018/01/nf1.html
• Flow-based Deep Generative Models, https://lilianweng.github.io/lil-log/2018/10/13/flow-
based-deep-generative-models.html
• Variational Inference with Normalizing Flowsを読んだのでメモ,
http://peluigi.hatenablog.com/entry/2018/07/12/140528
• Up to GLOW, https://www.slideshare.net/ShunsukeNAKATSUKA1/up-to-glow
• DL輪読会 Flow-based Deep Generative Models,
https://www.slideshare.net/DeepLearningJP2016/dlflowbased-deep-generative-models