21. 補足:関連するかもしれない研究
• Generalization in Deep Networks: The Role of Distance from Initialization [Vaishnavh
Nagarajan+, NIPSW2017] http://www.cs.cmu.edu/~vaishnan/papers/nips17_dltp.pdf
• Towards Understanding the Role of Over-Parametrization in Generalization of Neural
Networks [Behnam Neyshabur+, arXiv2018] https://arxiv.org/abs/1805.12076
ランダム初期化時と学習後の重みの値の距離に基づいて汎化誤差を分析
• DropBack: Continuous Pruning During Training [Maximilian Golub+, arXiv2018]
https://arxiv.org/abs/1806.06949
• Intriguing Properties of Randomly Weighted Networks: Generalizing While Learning Next
to Nothing [Amir Rosenfeld+,arXiv2018] https://arxiv.org/abs/1802.00844
重みの大半をランダム初期値で固定し、一部の重みのみを更新
• Insights on representational similarity in neural networks with canonical correlation
[Ari S. Morcos+, arXiv2018] https://arxiv.org/abs/1806.05759
大きなネットワークほど似た解に収束する