Essentially No Barriers in Neural Network Energy Landscape
Abstract
Training neural networks involves finding minima of a high-dimensional non-convex loss function. Relaxing from linear interpolations, we construct continuous paths between minima of recent neural network architectures on CIFAR10 and CIFAR100. Surprisingly, the paths are essentially flat in both the training and test landscapes. This implies that minima are perhaps best seen as points on a single connected manifold of low loss, rather than as the bottoms of distinct valleys.
Cite
Text
Draxler et al. "Essentially No Barriers in Neural Network Energy Landscape." International Conference on Machine Learning, 2018.Markdown
[Draxler et al. "Essentially No Barriers in Neural Network Energy Landscape." International Conference on Machine Learning, 2018.](https://mlanthology.org/icml/2018/draxler2018icml-essentially/)BibTeX
@inproceedings{draxler2018icml-essentially,
title = {{Essentially No Barriers in Neural Network Energy Landscape}},
author = {Draxler, Felix and Veschgini, Kambis and Salmhofer, Manfred and Hamprecht, Fred},
booktitle = {International Conference on Machine Learning},
year = {2018},
pages = {1309-1318},
volume = {80},
url = {https://mlanthology.org/icml/2018/draxler2018icml-essentially/}
}