On Connected Sublevel Sets in Deep Learning
Abstract
This paper shows that every sublevel set of the loss function of a class of deep over-parameterized neural nets with piecewise linear activation functions is connected and unbounded. This implies that the loss has no bad local valleys and all of its global minima are connected within a unique and potentially very large global valley.
Cite
Text
Nguyen. "On Connected Sublevel Sets in Deep Learning." International Conference on Machine Learning, 2019.Markdown
[Nguyen. "On Connected Sublevel Sets in Deep Learning." International Conference on Machine Learning, 2019.](https://mlanthology.org/icml/2019/nguyen2019icml-connected/)BibTeX
@inproceedings{nguyen2019icml-connected,
title = {{On Connected Sublevel Sets in Deep Learning}},
author = {Nguyen, Quynh},
booktitle = {International Conference on Machine Learning},
year = {2019},
pages = {4790-4799},
volume = {97},
url = {https://mlanthology.org/icml/2019/nguyen2019icml-connected/}
}