Connectedness of Loss Landscapes via the Lens of Morse Theory
Abstract
Mode connectivity is a recently discovered property of neural networks stating that two weight configurations of small loss can usually be connected by a path of small loss. The mode connectivity property is interesting practically as it has applications to design of optimizers with better generalization properties and various other applied topics as well as theoretically as it suggests that loss landscapes of deep networks have very nice properties even though they are known to be highly non-convex. The goal of this work is to study connectedness of loss landscapes via the lens of Morse theory. A brief introduction to Morse theory is provided.
Cite
Text
Akhtiamov and Thomson. "Connectedness of Loss Landscapes via the Lens of Morse Theory." NeurIPS 2022 Workshops: NeurReps, 2022.Markdown
[Akhtiamov and Thomson. "Connectedness of Loss Landscapes via the Lens of Morse Theory." NeurIPS 2022 Workshops: NeurReps, 2022.](https://mlanthology.org/neuripsw/2022/akhtiamov2022neuripsw-connectedness/)BibTeX
@inproceedings{akhtiamov2022neuripsw-connectedness,
title = {{Connectedness of Loss Landscapes via the Lens of Morse Theory}},
author = {Akhtiamov, Danil and Thomson, Matt},
booktitle = {NeurIPS 2022 Workshops: NeurReps},
year = {2022},
url = {https://mlanthology.org/neuripsw/2022/akhtiamov2022neuripsw-connectedness/}
}