Convergence Rates for the Stochastic Gradient Descent Method for Non-Convex Objective Functions
Abstract
We prove the convergence to minima and estimates on the rate of convergence for the stochastic gradient descent method in the case of not necessarily locally convex nor contracting objective functions. In particular, the analysis relies on a quantitative use of mini-batches to control the loss of iterates to non-attracted regions. The applicability of the results to simple objective functions arising in machine learning is shown.
Cite
Text
Fehrman et al. "Convergence Rates for the Stochastic Gradient Descent Method for Non-Convex Objective Functions." Journal of Machine Learning Research, 2020.Markdown
[Fehrman et al. "Convergence Rates for the Stochastic Gradient Descent Method for Non-Convex Objective Functions." Journal of Machine Learning Research, 2020.](https://mlanthology.org/jmlr/2020/fehrman2020jmlr-convergence/)BibTeX
@article{fehrman2020jmlr-convergence,
title = {{Convergence Rates for the Stochastic Gradient Descent Method for Non-Convex Objective Functions}},
author = {Fehrman, Benjamin and Gess, Benjamin and Jentzen, Arnulf},
journal = {Journal of Machine Learning Research},
year = {2020},
pages = {1-48},
volume = {21},
url = {https://mlanthology.org/jmlr/2020/fehrman2020jmlr-convergence/}
}