Revisit Last-Iterate Convergence of mSGD Under Milder Requirement on Step Size
Abstract
Understanding convergence of SGD-based optimization algorithms can help deal with enormous machine learning problems. To ensure last-iterate convergence of SGD and momentum-based SGD (mSGD), the existing studies usually constrain the step size $\epsilon_{n}$ to decay as $\sum_{n=1}^{+\infty}\epsilon_{n}^{2}0$ by removing the common requirement in the literature on the strong convexity of the loss function. Some experiments are given to illustrate the developed results.
Cite
Text
Jin et al. "Revisit Last-Iterate Convergence of mSGD Under Milder Requirement on Step Size." Neural Information Processing Systems, 2022.Markdown
[Jin et al. "Revisit Last-Iterate Convergence of mSGD Under Milder Requirement on Step Size." Neural Information Processing Systems, 2022.](https://mlanthology.org/neurips/2022/jin2022neurips-revisit/)BibTeX
@inproceedings{jin2022neurips-revisit,
title = {{Revisit Last-Iterate Convergence of mSGD Under Milder Requirement on Step Size}},
author = {Jin, Ruinan and He, Xingkang and Chen, Lang and Cheng, Difei and Gupta, Vijay},
booktitle = {Neural Information Processing Systems},
year = {2022},
url = {https://mlanthology.org/neurips/2022/jin2022neurips-revisit/}
}