Proximal Gradient Algorithm with Momentum and Flexible Parameter Restart for Nonconvex Optimization
Abstract
Various types of parameter restart schemes have been proposed for proximal gradient algorithm with momentum to facilitate their convergence in convex optimization. However, under parameter restart, the convergence of proximal gradient algorithm with momentum remains obscure in nonconvex optimization. In this paper, we propose a novel proximal gradient algorithm with momentum and parameter restart for solving nonconvex and nonsmooth problems. Our algorithm is designed to 1) allow for adopting flexible parameter restart schemes that cover many existing ones; 2) have a global sub-linear convergence rate in nonconvex and nonsmooth optimization; and 3) have guaranteed convergence to a critical point and have various types of asymptotic convergence rates depending on the parameterization of local geometry in nonconvex and nonsmooth optimization. Numerical experiments demonstrate the convergence and effectiveness of our proposed algorithm.
Cite
Text
Zhou et al. "Proximal Gradient Algorithm with Momentum and Flexible Parameter Restart for Nonconvex Optimization." International Joint Conference on Artificial Intelligence, 2020. doi:10.24963/IJCAI.2020/201Markdown
[Zhou et al. "Proximal Gradient Algorithm with Momentum and Flexible Parameter Restart for Nonconvex Optimization." International Joint Conference on Artificial Intelligence, 2020.](https://mlanthology.org/ijcai/2020/zhou2020ijcai-proximal/) doi:10.24963/IJCAI.2020/201BibTeX
@inproceedings{zhou2020ijcai-proximal,
title = {{Proximal Gradient Algorithm with Momentum and Flexible Parameter Restart for Nonconvex Optimization}},
author = {Zhou, Yi and Wang, Zhe and Ji, Kaiyi and Liang, Yingbin and Tarokh, Vahid},
booktitle = {International Joint Conference on Artificial Intelligence},
year = {2020},
pages = {1445-1451},
doi = {10.24963/IJCAI.2020/201},
url = {https://mlanthology.org/ijcai/2020/zhou2020ijcai-proximal/}
}