Accelerated Multiplicative Weights Update Avoids Saddle Points Almost Always
Abstract
We consider nonconvex optimization problem with constraint that is a product of simplices. A commonly used algorithm in solving this type of problem is the Multiplicative Weights Update (MWU), an algorithm that is widely used in game theory, machine learning and multi agent systems. Despite it has been known that MWU avoids saddle points, there is a question that remains unaddressed: ``Is there an accelerated version of MWU that avoids saddle points provably?'' In this paper we provide a positive answer to above question. We provide an accelerated MWU based on Riemannian Accelerated Gradient Descent, and prove that the Riemannian Accelerated Gradient Descent, thus the accelerated MWU, avoid saddle points.
Cite
Text
Feng et al. "Accelerated Multiplicative Weights Update Avoids Saddle Points Almost Always." International Joint Conference on Artificial Intelligence, 2022. doi:10.24963/IJCAI.2022/252Markdown
[Feng et al. "Accelerated Multiplicative Weights Update Avoids Saddle Points Almost Always." International Joint Conference on Artificial Intelligence, 2022.](https://mlanthology.org/ijcai/2022/feng2022ijcai-accelerated/) doi:10.24963/IJCAI.2022/252BibTeX
@inproceedings{feng2022ijcai-accelerated,
title = {{Accelerated Multiplicative Weights Update Avoids Saddle Points Almost Always}},
author = {Feng, Yi and Panageas, Ioannis and Wang, Xiao},
booktitle = {International Joint Conference on Artificial Intelligence},
year = {2022},
pages = {1811-1817},
doi = {10.24963/IJCAI.2022/252},
url = {https://mlanthology.org/ijcai/2022/feng2022ijcai-accelerated/}
}