Fast Stochastic Variance Reduced ADMM for Stochastic Composition Optimization
Abstract
We consider the stochastic composition optimization problem proposed in \cite{wang2017stochastic}, which has applications ranging from estimation to statistical and machine learning. We propose the first ADMM-based algorithm named com-SVR-ADMM, and show that com-SVR-ADMM converges linearly for strongly convex and Lipschitz smooth objectives, and has a convergence rate of $O( \log S/S)$, which improves upon the $O(S^{-4/9})$ rate in \cite{wang2016accelerating} when the objective is convex and Lipschitz smooth. Moreover, com-SVR-ADMM possesses a rate of $O(1/\sqrt{S})$ when the objective is convex but without Lipschitz smoothness. We also conduct experiments and show that it outperforms existing algorithms.
Cite
Text
Yu and Huang. "Fast Stochastic Variance Reduced ADMM for Stochastic Composition Optimization." International Joint Conference on Artificial Intelligence, 2017. doi:10.24963/IJCAI.2017/470Markdown
[Yu and Huang. "Fast Stochastic Variance Reduced ADMM for Stochastic Composition Optimization." International Joint Conference on Artificial Intelligence, 2017.](https://mlanthology.org/ijcai/2017/yu2017ijcai-fast/) doi:10.24963/IJCAI.2017/470BibTeX
@inproceedings{yu2017ijcai-fast,
title = {{Fast Stochastic Variance Reduced ADMM for Stochastic Composition Optimization}},
author = {Yu, Yue and Huang, Longbo},
booktitle = {International Joint Conference on Artificial Intelligence},
year = {2017},
pages = {3364-3370},
doi = {10.24963/IJCAI.2017/470},
url = {https://mlanthology.org/ijcai/2017/yu2017ijcai-fast/}
}