Linear Convergence of a Frank-Wolfe Type Algorithm over Trace-Norm Balls
Abstract
We propose a rank-k variant of the classical Frank-Wolfe algorithm to solve convex optimization over a trace-norm ball. Our algorithm replaces the top singular-vector computation (1-SVD) in Frank-Wolfe with a top-k singular-vector computation (k-SVD), which can be done by repeatedly applying 1-SVD k times. Alternatively, our algorithm can be viewed as a rank-k restricted version of projected gradient descent. We show that our algorithm has a linear convergence rate when the objective function is smooth and strongly convex, and the optimal solution has rank at most k. This improves the convergence rate and the total time complexity of the Frank-Wolfe method and its variants.
Cite
Text
Allen-Zhu et al. "Linear Convergence of a Frank-Wolfe Type Algorithm over Trace-Norm Balls." Neural Information Processing Systems, 2017.Markdown
[Allen-Zhu et al. "Linear Convergence of a Frank-Wolfe Type Algorithm over Trace-Norm Balls." Neural Information Processing Systems, 2017.](https://mlanthology.org/neurips/2017/allenzhu2017neurips-linear/)BibTeX
@inproceedings{allenzhu2017neurips-linear,
title = {{Linear Convergence of a Frank-Wolfe Type Algorithm over Trace-Norm Balls}},
author = {Allen-Zhu, Zeyuan and Hazan, Elad and Hu, Wei and Li, Yuanzhi},
booktitle = {Neural Information Processing Systems},
year = {2017},
pages = {6191-6200},
url = {https://mlanthology.org/neurips/2017/allenzhu2017neurips-linear/}
}