Revisiting the Landscape of Matrix Factorization
Abstract
Prior work has shown that low-rank matrix factorization has infinitely many critical points, each of which is either a global minimum or a (strict) saddle point. We revisit this problem and provide simple, intuitive proofs of a set of extended results for low-rank and general-rank problems. We couple our investigation with a known invariant manifold M0 of gradient flow. This restriction admits a uniform negative upper bound on the least eigenvalue of the Hessian map at all strict saddles in M0. The bound depends on the size of the nonzero singular values and the separation between distinct singular values of the matrix to be factorized.
Cite
Text
Valavi et al. "Revisiting the Landscape of Matrix Factorization." Artificial Intelligence and Statistics, 2020.Markdown
[Valavi et al. "Revisiting the Landscape of Matrix Factorization." Artificial Intelligence and Statistics, 2020.](https://mlanthology.org/aistats/2020/valavi2020aistats-revisiting/)BibTeX
@inproceedings{valavi2020aistats-revisiting,
title = {{Revisiting the Landscape of Matrix Factorization}},
author = {Valavi, Hossein and Liu, Sulin and Ramadge, Peter},
booktitle = {Artificial Intelligence and Statistics},
year = {2020},
pages = {1629-1638},
volume = {108},
url = {https://mlanthology.org/aistats/2020/valavi2020aistats-revisiting/}
}