Implicit Regularization in Matrix Factorization
Abstract
We study implicit regularization when optimizing an underdetermined quadratic objective over a matrix $X$ with gradient descent on a factorization of X. We conjecture and provide empirical and theoretical evidence that with small enough step sizes and initialization close enough to the origin, gradient descent on a full dimensional factorization converges to the minimum nuclear norm solution.
Cite
Text
Gunasekar et al. "Implicit Regularization in Matrix Factorization." Neural Information Processing Systems, 2017.Markdown
[Gunasekar et al. "Implicit Regularization in Matrix Factorization." Neural Information Processing Systems, 2017.](https://mlanthology.org/neurips/2017/gunasekar2017neurips-implicit/)BibTeX
@inproceedings{gunasekar2017neurips-implicit,
title = {{Implicit Regularization in Matrix Factorization}},
author = {Gunasekar, Suriya and Woodworth, Blake E and Bhojanapalli, Srinadh and Neyshabur, Behnam and Srebro, Nati},
booktitle = {Neural Information Processing Systems},
year = {2017},
pages = {6151-6159},
url = {https://mlanthology.org/neurips/2017/gunasekar2017neurips-implicit/}
}