Characterizing Implicit Bias in Terms of Optimization Geometry
Abstract
We study the bias of generic optimization methods, including Mirror Descent, Natural Gradient Descent and Steepest Descent with respect to different potentials and norms, when optimizing underdetermined linear models or separable linear classification problems. We ask the question of whether the global minimum (among the many possible global minima) reached by optimization can be characterized in terms of the potential or norm, and indecently of hyper-parameter choices, such as stepsize and momentum.
Cite
Text
Gunasekar et al. "Characterizing Implicit Bias in Terms of Optimization Geometry." International Conference on Machine Learning, 2018.Markdown
[Gunasekar et al. "Characterizing Implicit Bias in Terms of Optimization Geometry." International Conference on Machine Learning, 2018.](https://mlanthology.org/icml/2018/gunasekar2018icml-characterizing/)BibTeX
@inproceedings{gunasekar2018icml-characterizing,
title = {{Characterizing Implicit Bias in Terms of Optimization Geometry}},
author = {Gunasekar, Suriya and Lee, Jason and Soudry, Daniel and Srebro, Nathan},
booktitle = {International Conference on Machine Learning},
year = {2018},
pages = {1832-1841},
volume = {80},
url = {https://mlanthology.org/icml/2018/gunasekar2018icml-characterizing/}
}