On the Interaction Between Norm and Dimensionality: Multiple Regimes in Learning
Abstract
A learning problem might have several measures of complexity (e.g., norm and dimensionality) that affect the generalization error. What is the interaction between these complexities? Dimension-free learning theory bounds and parametric asymptotic analyses each provide a partial picture of the full learning curve. In this paper, we use high-dimensional asymptotics on two classical problems---mean estimation and linear regression---to explore the learning curve more completely. We show that these curves exhibit multiple regimes, where in each regime, the excess risk is controlled by a subset of the problem complexities.
Cite
Text
Liang and Srebro. "On the Interaction Between Norm and Dimensionality: Multiple Regimes in Learning." International Conference on Machine Learning, 2010.Markdown
[Liang and Srebro. "On the Interaction Between Norm and Dimensionality: Multiple Regimes in Learning." International Conference on Machine Learning, 2010.](https://mlanthology.org/icml/2010/liang2010icml-interaction/)BibTeX
@inproceedings{liang2010icml-interaction,
title = {{On the Interaction Between Norm and Dimensionality: Multiple Regimes in Learning}},
author = {Liang, Percy and Srebro, Nati},
booktitle = {International Conference on Machine Learning},
year = {2010},
pages = {647-654},
url = {https://mlanthology.org/icml/2010/liang2010icml-interaction/}
}