Long-Tailed Learning Requires Feature Learning
Abstract
We propose a simple data model inspired from natural data such as text or images, and use it to study the importance of learning features in order to achieve good generalization. Our data model follows a long-tailed distribution in the sense that some rare and uncommon subcategories have few representatives in the training set. In this context we provide evidence that a learner succeeds if and only if it identifies the correct features, and moreover derive non-asymptotic generalization error bounds that precisely quantify the penalty that one must pay for not learning features.
Cite
Text
Laurent et al. "Long-Tailed Learning Requires Feature Learning." International Conference on Learning Representations, 2023.Markdown
[Laurent et al. "Long-Tailed Learning Requires Feature Learning." International Conference on Learning Representations, 2023.](https://mlanthology.org/iclr/2023/laurent2023iclr-longtailed/)BibTeX
@inproceedings{laurent2023iclr-longtailed,
title = {{Long-Tailed Learning Requires Feature Learning}},
author = {Laurent, Thomas and von Brecht, James and Bresson, Xavier},
booktitle = {International Conference on Learning Representations},
year = {2023},
url = {https://mlanthology.org/iclr/2023/laurent2023iclr-longtailed/}
}