Heavy-Tailed Representations, Text Polarity Classification & Data Augmentation
Abstract
The dominant approaches to text representation in natural language rely on learning embeddings on massive corpora which have convenient properties such as compositionality and distance preservation. In this paper, we develop a novel method to learn a heavy-tailed embedding with desirable regularity properties regarding the distributional tails, which allows to analyze the points far away from the distribution bulk using the framework of multivariate extreme value theory. In particular, a classifier dedicated to the tails of the proposed embedding is obtained which exhibits a scale invariance property exploited in a novel text generation method for label preserving dataset augmentation. Experiments on synthetic and real text data show the relevance of the proposed framework and confirm that this method generates meaningful sentences with controllable attribute, e.g. positive or negative sentiments.
Cite
Text
Jalalzai et al. "Heavy-Tailed Representations, Text Polarity Classification & Data Augmentation." Neural Information Processing Systems, 2020.Markdown
[Jalalzai et al. "Heavy-Tailed Representations, Text Polarity Classification & Data Augmentation." Neural Information Processing Systems, 2020.](https://mlanthology.org/neurips/2020/jalalzai2020neurips-heavytailed/)BibTeX
@inproceedings{jalalzai2020neurips-heavytailed,
title = {{Heavy-Tailed Representations, Text Polarity Classification & Data Augmentation}},
author = {Jalalzai, Hamid and Colombo, Pierre and Clavel, Chloé and Gaussier, Eric and Varni, Giovanna and Vignon, Emmanuel and Sabourin, Anne},
booktitle = {Neural Information Processing Systems},
year = {2020},
url = {https://mlanthology.org/neurips/2020/jalalzai2020neurips-heavytailed/}
}