ML Anthology

Deep Learners Benefit More from Out-of-Distribution Examples

Yoshua Bengio, Frédéric Bastien, Arnaud Bergeron, Nicolas Boulanger–Lewandowski, Thomas Breuel, Youssouf Chherawala, Moustapha Cisse, Myriam Côté, Dumitru Erhan, Jeremy Eustache, Xavier Glorot, Xavier Muller, Sylvain Pannetier Lebeuf, Razvan Pascanu, Salah Rifai, François Savard, Guillaume Sicard
AISTATS 2011 pp. 164-172
/aistats/2011/bengio2011aistats-deep/

Abstract

Recent theoretical and empirical work in statistical machine learning has demonstrated the potential of learning algorithms for deep architectures, i.e., function classes obtained by composing multiple levels of representation. The hypothesis evaluated here is that intermediate levels of representation, because they can be shared across tasks and examples from different but related distributions, can yield even more benefits. Comparative experiments were performed on a large-scale handwritten character recognition setting with 62 classes (upper case, lower case, digits), using both a multi-task setting and perturbed examples in order to obtain out-of-distribution examples. The results agree with the hypothesis, and show that a deep learner did beat previously published results and reached human-level performance.

PDF AISTATS Semantic Scholar

Cite

Text

Bengio et al. "Deep Learners Benefit More from Out-of-Distribution Examples." Proceedings of the Fourteenth International Conference on Artificial Intelligence and Statistics, 2011.

Markdown

[Bengio et al. "Deep Learners Benefit More from Out-of-Distribution Examples." Proceedings of the Fourteenth International Conference on Artificial Intelligence and Statistics, 2011.](https://mlanthology.org/aistats/2011/bengio2011aistats-deep/)

BibTeX

@inproceedings{bengio2011aistats-deep,
  title     = {{Deep Learners Benefit More from Out-of-Distribution Examples}},
  author    = {Bengio, Yoshua and Bastien, Frédéric and Bergeron, Arnaud and Boulanger–Lewandowski, Nicolas and Breuel, Thomas and Chherawala, Youssouf and Cisse, Moustapha and Côté, Myriam and Erhan, Dumitru and Eustache, Jeremy and Glorot, Xavier and Muller, Xavier and Pannetier Lebeuf, Sylvain and Pascanu, Razvan and Rifai, Salah and Savard, François and Sicard, Guillaume},
  booktitle = {Proceedings of the Fourteenth International Conference on Artificial Intelligence and Statistics},
  year      = {2011},
  pages     = {164-172},
  volume    = {15},
  url       = {https://mlanthology.org/aistats/2011/bengio2011aistats-deep/}
}