ML Anthology

A Realizable Learning Task Which Exhibits Overfitting

Siegfried Bös
NeurIPS 1995 pp. 218-224
/neurips/1995/bos1995neurips-realizable/

Abstract

In this paper we examine a perceptron learning task. The task is realizable since it is provided by another perceptron with identi(cid:173) cal architecture. Both perceptrons have nonlinear sigmoid output functions. The gain of the output function determines the level of nonlinearity of the learning task. It is observed that a high level of nonlinearity leads to overfitting. We give an explanation for this rather surprising observation and develop a method to avoid the overfitting. This method has two possible interpretations, one is learning with noise, the other cross-validated early stopping.

PDF NeurIPS Semantic Scholar

Cite

Text

Bös. "A Realizable Learning Task Which Exhibits Overfitting." Neural Information Processing Systems, 1995.

Markdown

[Bös. "A Realizable Learning Task Which Exhibits Overfitting." Neural Information Processing Systems, 1995.](https://mlanthology.org/neurips/1995/bos1995neurips-realizable/)

BibTeX

@inproceedings{bos1995neurips-realizable,
  title     = {{A Realizable Learning Task Which Exhibits Overfitting}},
  author    = {Bös, Siegfried},
  booktitle = {Neural Information Processing Systems},
  year      = {1995},
  pages     = {218-224},
  url       = {https://mlanthology.org/neurips/1995/bos1995neurips-realizable/}
}