ML Anthology

A Numerical Study on Learning Curves in Stochastic Multilayer Feedforward Networks

Klaus-Robert Müller, Michael Finke, Noboru Murata, Klaus Schulten, Shun-ichi Amari
NeCo 1996 pp. 1085-1106
doi:10.1162/NECO.1996.8.5.1085 /neco/1996/muller1996neco-numerical/

Abstract

The universal asymptotic scaling laws proposed by Amari et al. are studied in large scale simulations using a CM5. Small stochastic multilayer feedforward networks trained with backpropagation are investigated. In the range of a large number of training patterns t, the asymptotic generalization error scales as 1/t as predicted. For a medium range t a faster 1/t2 scaling is observed. This effect is explained by using higher order corrections of the likelihood expansion. It is shown for small t that the scaling law changes drastically, when the network undergoes a transition from strong overfitting to effective learning.

NeCo Semantic Scholar

Cite

Text

Müller et al. "A Numerical Study on Learning Curves in Stochastic Multilayer Feedforward Networks." Neural Computation, 1996. doi:10.1162/NECO.1996.8.5.1085

Markdown

[Müller et al. "A Numerical Study on Learning Curves in Stochastic Multilayer Feedforward Networks." Neural Computation, 1996.](https://mlanthology.org/neco/1996/muller1996neco-numerical/) doi:10.1162/NECO.1996.8.5.1085

BibTeX

@article{muller1996neco-numerical,
  title     = {{A Numerical Study on Learning Curves in Stochastic Multilayer Feedforward Networks}},
  author    = {Müller, Klaus-Robert and Finke, Michael and Murata, Noboru and Schulten, Klaus and Amari, Shun-ichi},
  journal   = {Neural Computation},
  year      = {1996},
  pages     = {1085-1106},
  doi       = {10.1162/NECO.1996.8.5.1085},
  volume    = {8},
  url       = {https://mlanthology.org/neco/1996/muller1996neco-numerical/}
}