On-Line Learning from Finite Training Sets in Nonlinear Networks
Abstract
Online learning is one of the most common forms of neural net(cid:173) work training. We present an analysis of online learning from finite training sets for non-linear networks (namely, soft-committee ma(cid:173) chines), advancing the theory to more realistic learning scenarios. Dynamical equations are derived for an appropriate set of order parameters; these are exact in the limiting case of either linear networks or infinite training sets. Preliminary comparisons with simulations suggest that the theory captures some effects of finite training sets, but may not yet account correctly for the presence of local minima.
Cite
Text
Sollich and Barber. "On-Line Learning from Finite Training Sets in Nonlinear Networks." Neural Information Processing Systems, 1997.Markdown
[Sollich and Barber. "On-Line Learning from Finite Training Sets in Nonlinear Networks." Neural Information Processing Systems, 1997.](https://mlanthology.org/neurips/1997/sollich1997neurips-online/)BibTeX
@inproceedings{sollich1997neurips-online,
title = {{On-Line Learning from Finite Training Sets in Nonlinear Networks}},
author = {Sollich, Peter and Barber, David},
booktitle = {Neural Information Processing Systems},
year = {1997},
pages = {357-363},
url = {https://mlanthology.org/neurips/1997/sollich1997neurips-online/}
}