Backpropagation Convergence via Deterministic Nonmonotone Perturbed Minimization
Abstract
The fundamental backpropagation (BP) algorithm for training ar(cid:173) tificial neural networks is cast as a deterministic nonmonotone per(cid:173) turbed gradient method. Under certain natural assumptions, such as the series of learning rates diverging while the series of their squares converging, it is established that every accumulation point of the online BP iterates is a stationary point of the BP error func(cid:173) tion. The results presented cover serial and parallel online BP, modified BP with a momentum term, and BP with weight decay.
Cite
Text
Mangasarian and Solodov. "Backpropagation Convergence via Deterministic Nonmonotone Perturbed Minimization." Neural Information Processing Systems, 1993.Markdown
[Mangasarian and Solodov. "Backpropagation Convergence via Deterministic Nonmonotone Perturbed Minimization." Neural Information Processing Systems, 1993.](https://mlanthology.org/neurips/1993/mangasarian1993neurips-backpropagation/)BibTeX
@inproceedings{mangasarian1993neurips-backpropagation,
title = {{Backpropagation Convergence via Deterministic Nonmonotone Perturbed Minimization}},
author = {Mangasarian, O. L. and Solodov, M. V.},
booktitle = {Neural Information Processing Systems},
year = {1993},
pages = {383-390},
url = {https://mlanthology.org/neurips/1993/mangasarian1993neurips-backpropagation/}
}