Relationships Between the a Priori and a Posteriori Errors in Nonlinear Adaptive Neural Filters
Abstract
The lower bounds for the a posteriori prediction error of a nonlinear predictor realized as a neural network are provided. These are obtained for a priori adaptation and a posteriori error networks with sigmoid nonlinearities trained by gradient-descent learning algorithms. A contractivity condition is imposed on a nonlinear activation function of a neuron so that the a posteriori prediction error is smaller in magnitude than the corresponding a priori one. Furthermore, an upper bound is imposed on the learning rate η so that the approach is feasible. The analysis is undertaken for both feedforward and recurrent nonlinear predictors realized as neural networks.
Cite
Text
Mandic and Chambers. "Relationships Between the a Priori and a Posteriori Errors in Nonlinear Adaptive Neural Filters." Neural Computation, 2000. doi:10.1162/089976600300015358Markdown
[Mandic and Chambers. "Relationships Between the a Priori and a Posteriori Errors in Nonlinear Adaptive Neural Filters." Neural Computation, 2000.](https://mlanthology.org/neco/2000/mandic2000neco-relationships/) doi:10.1162/089976600300015358BibTeX
@article{mandic2000neco-relationships,
title = {{Relationships Between the a Priori and a Posteriori Errors in Nonlinear Adaptive Neural Filters}},
author = {Mandic, Danilo P. and Chambers, Jonathon A.},
journal = {Neural Computation},
year = {2000},
pages = {1285-1292},
doi = {10.1162/089976600300015358},
volume = {12},
url = {https://mlanthology.org/neco/2000/mandic2000neco-relationships/}
}