Generalization by Weight-Elimination with Application to Forecasting
Abstract
Inspired by the information theoretic idea of minimum description length, we add a term to the back propagation cost function that penalizes network complexity. We give the details of the procedure, called weight-elimination, describe its dynamics, and clarify the meaning of the parameters involved. From a Bayesian perspective, the complexity term can be usefully interpreted as an assumption about prior distribution of the weights. We use this procedure to predict the sunspot time series and the notoriously noisy series of currency exchange rates.
Cite
Text
Weigend et al. "Generalization by Weight-Elimination with Application to Forecasting." Neural Information Processing Systems, 1990.Markdown
[Weigend et al. "Generalization by Weight-Elimination with Application to Forecasting." Neural Information Processing Systems, 1990.](https://mlanthology.org/neurips/1990/weigend1990neurips-generalization/)BibTeX
@inproceedings{weigend1990neurips-generalization,
title = {{Generalization by Weight-Elimination with Application to Forecasting}},
author = {Weigend, Andreas S. and Rumelhart, David E. and Huberman, Bernardo A.},
booktitle = {Neural Information Processing Systems},
year = {1990},
pages = {875-882},
url = {https://mlanthology.org/neurips/1990/weigend1990neurips-generalization/}
}