Globally Optimal On-Line Learning Rules
Abstract
We present a method for determining the globally optimal on-line learning rule for a soft committee machine under a statistical me(cid:173) chanics framework. This work complements previous results on locally optimal rules, where only the rate of change in general(cid:173) ization error was considered. We maximize the total reduction in generalization error over the whole learning process and show how the resulting rule can significantly outperform the locally optimal rule.
Cite
Text
Rattray and Saad. "Globally Optimal On-Line Learning Rules." Neural Information Processing Systems, 1997.Markdown
[Rattray and Saad. "Globally Optimal On-Line Learning Rules." Neural Information Processing Systems, 1997.](https://mlanthology.org/neurips/1997/rattray1997neurips-globally/)BibTeX
@inproceedings{rattray1997neurips-globally,
title = {{Globally Optimal On-Line Learning Rules}},
author = {Rattray, Magnus and Saad, David},
booktitle = {Neural Information Processing Systems},
year = {1997},
pages = {322-328},
url = {https://mlanthology.org/neurips/1997/rattray1997neurips-globally/}
}