Second Order Properties of Error Surfaces: Learning Time and Generalization
Abstract
The learning time of a simple neural network model is obtained through an analytic computation of the eigenvalue spectrum for the Hessian matrix, which describes the second order properties of the cost function in the space of coupling coefficients. The form of the eigenvalue distribution suggests new techniques for accelerating the learning process, and provides a theoretical justification for the choice of centered versus biased state variables.
Cite
Text
LeCun et al. "Second Order Properties of Error Surfaces: Learning Time and Generalization." Neural Information Processing Systems, 1990.Markdown
[LeCun et al. "Second Order Properties of Error Surfaces: Learning Time and Generalization." Neural Information Processing Systems, 1990.](https://mlanthology.org/neurips/1990/lecun1990neurips-second/)BibTeX
@inproceedings{lecun1990neurips-second,
title = {{Second Order Properties of Error Surfaces: Learning Time and Generalization}},
author = {LeCun, Yann and Kanter, Ido and Solla, Sara A.},
booktitle = {Neural Information Processing Systems},
year = {1990},
pages = {918-924},
url = {https://mlanthology.org/neurips/1990/lecun1990neurips-second/}
}