Geometrical Singularities in the Neuromanifold of Multilayer Perceptrons
Abstract
Singularities are ubiquitous in the parameter space of hierarchical models such as multilayer perceptrons. At singularities, the Fisher information matrix degenerates, and the Cramer-Rao paradigm does no more hold, implying that the classical model selection the(cid:173) ory such as AIC and MDL cannot be applied. It is important to study the relation between the generalization error and the training error at singularities. The present paper demonstrates a method of analyzing these errors both for the maximum likelihood estima(cid:173) tor and the Bayesian predictive distribution in terms of Gaussian random fields, by using simple models.
Cite
Text
Amari et al. "Geometrical Singularities in the Neuromanifold of Multilayer Perceptrons." Neural Information Processing Systems, 2001.Markdown
[Amari et al. "Geometrical Singularities in the Neuromanifold of Multilayer Perceptrons." Neural Information Processing Systems, 2001.](https://mlanthology.org/neurips/2001/amari2001neurips-geometrical/)BibTeX
@inproceedings{amari2001neurips-geometrical,
title = {{Geometrical Singularities in the Neuromanifold of Multilayer Perceptrons}},
author = {Amari, Shun-ichi and Park, Hyeyoung and Ozeki, Tomoko},
booktitle = {Neural Information Processing Systems},
year = {2001},
pages = {343-350},
url = {https://mlanthology.org/neurips/2001/amari2001neurips-geometrical/}
}