Multilayer Neural Networks: One or Two Hidden Layers?
Abstract
We study the number of hidden layers required by a multilayer neu(cid:173) ral network with threshold units to compute a function f from n d to O, I. In dimension d = 2, Gibson characterized the functions computable with just one hidden layer, under the assumption that there is no "multiple intersection point" and that f is only defined on a compact set. We consider the restriction of f to the neighbor(cid:173) hood of a multiple intersection point or of infinity, and give neces(cid:173) sary and sufficient conditions for it to be locally computable with one hidden layer. We show that adding these conditions to Gib(cid:173) son's assumptions is not sufficient to ensure global computability with one hidden layer, by exhibiting a new non-local configuration, the "critical cycle", which implies that f is not computable with one hidden layer.
Cite
Text
Brightwell et al. "Multilayer Neural Networks: One or Two Hidden Layers?." Neural Information Processing Systems, 1996.Markdown
[Brightwell et al. "Multilayer Neural Networks: One or Two Hidden Layers?." Neural Information Processing Systems, 1996.](https://mlanthology.org/neurips/1996/brightwell1996neurips-multilayer/)BibTeX
@inproceedings{brightwell1996neurips-multilayer,
title = {{Multilayer Neural Networks: One or Two Hidden Layers?}},
author = {Brightwell, Graham and Kenyon, Claire and Paugam-Moisy, Hélène},
booktitle = {Neural Information Processing Systems},
year = {1996},
pages = {148-154},
url = {https://mlanthology.org/neurips/1996/brightwell1996neurips-multilayer/}
}