Connectivity Versus Entropy
Abstract
How does the connectivity of a neural network (number of synapses per neuron) relate to the complexity of the problems it can handle (measured by the entropy)? Switching theory would suggest no relation at all, since all Boolean functions can be implemented using a circuit with very low connectivity (e.g., using two-input NAND gates). However, for a network that learns a problem from examples using a local learning rule, we prove that the entropy of the problem becomes a lower bound for the connectivity of the network.
Cite
Text
Abu-Mostafa. "Connectivity Versus Entropy." Neural Information Processing Systems, 1987.Markdown
[Abu-Mostafa. "Connectivity Versus Entropy." Neural Information Processing Systems, 1987.](https://mlanthology.org/neurips/1987/abumostafa1987neurips-connectivity/)BibTeX
@inproceedings{abumostafa1987neurips-connectivity,
title = {{Connectivity Versus Entropy}},
author = {Abu-Mostafa, Yaser S.},
booktitle = {Neural Information Processing Systems},
year = {1987},
pages = {1-8},
url = {https://mlanthology.org/neurips/1987/abumostafa1987neurips-connectivity/}
}