Duality Between Learning Machines: A Bridge Between Supervised and Unsupervised Learning
Abstract
We exhibit a duality between two perceptrons that allows us to compare the theoretical analysis of supervised and unsupervised learning tasks. The first perceptron has one output and is asked to learn a classification of p patterns. The second (dual) perceptron has p outputs and is asked to transmit as much information as possible on a distribution of inputs. We show in particular that the maximum information that can be stored in the couplings for the supervised learning task is equal to the maximum information that can be transmitted by the dual perceptron.
Cite
Text
Nadal and Parga. "Duality Between Learning Machines: A Bridge Between Supervised and Unsupervised Learning." Neural Computation, 1994. doi:10.1162/NECO.1994.6.3.491Markdown
[Nadal and Parga. "Duality Between Learning Machines: A Bridge Between Supervised and Unsupervised Learning." Neural Computation, 1994.](https://mlanthology.org/neco/1994/nadal1994neco-duality/) doi:10.1162/NECO.1994.6.3.491BibTeX
@article{nadal1994neco-duality,
title = {{Duality Between Learning Machines: A Bridge Between Supervised and Unsupervised Learning}},
author = {Nadal, Jean-Pierre and Parga, Néstor},
journal = {Neural Computation},
year = {1994},
pages = {491-508},
doi = {10.1162/NECO.1994.6.3.491},
volume = {6},
url = {https://mlanthology.org/neco/1994/nadal1994neco-duality/}
}