A Non-Linear Information Maximisation Algorithm That Performs Blind Separation
Abstract
A new learning algorithm is derived which performs online stochas(cid:173) tic gradient ascent in the mutual information between outputs and inputs of a network. In the absence of a priori knowledge about the 'signal' and 'noise' components of the input, propagation of information depends on calibrating network non-linearities to the detailed higher-order moments of the input density functions. By incidentally minimising mutual information between outputs, as well as maximising their individual entropies, the network 'fac(cid:173) torises' the input into independent components. As an example application, we have achieved near-perfect separation of ten digi(cid:173) tally mixed speech signals. Our simulations lead us to believe that our network performs better at blind separation than the Herault(cid:173) J utten network, reflecting the fact that it is derived rigorously from the mutual information objective.
Cite
Text
Bell and Sejnowski. "A Non-Linear Information Maximisation Algorithm That Performs Blind Separation." Neural Information Processing Systems, 1994.Markdown
[Bell and Sejnowski. "A Non-Linear Information Maximisation Algorithm That Performs Blind Separation." Neural Information Processing Systems, 1994.](https://mlanthology.org/neurips/1994/bell1994neurips-nonlinear/)BibTeX
@inproceedings{bell1994neurips-nonlinear,
title = {{A Non-Linear Information Maximisation Algorithm That Performs Blind Separation}},
author = {Bell, Anthony J. and Sejnowski, Terrence J.},
booktitle = {Neural Information Processing Systems},
year = {1994},
pages = {467-474},
url = {https://mlanthology.org/neurips/1994/bell1994neurips-nonlinear/}
}