Ensemble Learning and Linear Response Theory for ICA

Abstract

We propose a general Bayesian framework for performing independent component analysis (leA) which relies on ensemble learning and lin(cid:173) ear response theory known from statistical physics. We apply it to both discrete and continuous sources. For the continuous source the underde(cid:173) termined (overcomplete) case is studied. The naive mean-field approach fails in this case whereas linear response theory-which gives an improved estimate of covariances-is very efficient. The examples given are for sources without temporal correlations. However, this derivation can eas(cid:173) ily be extended to treat temporal correlations. Finally, the framework offers a simple way of generating new leA algorithms without needing to define the prior distribution of the sources explicitly.