A Multiplicative Up-Propagation Algorithm

Abstract

We present a generalization of the nonnegative matrix factorization (NMF),where a multilayer generative network with nonnegative weights is used toapproximate the observed nonnegative data. The multilayer generative networkwith nonnegativity constraints, is learned by a multiplicative up-propagationalgorithm, where the weights in each layer are updated in a multiplicativefashion while the mismatch ratio is propagated from the bottom to the toplayer. The monotonic convergence of the multiplicative up-propagation algorithm isshown. In contrast to NMF, the multiplicative up-propagation is an algorithmthat can learn hierarchical representations, where complex higher-levelrepresentations are defined in terms of less complex lower-levelrepresentations. The interesting behavior of our algorithm is demonstratedwith face image data.