Hierarchical Non-Linear Factor Analysis and Topographic Maps

Abstract

We first describe a hierarchical, generative model that can be viewed as a non-linear generalisation of factor analysis and can be implemented in a neural network. The model performs per(cid:173) ceptual inference in a probabilistically consistent manner by using top-down, bottom-up and lateral connections. These connections can be learned using simple rules that require only locally avail(cid:173) able information. We then show how to incorporate lateral con(cid:173) nections into the generative model. The model extracts a sparse, distributed, hierarchical representation of depth from simplified random-dot stereograms and the localised disparity detectors in the first hidden layer form a topographic map. When presented with image patches from natural scenes, the model develops topo(cid:173) graphically organised local feature detectors.