Dimensionality of Illumination Manifolds in Appearance Matching

Abstract

Appearance matching was recently demonstrated as a robust and efficient approach to 3D object recognition and pose estimation. Each object is represented as a continuous appearance manifold in a low-dimensional subspace parametrized by object pose and illumination direction. Here, the structural properties of appearance manifolds are analyzed with the aim of making appearance representation efficient in off-line computation, storage requirements, and on-line recognition time. In particular, the effect of illumination on the structure of the appearance manifold is studied. For an ideal diffuse surface of arbitrary texture, the appearance manifold is linear and of dimensionality 3.This enables the construction of the entire illumination manifold from just three images of the object taken using linearly independent light sources. This result is shown to hold even for illumination by multiple light sources and for concave surfaces that exhibit interreflections. Finally, a simple but efficient algorithm is presented that uses just three manifold points for recognizing images taken under novel illuminations.