Finding Nemo: Deformable Object Class Modelling Using Curve Matching

Abstract

An image search for "clownfish" yields many photos of clownfish, each of a different individual of a different 3D shape in a different pose. Yet, to the human observer, this set of images contains enough information to infer the underlying 3D deformable object class. Our goal is to recover such a deformable object class model directly from unordered images. For classes where feature-point correspondences can be found, this is a straightforward extension of non-rigid factorization, yielding a set of 3D basis shapes to explain the 2D data. However, when each image is of a different object instance, surface texture is generally unique to each individual, and does not give rise to usable image point correspondences. We overcome this sparsity using curve correspondences (crease-edge silhouettes or class-specific internal texture edges). Even rigid contour reconstruction is difficult due to the lack of reliable correspondences. We incorporate correspondence variation into the optimization, thereby extending contour-based reconstruction techniques to deformable object modelling. The notion of correspondence is extended to include mappings between 2D image curves and corresponding parts of the desired 3D object surface. Combined with class-specific priors, our method enables effective deformable class reconstruction from unordered images, despite significant occlusion and the scarcity of shared 2D image features. ©2010 IEEE.