Optimal Multi-Frame Correspondence with Assignment Tensors

Abstract

Establishing correspondence between features of a set of images has been a long-standing issue amongst the computer vision community. We propose a method that solves the multi-frame correspondence problem by imposing a rank constraint on the observed scene, i.e. rigidity is assumed. Since our algorithm is based solely on a geometrical (global) criterion, it does not suffer from issues usually associated to local methods, such as the aperture problem. We model feature matching by introducing the assignment tensor , which allows simultaneous feature alignment for all images, thus providing a coherent solution to the calibrated multi-frame correspondence problem in a single step of linear complexity. Also, an iterative method is presented that is able to cope with the non-calibrated case. Moreover, our method is able to seamlessly reject a large number of outliers in every image, thus also handling occlusion in an integrated manner.