Parallax Geometry of Pairs of Points for 3D Scene Analysis

Abstract

We present a geometric relationship between the image motion of pairs of points over multiple frames. This relationship is based on the parallax displacements of points with respect to an arbitrary planar surface, and does not involve epipolar geometry. A constraint is derived over two frames for any pair of points, relating their projective structure (with respect to the plane) based only on their image coordinates and their parallax displacements. Similarly, a 3 D -rigidity constraint between pairs of points over multiple frames is derived. We show applications of these parallax-based constraints to solving three important problems in 3 D scene analysis: (i) the recovery of 3 D scene structure, (ii) the detection of moving objects in the presence of camera induced motion, and (iii) the synthesis of new camera views based on a given set of views. Moreover, we show that this approach can handle difficult situations for 3 D scene analysis, e.g., where there is only a small set of parallax vectors, and in the presence of independently moving objects.