Multi-Frame Optical Flow Estimation Using Subspace Constraints

Abstract

We show that the set of all ow- elds in a sequence of frames imaging a rigid scene resides in a lowdimensional linear subspace. Based on this observation, we develop a method for simultaneous estimation of optical- ow across multiple frames, which uses these subspace constraints. The multi-frame subspace constraints are strong constraints, and replace commonly used heuristic constraints, such as spatial or temporal smoothness. The subspace constraints are geometrically meaningful, and are not violated at depth discontinuities, or when the camera-motion changes abruptly. Furthermore, we show that the subspace constraints on ow- elds apply for a variety of imaging models, scene models, and motion models. Hence, the presented approach forconstrained multi-frame ow estimation is general. However, our approach doesnot require prior knowledge of the underlying world or camera model. Although linear subspace constraints have been used successfully in the past for recovering 3D information (e.g., [18]), it has been assumed that 2D correspondences are given. However, correspondence estimation is a fundamental problem in motion analysis. In this paper, we usemulti-frame subspace constraints to constrain the 2D correspondence estimation process itself, and not for 3D recovery. 1