Video from Nearly Still: An Application to Low Frame-Rate Gait Recognition

Abstract

In this paper, we propose a temporal super resolution approach for quasi-periodic image sequence such as human gait. The proposed method effectively combines example-based and reconstruction-based temporal super resolution approaches. A periodic image sequence is expressed as a manifold parameterized by a phase and a standard manifold is learned from multiple high frame-rate sequences in the training stage. In the test stage, an initial phase for each frame of an input low frame-rate image sequence is estimated based on the standard manifold at first, and the manifold reconstruction and the phase estimation are then iterated to generate better high frame-rate images in the energy minimization framework that ensures the fitness to both the input images and the standard manifold. The proposed method is applied to low frame-rate gait recognition and experiments with real data of 100 subjects demonstrate a significant improvement by the proposed method, particularly for quite low frame-rate videos (e.g., 1 fps).