A Geometric ConvNet on 3D Shape Manifold for Gait Recognition

Abstract

In this work we propose a geometric deep convolutional auto-encoder (DCAE) for the purpose of gait recognition by analyzing time-varying 3D skeletal data. Sequences are viewed as time-parameterized trajectories on the Kendall shape space S, results of modding out shape-preserving transformations (scaling, translations and rotations). The accommodation of ConvNet architectures to properly approximate manifold-valued trajectories on the underlying non-linear space $\mathcal{S}$ is a must. Thus, we make use of geometric steps prior to the encoding-decoding scheme. That is, shape trajectories are first log-mapped to tangent spaces attached to the shape space at a time-varying average trajectory μ, then, obtained vectors are transported to a common tangent space ${T_{\mu (0)}}(\mathcal{S})$ at the starting point of μ. Without applying any prior temporal alignment (e.g. Dynamic Time Warping) or modeling (e.g. HMM, RNN), the transported trajectories are then fed to a convolutional auto-encoder to build subject-specific latent spaces. The proposed approach was tested on two publicly available datasets. Our approach outperforms existing approaches on CMU gait dataset, while performances on UPCV K2 are comparable to existing approaches. We demonstrate that combining geometric invariance (i.e. Kendall’s representation) with our data-driven ConvNet model is suitable to alleviate spatial and temporal variability, respectively.