Learning Skeleton Stream Patterns with Slow Feature Analysis for Action Recognition

Abstract

Previous studies on MoCap (Motion Capturing (MoCap) System tracks the key points which are marked with conspicuous color or other materials (such as LED lights). The motion sequences are collected into MoCap action datasets, e.g., 1973 [ 3 ] and CMU [ 4 ] MoCap action datasets.) action data suggest that skeleton joint streams contain sufficient intrinsic information for understanding human body actions. With the advancement in depth sensors, e.g., Kinect, pose estimation with depth image provides more available realistic skeleton stream data. However, the locations of joints are always unstable due to noises. Moreover, as the estimated skeletons of different persons are not the same, the variance of intra-class is large. In this paper, we first expand the coordinate stream of each joint into multi-order streams by fusing hierarchical global information to improve the stability of joint streams. Then, Slow Feature Analysis is applied to learn the visual pattern of each joint, and the high-level information in the learnt general patterns is encoded into each skeleton to reduce the intra-variance of the skeletons. Temporal pyramid of posture word histograms is used to describe the global temporal information of action sequence. Our approach is verified with Support Vector Machine (SVM) classifier on MSR Action3D dataset, and the experimental results demonstrate that our approach achieves the state-of-the-art level.