LPSNet: A Novel Log Path Signature Feature Based Hand Gesture Recognition Framework

Abstract

Hand gesture recognition is gaining more attentions because it's a natural and intuitive mode of human computer interaction. Hand gesture recognition still faces great challenges for the real-world applications due to the gesture variance and individual difference. In this paper, we propose the LPSNet, an end-to-end deep neural network based hand gesture recognition framework with novel log path signature features. We pioneer a robust feature, path signature (PS) and its compressed version, log path signature (LPS) to extract effective feature of hand gestures. Also, we present a new method based on PS and LPS to effectively combine RGB and depth videos. Further, we propose a statistical method, DropFrame, to enlarge the data set and increase its diversity. By testing on a well-known public dataset, Sheffield Kinect Gesture (SKIG), our method achieves classification rate as 96.7% (only use RGB videos) and 98.7% (combining RGB and Depth videos), which is the best result comparing with state-of-the-art methods.