Action-Agnostic Human Pose Forecasting

Abstract

Forecasting human dynamics is a very interesting but challenging task with several prospective applications in robotics, health-care, among others. Researchers have recently developed methods for human pose forecasting; but unfortunately, they often introduce a number of simplification assumptions. For instance, previous work either focuses only on short-term or long-term predictions, while sacrificing one or the other. Furthermore, they use the activity labels as part of the training process and require them to be available at testing time. These simplifications limit the usage of such pose forecasting models for real-world applications. To overcome these limitations, we propose a new action-agnostic method for short-and long-term human pose forecasting. Our triangular-prism recurrent neural network (TP-RNN) models the hierarchical and multi-scale characteristics of human dynamics. Our model captures the latent hierarchical structure in human pose sequences by encoding temporal dependencies with different time-scales. We run an extensive set of experiments on Human 3.6M and Penn Action datasets and show that our method outperforms baseline and state-of-the-art methods quantitatively and qualitatively. Code is available at https://github.com/eddyhkchiu/pose_forecast_wacv/.