DISTFormer: Enhance 3D Human Pose Estimation via Dual Inverse-Order Spatial-Temporal Transformer

Abstract

Recent advances in transformer-based methods have significantly improved performance in 3D human pose estimation, establishing new benchmarks in the field. In this paper, we aim to capture the spatial relations of human joints within each video frame and the human dynamics across frames, proposing a novel Dual Inverse-Order Spatial-Temporal Transformer (DIST) block. Specifically, each processing stream incorporates two complementary components: an Enhanced Temporal Multi-head Self-Attention (ET-MHSA) module for temporal context modeling and an Enhanced Spatial Multi-head Self-Attention (ES-MHSA) unit for modeling spatial dependencies between joints. Building on this foundation, we construct DISTFormer by stacking multiple DIST blocks and further incorporate a novel Spatial-Temporal Enhanced Positional Embedding (ST-EPE). The embedding simultaneously extracts features from the perspectives of spatial structure and temporal position, facilitating in-depth interactions of spatio-temporal features within the model. In addition, we ingeniously integrate joint grouping information into the regression head to ensure the anatomical plausibility and accuracy of pose estimation. We conduct extensive experiments on the Human3.6M and MPI-INF-3DHP benchmark datasets. Results demonstrate that our model not only achieves superior performance with significantly fewer parameters but also outperforms multiple state-of-the-art methods. Specifically, on the Human3.6M dataset, compared to the baseline STCFormer, our approach achieves a 0.2 mm reduction in P1 error while reducing the parameter count by 29.1%.