REMOTE: Reinforced Motion Transformation Network for Semi-Supervised 2D Pose Estimation in Videos
Abstract
Existing approaches for 2D pose estimation in videos often require a large number of dense annotations, which are costly and labor intensive to acquire. In this paper, we propose a semi-supervised REinforced MOtion Transformation nEtwork (REMOTE) to leverage a few labeled frames and temporal pose variations in videos, which enables effective learning of 2D pose estimation in sparsely annotated videos. Specifically, we introduce a Motion Transformer (MT) module to perform cross frame reconstruction, aiming to learn motion dynamic knowledge in videos. Besides, a novel reinforcement learning-based Frame Selection Agent (FSA) is designed within our framework, which is able to harness informative frame pairs on the fly to enhance the pose estimator under our cross reconstruction mechanism. We conduct extensive experiments that show the efficacy of our proposed REMOTE framework.
Cite
Text
Ma et al. "REMOTE: Reinforced Motion Transformation Network for Semi-Supervised 2D Pose Estimation in Videos." AAAI Conference on Artificial Intelligence, 2022. doi:10.1609/AAAI.V36I2.20089Markdown
[Ma et al. "REMOTE: Reinforced Motion Transformation Network for Semi-Supervised 2D Pose Estimation in Videos." AAAI Conference on Artificial Intelligence, 2022.](https://mlanthology.org/aaai/2022/ma2022aaai-remote/) doi:10.1609/AAAI.V36I2.20089BibTeX
@inproceedings{ma2022aaai-remote,
title = {{REMOTE: Reinforced Motion Transformation Network for Semi-Supervised 2D Pose Estimation in Videos}},
author = {Ma, Xianzheng and Rahmani, Hossein and Fan, Zhipeng and Yang, Bin and Chen, Jun and Liu, Jun},
booktitle = {AAAI Conference on Artificial Intelligence},
year = {2022},
pages = {1944-1952},
doi = {10.1609/AAAI.V36I2.20089},
url = {https://mlanthology.org/aaai/2022/ma2022aaai-remote/}
}