You2Me: Inferring Body Pose in Egocentric Video via First and Second Person Interactions
Abstract
The body pose of a person wearing a camera is of great interest for applications in augmented reality, healthcare, and robotics, yet much of the person's body is out of view for a typical wearable camera. We propose a learning-based approach to estimate the camera wearer's 3D body pose from egocentric video sequences. Our key insight is to leverage interactions with another person---whose body pose we can directly observe---as a signal inherently linked to the body pose of the first-person subject. We show that since interactions between individuals often induce a well-ordered series of back-and-forth responses, it is possible to learn a temporal model of the interlinked poses even though one party is largely out of view. We demonstrate our idea on a variety of domains with dyadic interaction and show the substantial impact on egocentric body pose estimation, which improves the state of the art.
Cite
Text
Ng et al. "You2Me: Inferring Body Pose in Egocentric Video via First and Second Person Interactions." Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020. doi:10.1109/CVPR42600.2020.00991Markdown
[Ng et al. "You2Me: Inferring Body Pose in Egocentric Video via First and Second Person Interactions." Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020.](https://mlanthology.org/cvpr/2020/ng2020cvpr-you2me/) doi:10.1109/CVPR42600.2020.00991BibTeX
@inproceedings{ng2020cvpr-you2me,
title = {{You2Me: Inferring Body Pose in Egocentric Video via First and Second Person Interactions}},
author = {Ng, Evonne and Xiang, Donglai and Joo, Hanbyul and Grauman, Kristen},
booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition},
year = {2020},
doi = {10.1109/CVPR42600.2020.00991},
url = {https://mlanthology.org/cvpr/2020/ng2020cvpr-you2me/}
}