BoDiffusion: Diffusing Sparse Observations for Full-Body Human Motion Synthesis

Angela Castillo, María Escobar, Guillaume Jeanneret, Albert Pumarola, Pablo Arbeláez, Ali K. Thabet, Artsiom Sanakoyeu

ICCVW 2023 pp. 4223-4233

doi:10.1109/ICCVW60793.2023.00456 /iccvw/2023/castillo2023iccvw-bodiffusion/

Abstract

Mixed reality applications require tracking the user’s full-body motion to enable an immersive experience. However, typical head-mounted devices can only track head and hand movements, leading to a limited reconstruction of full-body motion due to variability in lower body configurations. We propose BoDiffusion – a generative diffusion model for motion synthesis to tackle this under-constrained reconstruction problem. We present a time and space conditioning scheme that allows BoDiffusion to leverage sparse tracking inputs while generating smooth and realistic full-body motion sequences. To the best of our knowledge, this is the first approach that uses the reverse diffusion process to model full-body tracking as a conditional sequence generation task. We conduct experiments on the large-scale motion-capture dataset AMASS and show that our approach outperforms the state-of-the-art approaches by a significant margin in terms of full-body motion realism and joint reconstruction error.

PDF ICCVW Semantic Scholar

Cite

Text

Castillo et al. "BoDiffusion: Diffusing Sparse Observations for Full-Body Human Motion Synthesis." IEEE/CVF International Conference on Computer Vision Workshops, 2023. doi:10.1109/ICCVW60793.2023.00456

Markdown

[Castillo et al. "BoDiffusion: Diffusing Sparse Observations for Full-Body Human Motion Synthesis." IEEE/CVF International Conference on Computer Vision Workshops, 2023.](https://mlanthology.org/iccvw/2023/castillo2023iccvw-bodiffusion/) doi:10.1109/ICCVW60793.2023.00456

BibTeX

@inproceedings{castillo2023iccvw-bodiffusion,
  title     = {{BoDiffusion: Diffusing Sparse Observations for Full-Body Human Motion Synthesis}},
  author    = {Castillo, Angela and Escobar, María and Jeanneret, Guillaume and Pumarola, Albert and Arbeláez, Pablo and Thabet, Ali K. and Sanakoyeu, Artsiom},
  booktitle = {IEEE/CVF International Conference on Computer Vision Workshops},
  year      = {2023},
  pages     = {4223-4233},
  doi       = {10.1109/ICCVW60793.2023.00456},
  url       = {https://mlanthology.org/iccvw/2023/castillo2023iccvw-bodiffusion/}
}