PPR: Physically Plausible Reconstruction from Monocular Videos

Gengshan Yang, Shuo Yang, John Z. Zhang, Zachary Manchester, Deva Ramanan

ICCV 2023 pp. 3914-3924

doi:10.1109/ICCV51070.2023.00362 /iccv/2023/yang2023iccv-ppr/

Abstract

Given monocular videos, we build 3D models of articulated objects and environments whose 3D configurations satisfy dynamics and contact constraints. At its core, our method leverages differentiable physics simulation to aid visual reconstructions. We couple differentiable physics simulation with differentiable rendering via coordinate descent, which enables end-to-end optimization of, not only 3D reconstructions, but also physical system parameters from videos. We demonstrate the effectiveness of physics-informed reconstruction on monocular videos of quadruped animals and humans. It reduces reconstruction artifacts (e.g., scale ambiguity, unbalanced poses, and foot swapping) that are challenging to address by visual cues alone, and produces better foot contact estimation.

PDF ICCV Semantic Scholar

Cite

Text

Yang et al. "PPR: Physically Plausible Reconstruction from Monocular Videos." International Conference on Computer Vision, 2023. doi:10.1109/ICCV51070.2023.00362

Markdown

[Yang et al. "PPR: Physically Plausible Reconstruction from Monocular Videos." International Conference on Computer Vision, 2023.](https://mlanthology.org/iccv/2023/yang2023iccv-ppr/) doi:10.1109/ICCV51070.2023.00362

BibTeX

@inproceedings{yang2023iccv-ppr,
  title     = {{PPR: Physically Plausible Reconstruction from Monocular Videos}},
  author    = {Yang, Gengshan and Yang, Shuo and Zhang, John Z. and Manchester, Zachary and Ramanan, Deva},
  booktitle = {International Conference on Computer Vision},
  year      = {2023},
  pages     = {3914-3924},
  doi       = {10.1109/ICCV51070.2023.00362},
  url       = {https://mlanthology.org/iccv/2023/yang2023iccv-ppr/}
}