Vision-Based System Identification and 3D Keypoint Discovery Using Dynamics Constraints
Abstract
This paper introduces V-SysId, a novel method that enables simultaneous keypoint discovery, 3D system identification, and extrinsic camera calibration from an unlabeled video taken from a static camera, using only the family of equations of motion of the object of interest as weak supervision. V-SysId takes keypoint trajectory proposals and alternates between maximum likelihood parameter estimation and extrinsic camera calibration, before applying a suitable selection criterion to identify the track of interest. This is then used to train a keypoint tracking model using supervised learning. Results on a range of settings (robotics, physics, physiology) highlight the utility of this approach.
Cite
Text
Jaques et al. "Vision-Based System Identification and 3D Keypoint Discovery Using Dynamics Constraints." Proceedings of The 4th Annual Learning for Dynamics and Control Conference, 2022.Markdown
[Jaques et al. "Vision-Based System Identification and 3D Keypoint Discovery Using Dynamics Constraints." Proceedings of The 4th Annual Learning for Dynamics and Control Conference, 2022.](https://mlanthology.org/l4dc/2022/jaques2022l4dc-visionbased/)BibTeX
@inproceedings{jaques2022l4dc-visionbased,
title = {{Vision-Based System Identification and 3D Keypoint Discovery Using Dynamics Constraints}},
author = {Jaques, Miguel and Asenov, Martin and Burke, Michael and Hospedales, Timothy},
booktitle = {Proceedings of The 4th Annual Learning for Dynamics and Control Conference},
year = {2022},
pages = {316-329},
volume = {168},
url = {https://mlanthology.org/l4dc/2022/jaques2022l4dc-visionbased/}
}