Monocular Differentiable Rendering for Self-Supervised 3D Object Detection
Abstract
3D object detection from monocular images is an ill-posed problem due to the projective entanglement of depth and scale. To overcome this ambiguity, we present a novel self-supervised method for textured 3D shape reconstruction and pose estimation of rigid objects with the help of strong shape priors and 2D instance masks. Our method predicts the 3D location and meshes of each object in an image using differentiable rendering and a self-supervised objective derived from a pretrained monocular depth estimation network. We use the KITTI 3D object detection dataset to evaluate the accuracy of the method. Experiments demonstrate that we can effectively use noisy monocular depth and differentiable rendering as an alternative to expensive 3D ground-truth labels or LiDAR information.
Cite
Text
Beker et al. "Monocular Differentiable Rendering for Self-Supervised 3D Object Detection." Proceedings of the European Conference on Computer Vision (ECCV), 2020. doi:10.1007/978-3-030-58589-1_31Markdown
[Beker et al. "Monocular Differentiable Rendering for Self-Supervised 3D Object Detection." Proceedings of the European Conference on Computer Vision (ECCV), 2020.](https://mlanthology.org/eccv/2020/beker2020eccv-monocular/) doi:10.1007/978-3-030-58589-1_31BibTeX
@inproceedings{beker2020eccv-monocular,
title = {{Monocular Differentiable Rendering for Self-Supervised 3D Object Detection}},
author = {Beker, Deniz and Kato, Hiroharu and Morariu, Mihai Adrian and Ando, Takahiro and Matsuoka, Toru and Kehl, Wadim and Gaidon, Adrien},
booktitle = {Proceedings of the European Conference on Computer Vision (ECCV)},
year = {2020},
doi = {10.1007/978-3-030-58589-1_31},
url = {https://mlanthology.org/eccv/2020/beker2020eccv-monocular/}
}