ROCA: Robust CAD Model Retrieval and Alignment from a Single Image

Can Gümeli, Angela Dai, Matthias Nießner

CVPR 2022 pp. 4022-4031

doi:10.1109/CVPR52688.2022.00399 /cvpr/2022/gumeli2022cvpr-roca/

Abstract

We present ROCA, a novel end-to-end approach that retrieves and aligns 3D CAD models from a shape database to a single input image. This enables 3D perception of an observed scene from a 2D RGB observation, characterized as a lightweight, compact, clean CAD representation. Core to our approach is our differentiable alignment optimization based on dense 2D-3D object correspondences and Procrustes alignment. ROCA can thus provide a robust CAD alignment while simultaneously informing CAD retrieval by leveraging the 2D-3D correspondences to learn geometrically similar CAD models. Experiments on challenging, real-world imagery from ScanNet show that ROCA significantly improves on state of the art, from 9.5% to 17.6% in retrieval-aware CAD alignment accuracy.

PDF CVPR Semantic Scholar

Cite

Text

Gümeli et al. "ROCA: Robust CAD Model Retrieval and Alignment from a Single Image." Conference on Computer Vision and Pattern Recognition, 2022. doi:10.1109/CVPR52688.2022.00399

Markdown

[Gümeli et al. "ROCA: Robust CAD Model Retrieval and Alignment from a Single Image." Conference on Computer Vision and Pattern Recognition, 2022.](https://mlanthology.org/cvpr/2022/gumeli2022cvpr-roca/) doi:10.1109/CVPR52688.2022.00399

BibTeX

@inproceedings{gumeli2022cvpr-roca,
  title     = {{ROCA: Robust CAD Model Retrieval and Alignment from a Single Image}},
  author    = {Gümeli, Can and Dai, Angela and Nießner, Matthias},
  booktitle = {Conference on Computer Vision and Pattern Recognition},
  year      = {2022},
  pages     = {4022-4031},
  doi       = {10.1109/CVPR52688.2022.00399},
  url       = {https://mlanthology.org/cvpr/2022/gumeli2022cvpr-roca/}
}