Surface Snapping Optimization Layer for Single Image Object Shape Reconstruction
Abstract
Reconstructing the 3D shape of objects observed in a single image is a challenging task. Recent approaches rely on visual cues extracted from a given image learned from a deep net. In this work, we leverage recent advances in monocular scene understanding to incorporate an additional geometric cue of surface normals. For this, we proposed a novel optimization layer that encourages the face normals of the reconstructed shape to be aligned with estimated surface normals. We develop a computationally efficient conjugate-gradient-based method that avoids the computation of a high-dimensional sparse matrix. We show this framework to achieve compelling shape reconstruction results on the challenging Pix3D and ShapeNet datasets.
Cite
Text
Hu et al. "Surface Snapping Optimization Layer for Single Image Object Shape Reconstruction." International Conference on Machine Learning, 2023.Markdown
[Hu et al. "Surface Snapping Optimization Layer for Single Image Object Shape Reconstruction." International Conference on Machine Learning, 2023.](https://mlanthology.org/icml/2023/hu2023icml-surface/)BibTeX
@inproceedings{hu2023icml-surface,
title = {{Surface Snapping Optimization Layer for Single Image Object Shape Reconstruction}},
author = {Hu, Yuan-Ting and Schwing, Alex and Yeh, Raymond A.},
booktitle = {International Conference on Machine Learning},
year = {2023},
pages = {13599-13609},
volume = {202},
url = {https://mlanthology.org/icml/2023/hu2023icml-surface/}
}