Segment3D: Learning Fine-Grained Class-Agnostic 3D Segmentation Without Manual Labels
Abstract
Current 3D scene segmentation methods are heavily dependent on manually annotated 3D training datasets. Such manual annotations are labor-intensive, and often lack fine-grained details. Furthermore, models trained on this data typically struggle to recognize object classes beyond the annotated training classes, , they do not generalize well to unseen domains and require additional domain-specific annotations. In contrast, recent 2D foundation models have demonstrated strong generalization and impressive zero-shot abilities, inspiring us to incorporate these characteristics from 2D models into 3D models. Therefore, we explore the use of image segmentation foundation models to automatically generate high-quality training labels for 3D segmentation models. The resulting model, , generalizes significantly better than the models trained on costly manual 3D labels and enables easily adding new training data to further boost the segmentation performance.
Cite
Text
Huang et al. "Segment3D: Learning Fine-Grained Class-Agnostic 3D Segmentation Without Manual Labels." Proceedings of the European Conference on Computer Vision (ECCV), 2024. doi:10.1007/978-3-031-72754-2_16Markdown
[Huang et al. "Segment3D: Learning Fine-Grained Class-Agnostic 3D Segmentation Without Manual Labels." Proceedings of the European Conference on Computer Vision (ECCV), 2024.](https://mlanthology.org/eccv/2024/huang2024eccv-segment3d/) doi:10.1007/978-3-031-72754-2_16BibTeX
@inproceedings{huang2024eccv-segment3d,
title = {{Segment3D: Learning Fine-Grained Class-Agnostic 3D Segmentation Without Manual Labels}},
author = {Huang, Rui and Peng, Songyou and Takmaz, Ayca and Tombari, Federico and Pollefeys, Marc and Song, Shiji and Huang, Gao and Engelmann, Francis},
booktitle = {Proceedings of the European Conference on Computer Vision (ECCV)},
year = {2024},
doi = {10.1007/978-3-031-72754-2_16},
url = {https://mlanthology.org/eccv/2024/huang2024eccv-segment3d/}
}