ML Anthology

Transfer Learning from Simulated to Real Scenes for Monocular 3D Object Detection

Sondos Mohamed, Walter Zimmer, Ross Greer, Ahmed Alaaeldin Ghita, Modesto Castrillón Santana, Mohan M. Trivedi, Alois Knoll, Salvatore M. Carta, Mirko Marras
ECCVW 2024 pp. 309-325
doi:10.1007/978-3-031-91813-1_20 /eccvw/2024/mohamed2024eccvw-transfer/

Abstract

Accurately detecting 3D objects from monocular images in dynamic roadside scenarios remains a challenging problem due to varying camera perspectives and unpredictable scene conditions. This paper introduces a two-stage training strategy to address these challenges. Our approach initially trains a model on the large-scale synthetic dataset, RoadSense3D , which offers a diverse range of scenarios for robust feature learning. Subsequently, we fine-tune the model on a combination of real-world datasets to enhance its adaptability to practical conditions. Experimental results of the Cube R-CNN model on challenging public benchmarks show a remarkable improvement in detection performance, with a mean average precision rising from 0.26 to 12.76 on the TUM Traffic A9 Highway dataset and from 2.09 to 6.60 on the DAIR-V2X-I dataset, when performing transfer learning. Code, data, and qualitative video results are available at https://roadsense3d.github.io .

PDF ECCVW Semantic Scholar

Cite

Text

Mohamed et al. "Transfer Learning from Simulated to Real Scenes for Monocular 3D Object Detection." European Conference on Computer Vision Workshops, 2024. doi:10.1007/978-3-031-91813-1_20

Markdown

[Mohamed et al. "Transfer Learning from Simulated to Real Scenes for Monocular 3D Object Detection." European Conference on Computer Vision Workshops, 2024.](https://mlanthology.org/eccvw/2024/mohamed2024eccvw-transfer/) doi:10.1007/978-3-031-91813-1_20

BibTeX

@inproceedings{mohamed2024eccvw-transfer,
  title     = {{Transfer Learning from Simulated to Real Scenes for Monocular 3D Object Detection}},
  author    = {Mohamed, Sondos and Zimmer, Walter and Greer, Ross and Ghita, Ahmed Alaaeldin and Santana, Modesto Castrillón and Trivedi, Mohan M. and Knoll, Alois and Carta, Salvatore M. and Marras, Mirko},
  booktitle = {European Conference on Computer Vision Workshops},
  year      = {2024},
  pages     = {309-325},
  doi       = {10.1007/978-3-031-91813-1_20},
  url       = {https://mlanthology.org/eccvw/2024/mohamed2024eccvw-transfer/}
}