State-of-the-Art Text-Prompted Medical Segmentation Models Struggle to Ground Chest CT Findings

Mohammed Baharoon, Luyang Luo, Michael Moritz, Abhinav Kumar, Sung Eun Kim, Xiaoman Zhang, Miao Zhu, Kent Kleinschmidt, Sri Sai Dinesh Jaliparthi, Sathvik Suryadevara, Rithvik Akula, Mark Marino, Wenhui Lei, Ibrahim Ethem Hamamci, Pranav Rajpurkar

MLHC 2025

/mlhc/2025/baharoon2025mlhc-stateoftheart/

Abstract

This study presents a comprehensive evaluation of state-of-the-art text-prompted segmentation models, including SAM2, MedSAM2, SegVol, SAT, and BiomedParse, on ReXGrounding, a novel dataset that pairs chest CT findings with corresponding segmentation masks. Our results demonstrate that despite recent advances, current models struggle to accurately segment diverse findings from chest CTs, particularly when dealing with non-focal abnormalities described in natural language reports. While existing models are primarily optimized for fixed categorical labels rather than nuanced clinical descriptions, even fine-tuning these models with free-text descriptions yields limited improvement in segmentation accuracy. These insights highlight that report grounding on 3D medical volumes through segmentation remains an open challenge, necessitating future models that better comprehend complex clinical language and irregular object patterns across volumetric data.

PDF MLHC Semantic Scholar

Cite

Text

Baharoon et al. "State-of-the-Art Text-Prompted Medical Segmentation Models Struggle to Ground Chest CT Findings." Proceedings of the 10th Machine Learning for Healthcare Conference, 2025.

Markdown

[Baharoon et al. "State-of-the-Art Text-Prompted Medical Segmentation Models Struggle to Ground Chest CT Findings." Proceedings of the 10th Machine Learning for Healthcare Conference, 2025.](https://mlanthology.org/mlhc/2025/baharoon2025mlhc-stateoftheart/)

BibTeX

@inproceedings{baharoon2025mlhc-stateoftheart,
  title     = {{State-of-the-Art Text-Prompted Medical Segmentation Models Struggle to Ground Chest CT Findings}},
  author    = {Baharoon, Mohammed and Luo, Luyang and Moritz, Michael and Kumar, Abhinav and Kim, Sung Eun and Zhang, Xiaoman and Zhu, Miao and Kleinschmidt, Kent and Jaliparthi, Sri Sai Dinesh and Suryadevara, Sathvik and Akula, Rithvik and Marino, Mark and Lei, Wenhui and Hamamci, Ibrahim Ethem and Rajpurkar, Pranav},
  booktitle = {Proceedings of the 10th Machine Learning for Healthcare Conference},
  year      = {2025},
  volume    = {298},
  url       = {https://mlanthology.org/mlhc/2025/baharoon2025mlhc-stateoftheart/}
}