Zero-Shot Capability of 2D SAM-Family Models for Bone Segmentation in CT Scans

Abstract

The Segment Anything Model (Sam) and similar models build a family of promptable foundation models (FMs) for image and video segmentation. The object of interest is identified using prompts---user provided input such as bounding boxes or points---and the models have shown very promising results when it comes to generalization to new tasks. However, extensive evaluation studies are required for medical applications, to assess their strengths and weaknesses in clinical settings. As the performance of those models is highly dependent on the quality and quantity of their prompts, it is necessary to thoroughly benchmark the different options. Currently, no dedicated evaluation studies exist specifically for bone segmentation in CT scans. Leveraging high-quality private and public datasets on four skeletal regions, we test the zero-shot capabilities of SAM-family models for bone CT segmentation, using non-interactive prompting strategies, composed of bounding box, points and combinations of the two. Additionally, we design a guideline for informed decision-making in 2D non-interactive prompting based on our insights on segmentation performance and inference time. Our results show that SAM and SAM2 currently outperform medically fine-tuned FMs, and prompted with a bounding box together with a center point have the best performance across all tested settings. Our code is available in this github repository (https://github.com/CarolineMagg/SAM-family-2D-benchmark).