MedDelinea: Scalable and Efficient Medical Image Segmentation via Controllable Diffusion Transformers

Abstract

We introduce MedDelinea, a novel medical image segmentation architecture that leverages a controllable module, drawing inspiration from ControlNet, within the Diffusion Transformers (DiT) framework. By doing so, we effectively address three key challenges inherent to segmentation tasks: (1) limited availability of labeled data, (2) variability in image modalities, and (3) the need for precise boundary delineation. MedDelinea is pre-trained on a large-scale medical dataset, thereby mitigating overfitting risks and enabling efficient transfer across diverse imaging scenarios with minimal fine-tuning requirements. The modular design of MedDelinea facilitates scalable and efficient computation, while maintaining high-quality segmentation performance in both supervised and zero-shot settings. Through extensive empirical evaluations on multiple datasets, we demonstrate that MedDelinea outperforms existing state-of-the-art segmentation approaches, showcasing its potential for robust and accurate medical image analysis