ACE: Anatomically Consistent Embeddings in Composition and Decomposition

Abstract

Medical images acquired from standardized protocols show consistent macroscopic or microscopic anatomical structures and these structures consist of composable/decomposable organs and tissues but existing self-supervised learning (SSL) methods do not appreciate such composable/decomposable structure attributes inherent to medical images. To overcome this limitation this paper introduces a novel SSL approach called ACE to learn anatomically consistent embedding via composition and decomposition with two key branches: (1) global consistency capturing discriminative macro-structures via extracting global features; (2) local consistency learning fine-grained anatomical details from composable/decomposable patch features via corresponding matrix matching. Experimental results across 6 datasets and 2 backbones evaluated in few-shot learning fine-tuning and property analysis show ACE's superior robustness transferability and clinical potential. The innovations of our ACE lie in grid-wise image cropping leveraging the intrinsic properties of compositionality and decompositionality of medical images bridging the semantic gap from high-level pathologies to low-level tissue anomalies and providing a new SSL method for medical imaging.