CL-GAN: Contrastive Learning-Based Generative Adversarial Network for Modality Transfer with Limited Paired Data

Abstract

Separate acquisition of multiple modalities in medical imaging is time-consuming, costly and increases unnecessary irradiation to patients. This paper proposes a novel deep learning method, contrastive learning-based Generative Adversarial Network (CL-GAN) for modality transfer with limited paired data. We employ CL-GAN to generate synthetic PET (synPET) images from MRI data, and it has a three-phase training pipeline: 1) intra-modality training for separate source (MRI) and target (PET) domain encoders, 2) cross-modality encoder training with MRI-PET pairs and 3) GAN training. As obtaining paired MRI-PET training data in sufficient quantities is often very costly and cumbersome in clinical practice, we integrate contrastive learning (CL) in all three training phases to fully leverage paired and unpaired data, leading to more accurate and realistic synPET images. Experimental results on benchmark datasets demonstrate the superior performance of CL-GAN, both qualitatively and quantitatively, when compared with current state-of-the-art methods.