Transferring Microscopy Image Modalities with Conditional Generative Adversarial Networks

Abstract

Phase Contrast (PC) and Differential Interference Contrast (DIC) microscopy are two popular non-invasive techniques for monitoring live cells. Each of these two image modalities has its own advantages and disadvantages to visualize specimens, so biologists need these two complementary modalities together to analyze specimens. In this paper, we investigate a conditional Generative Adversarial Network (conditional GAN), which contains one generator and two discriminators, to transfer microscopy image modalities. Given a training dataset consisting of pairs of images (source and destination) captured on the same set of specimens by DIC and Phase Contrast microscopes, we can train a conditional GAN, and with this well-trained GAN, we can generate the corresponding Phase Contrast image given a new DIC image, vice versa. The preliminary experiments demonstrate that our approach outperforms one state-of-the-arts method, and can provide biologists a computational way to switch between microscopy image modalities, so biologists can combine the advantages of different image modalities to better visualize and analyze specimens over time, without purchasing all types of microscopy image modalities or switching between imaging systems back-andforth during time-lapse experiments.