Joint Demosaicking and Denoising for CFA and MSFA Images Using a Mosaic-Adaptive Dense Residual Network

Abstract

Color filter array (CFA) has been a basis for modern photography and recently multispectral filter array (MSFA) has gradually found its wide application. A deep learning network capable of joint demosaicking and denoising for both CFA and MSFA raw images is proposed in this paper. First, a novel dense residual network that includes multiple types of skip connections is introduced to learn features at different resolutions. Then, mosaic adaptive convolution and data augmentation based on mosaic shifting are put forward to fully make use of common characteristics of CFA and MSFA mosaic images. Moreover, an L 1 loss function normalized by noise standard deviation is suggested to train the deep residual network so it does not rely on an explicit input of known or estimated noise standard deviation. Extensive experiments using simulated and real mosaic images from CFA cameras demonstrate that the proposed mosaic-adaptive dense residual network (MDRN) outperforms other state-of-the-art deep learning algorithms significantly. For simulated MSFA mosaics and real MSFA raw images, it also shows much improved results compared to other methods.