OTST: A Two-Phase Framework for Joint Denoising and Remosaicing in RGBW CFA

Abstract

RGBW, a newly emerged type of Color Filter Array (CFA), possesses strong low-light photography capabilities. RGBW CFA shows significant application value when low-light sensitivity is critical, such as in security cameras and smartphones. However, the majority of commercial image signal processors (ISP) are primarily designed for Bayer CFA, research pertaining to RGBW CFA is very rare. To address above limitations, in this study, we propose a two-phase framework named OTST for the RGBW Joint Denoising and Remosaicing (RGBW-JRD) task. For the denoising stage, we propose Omni-dimensional Dynamic Convolution based Half-Shuffle Transformer (ODC-HST) which can fully utilize image’s long-range dependencies to dynamically remove the noise. For the remosaicing stage, we propose a Spatial Compressive Transformer (SCT) to efficiently capture both local and global dependencies across spatial and channel dimensions. Experimental results demonstrate that our two-phase RGBW-JRD framework outperforms existing RGBW denoising and remosaicing solutions across a wide range of noise levels. In addition, the proposed approach ranks the 2nd place in MIPI 2023 RGBW Joint Remosaic and Denoise competition.