Segmentation CNNs Are Denoising Models

Abstract

Encoder-decoder CNNs, such as the U-Net are the de-facto approach for image segmentation. Despite their good properties, U-Net-like models are often treated as black boxes, which hides the signal processing performed to the images, as well as their potential downsides/limitations. To address these disadvantages, this paper studies the signal processing performed by segmentation models such as the U-Net by employing a proxy CNN, in which its linear behavior can be analyzed. The suggested proxy model has been trained for image segmentation and its impulse response is computed for different training and test settings. The impulse and frequency responses show that the processing of U-Net-like models trained for segmentation are similar to sparse modeling techniques employed in image denoising and in signal detection. Furthermore, this simple approach of using a proxy CNN can indicate also properties of the filter banks that compose the CNN.