Toward Real-World Panoramic Image Enhancement

Abstract

Panoramic images captured by the fisheye lens cameras cover very wide field of view (FoV) ranging from 180° to 360°, but the image quality is very low compared to that of high-end cameras such as DSLR or compact cameras with APS-C or full frame sensors. In this paper, we aim to use deep neural network (DNN) based methods to improve panoramic image quality. Specifically, we enhance low quality panoramic images of 5K resolution (5376×2688) to high-end camera quality at the same resolution, which is good for applications that requires limited resources, low-cost but high image quality. We build a Panoramic-High-end dataset which is the first real world panoramic image dataset as far as we know. Based on the generative adversarial network (GAN) architecture, we also design a compact network employing multi-frequency structure with compressed Residual-in-Residual Dense Blocks (RRDBs) and convolution layers from each dense block. Experiments show that our method surpasses several state-of-the-art DNN based methods in both no-reference and full-reference evaluations as well as the processing speed. Our results show that it’s practical to integrate DNN based image enhancer into optics design to achieve a balance between optical cost and image quality.