Deep Image Interpolation: A Unified Unsupervised Framework for Pansharpening

Abstract

Pansharpening, whose aim is to acquire high resolution multispectral data (HRMS) by the fusion of low resolution multispectral data (LRMS) and panchromatic data (PAN), is a specific mission of spatial-spectral fusion in remote sensing field. In recent years, deep learning methods have proved the most feasible methods for pansharpening task. However, these deep learning methods have difficulty in training in an unsupervised manner and become useless when it comes to the condition where no training dataset is available. In this paper, we propose a universal algorithm called deep image interpolation for pansharpening task. The main idea is achieving high-quality fusion results by interpolating two low-quality multispectral images in a deep neural network. We apply it to two conditions: 1) unsupervised training a network when there are enough datasets; 2) directly optimizing the fusion result where no training datasets are available. Simulation and real-data experiments are conducted on various kinds of satellite data. Quantitative and qualitative evaluation results illustrate that the proposed method outperforms traditional pansharpening methods and even catch up with those supervised methods to some extent.