Hyperspectral Image Super-Resolution with RGB Image Super-Resolution as an Auxiliary Task

Abstract

This work studies Hyperspectral image (HSI) super-resolution (SR). HSI SR is characterized by high-dimensional data and a limited amount of training exam-ples. This raises challenges for training deep neural net-works that are known to be data hungry. This work ad-dresses this issue with two contributions. First, we observethat HSI SR and RGB image SR are correlated and developa novel multi-tasking network to train them jointly so thatthe auxiliary task RGB image SR can provide additionalsupervision and regulate the network training. Second,we extend the network to a semi-supervised setting so thatit can learn from datasets containing only low-resolutionHSIs. With these contributions, our method is able to learnhyperspectral image super-resolution from heterogeneousdatasets and lifts the requirement for having a large amountof HD HSI training samples. Extensive experiments onthree standard datasets show that our method outperformsexisting methods significantly and underpin the relevance ofour contributions.