High-Quality Single-Model Deep Video Compression with Frame-Conv3D and Multi-Frame Differential Modulation

Abstract

Deep learning (DL) methods have revolutionized the paradigm of computer vision tasks and DL-based video compression is becoming a hot topic. This paper proposes a deep video compression method to simultaneously encode multiple frames with Frame-Conv3D and differential modulation. We first adopt Frame-Conv3D instead of traditional Channel-Conv3D for efficient multi-frame fusion. When generating the binary representation, the multi-frame differential modulation is utilized to alleviate the effect of quantization noise. By analyzing the forward and backward computing flow of the modulator, we identify that this technique can make full use of past frames' information to remove the redundancy between multiple frames, and thus achieves better performance. A dropout scheme combined with the differential modulator is proposed to enable bit rate optimization within a single model. Experimental results show that the proposed approach outperforms the H.264 and H.265 codecs in the region of low bit rate. Compared with recent DL-based methods, our model also achieves competitive performance.