SR-CL-DMC: P-Frame Coding with Super-Resolution, Color Learning, and Deep Motion Compensation

Abstract

This paper proposes a deep learning based video coding framework to greatly increase the compression ratio and keep the video quality by efficiently leveraging the information from a reference. In the encoder, the input frame is compressed by down-sampling to a lower resolution, eliminating color information, and then encoding residual between the current frame and the reference frame using Versatile Video Coding (VVC). The decoder consists of two main parts: Super-Resolution with Color Learning (SR-CL), and Deep Motion Compensation (DMC). For the SR-CL part, we adopt Restoration-Reconstruction Deep Neural Network to firstly restore the missing information from compression at low resolution and compression without color. And then, the sampling degradation at high-resolution is compensated. For the DMC part, we adopt recursive-feedback architectures to propose an optical flow estimation and refinement using Dilated Inception Blocks. As a result, the work achieves 64:1 compression ratio with 41.81/41.34 dB PSNR and 0.9959/0.9962 MS-SSIM on the validation/test set provided by the CLIC P-frame track challenge.