Deep Space-Time Video Upsampling Networks
Abstract
Video super-resolution (VSR) and frame interpolation (FI) are traditional computer vision problems, and the performance have been improving by incorporating deep learning recently. In this paper, we investigate the problem of jointly upsampling videos both in space and time, which is becoming more important with advances in display systems. One solution for this is to run VSR and FI, one by one, independently. This is highly inefficient as heavy deep neural networks (DNN) are involved in each solution. To this end, we propose an end-to-end DNN framework for the space-time video upsampling by efficiently merging VSR and FI into a joint framework. In our framework, a novel weighting scheme is proposed to fuse input frames effectively without explicit motion compensation for efficient processing of videos. The results show better results both quantitatively and qualitatively, while reducing the computation time (x7 faster) and the number of parameters (30%) compared to baselines. Our source code is available at https://github.com/JaeYeonKang/STVUN-Pytorch.
Cite
Text
Kang et al. "Deep Space-Time Video Upsampling Networks." Proceedings of the European Conference on Computer Vision (ECCV), 2020. doi:10.1007/978-3-030-58607-2_41Markdown
[Kang et al. "Deep Space-Time Video Upsampling Networks." Proceedings of the European Conference on Computer Vision (ECCV), 2020.](https://mlanthology.org/eccv/2020/kang2020eccv-deep/) doi:10.1007/978-3-030-58607-2_41BibTeX
@inproceedings{kang2020eccv-deep,
title = {{Deep Space-Time Video Upsampling Networks}},
author = {Kang, Jaeyeon and Jo, Younghyun and Oh, Seoung Wug and Vajda, Peter and Kim, Seon Joo},
booktitle = {Proceedings of the European Conference on Computer Vision (ECCV)},
year = {2020},
doi = {10.1007/978-3-030-58607-2_41},
url = {https://mlanthology.org/eccv/2020/kang2020eccv-deep/}
}