Space-Time-Aware Multi-Resolution Video Enhancement
Abstract
We consider the problem of space-time super-resolution (ST-SR): increasing spatial resolution of video frames and simultaneously interpolating frames to increase the frame rate. Modern approaches handle these axes one at a time. In contrast, our proposed model called STARnet super-resolves jointly in space and time. This allows us to leverage mutually informative relationships between time and space: higher resolution can provide more detailed information about motion, and higher frame-rate can provide better pixel alignment. The components of our model that generate latent low- and high-resolution representations during ST-SR can be used to finetune a specialized mechanism for just spatial or just temporal super-resolution. Experimental results demonstrate that STARnet improves the performances of space-time, spatial, and temporal video super-resolution by substantial margins on publicly available datasets.
Cite
Text
Haris et al. "Space-Time-Aware Multi-Resolution Video Enhancement." Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020. doi:10.1109/CVPR42600.2020.00293Markdown
[Haris et al. "Space-Time-Aware Multi-Resolution Video Enhancement." Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020.](https://mlanthology.org/cvpr/2020/haris2020cvpr-spacetimeaware/) doi:10.1109/CVPR42600.2020.00293BibTeX
@inproceedings{haris2020cvpr-spacetimeaware,
title = {{Space-Time-Aware Multi-Resolution Video Enhancement}},
author = {Haris, Muhammad and Shakhnarovich, Greg and Ukita, Norimichi},
booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition},
year = {2020},
doi = {10.1109/CVPR42600.2020.00293},
url = {https://mlanthology.org/cvpr/2020/haris2020cvpr-spacetimeaware/}
}