MoTIF: Learning Motion Trajectories with Local Implicit Neural Functions for Continuous Space-Time Video Super-Resolution

Yi-Hsin Chen, Si-Cun Chen, Yi-Hsin Chen, Yen-Yu Lin, Wen-Hsiao Peng

ICCV 2023 pp. 23131-23141

doi:10.1109/ICCV51070.2023.02114 /iccv/2023/chen2023iccv-motif/

Abstract

This work addresses continuous space-time video super-resolution (C-STVSR) that aims to up-scale an input video both spatially and temporally by any scaling factors. One key challenge of C-STVSR is to propagate information temporally among the input video frames. To this end, we introduce a space-time local implicit neural function. It has the striking feature of learning forward motion for a continuum of pixels. We motivate the use of forward motion from the perspective of learning individual motion trajectories, as opposed to learning a mixture of motion trajectories with backward motion. To ease motion interpolation, we encode sparsely sampled forward motion extracted from the input video as the contextual input. Along with a reliability-aware splatting and decoding scheme, our framework, termed MoTIF, achieves the state-of-the-art performance on C-STVSR. The source code of MoTIF is available at https://github.com/sichun233746/MoTIF.

PDF ICCV Semantic Scholar

Cite

Text

Chen et al. "MoTIF: Learning Motion Trajectories with Local Implicit Neural Functions for Continuous Space-Time Video Super-Resolution." International Conference on Computer Vision, 2023. doi:10.1109/ICCV51070.2023.02114

Markdown

[Chen et al. "MoTIF: Learning Motion Trajectories with Local Implicit Neural Functions for Continuous Space-Time Video Super-Resolution." International Conference on Computer Vision, 2023.](https://mlanthology.org/iccv/2023/chen2023iccv-motif/) doi:10.1109/ICCV51070.2023.02114

BibTeX

@inproceedings{chen2023iccv-motif,
  title     = {{MoTIF: Learning Motion Trajectories with Local Implicit Neural Functions for Continuous Space-Time Video Super-Resolution}},
  author    = {Chen, Yi-Hsin and Chen, Si-Cun and Chen, Yi-Hsin and Lin, Yen-Yu and Peng, Wen-Hsiao},
  booktitle = {International Conference on Computer Vision},
  year      = {2023},
  pages     = {23131-23141},
  doi       = {10.1109/ICCV51070.2023.02114},
  url       = {https://mlanthology.org/iccv/2023/chen2023iccv-motif/}
}