Explicit Guidance for Robust Video Frame Interpolation Against Discontinuous Motions

Abstract

Nowadays many videos contain graphic elements such as logos subtitles and user interfaces. These overlayed elements exhibit discontinuous motions characterized by static or instantaneous motions that are neither spatially nor temporally coherent. As existing Video Frame Interpolation (VFI) methods rely on motion-compensation techniques they work best for videos with continuous motions but face limitations against videos with discontinuous motion. In this paper we propose a simple framework to enhance the robustness of existing VFI models against discontinuous motion. We first identify key properties that distinguish discontinuous from continuous motion. They are then leveraged by the Discontinuity map (D-map) estimator to explicitly guide the classification of continuous and discontinuous areas through a coherence mask and an additional supervisory signal. Our framework separately interpolates the predicted continuous and discontinuous regions to achieve state-of-the-art performance against synthetic discontinuous motions while also generalizing well to real-world discontinuous motions. Moreover our framework's `plug-and-play' design enables easy application to existing VFI models without the need for retraining and maintains strong performance on continuous motion.