Guiding Video Prediction with Explicit Procedural Knowledge

Patrick Takenaka, Johannes Maucher, Marco F. Huber

ICCVW 2023 pp. 1076-1084

doi:10.1109/ICCVW60793.2023.00116 /iccvw/2023/takenaka2023iccvw-guiding/

Abstract

We propose a general way to integrate procedural knowledge of a domain into deep learning models. We apply it to the case of video prediction, building on top of object-centric deep models and show that this leads to a better performance than using data-driven models alone. We develop an architecture that facilitates latent space disentanglement in order to use the integrated procedural knowledge, and establish a setup that allows the model to learn the procedural interface in the latent space using the downstream task of video prediction. We contrast the performance to a state-of-the-art data-driven approach and show that problems where purely data-driven approaches struggle can be handled by using knowledge about the domain, providing an alternative to simply collecting more data.

ICCVW Semantic Scholar

Cite

Text

Takenaka et al. "Guiding Video Prediction with Explicit Procedural Knowledge." IEEE/CVF International Conference on Computer Vision Workshops, 2023. doi:10.1109/ICCVW60793.2023.00116

Markdown

[Takenaka et al. "Guiding Video Prediction with Explicit Procedural Knowledge." IEEE/CVF International Conference on Computer Vision Workshops, 2023.](https://mlanthology.org/iccvw/2023/takenaka2023iccvw-guiding/) doi:10.1109/ICCVW60793.2023.00116

BibTeX

@inproceedings{takenaka2023iccvw-guiding,
  title     = {{Guiding Video Prediction with Explicit Procedural Knowledge}},
  author    = {Takenaka, Patrick and Maucher, Johannes and Huber, Marco F.},
  booktitle = {IEEE/CVF International Conference on Computer Vision Workshops},
  year      = {2023},
  pages     = {1076-1084},
  doi       = {10.1109/ICCVW60793.2023.00116},
  url       = {https://mlanthology.org/iccvw/2023/takenaka2023iccvw-guiding/}
}