Consciousness-Inspired Spatio-Temporal Abstractions for Better Generalization in Reinforcement Learning
Abstract
Inspired by human conscious planning, we propose Skipper, a model-based reinforcement learning framework utilizing spatio-temporal abstractions to generalize better in novel situations. It automatically decomposes the given task into smaller, more manageable subtasks, and thus enables sparse decision-making and focused computation on the relevant parts of the environment. The decomposition relies on the extraction of an abstracted proxy problem represented as a directed graph, in which vertices and edges are learned end-to-end from hindsight. Our theoretical analyses provide performance guarantees under appropriate assumptions and establish where our approach is expected to be helpful. Generalization-focused experiments validate Skipper's significant advantage in zero-shot generalization, compared to some existing state-of-the-art hierarchical planning methods.
Cite
Text
Zhao et al. "Consciousness-Inspired Spatio-Temporal Abstractions for Better Generalization in Reinforcement Learning." NeurIPS 2024 Workshops: Sys2-Reasoning, 2024.Markdown
[Zhao et al. "Consciousness-Inspired Spatio-Temporal Abstractions for Better Generalization in Reinforcement Learning." NeurIPS 2024 Workshops: Sys2-Reasoning, 2024.](https://mlanthology.org/neuripsw/2024/zhao2024neuripsw-consciousnessinspired/)BibTeX
@inproceedings{zhao2024neuripsw-consciousnessinspired,
title = {{Consciousness-Inspired Spatio-Temporal Abstractions for Better Generalization in Reinforcement Learning}},
author = {Zhao, Harry and Alver, Safa and van Seijen, Harm and Laroche, Romain and Precup, Doina and Bengio, Yoshua},
booktitle = {NeurIPS 2024 Workshops: Sys2-Reasoning},
year = {2024},
url = {https://mlanthology.org/neuripsw/2024/zhao2024neuripsw-consciousnessinspired/}
}