Relational Abstractions for Generalized Reinforcement Learning on Symbolic Problems
Abstract
Reinforcement learning in problems with symbolic state spaces is challenging due to the need for reasoning over long horizons. This paper presents a new approach that utilizes relational abstractions in conjunction with deep learning to learn a generalizable Q-function for such problems. The learned Q-function can be efficiently transferred to related problems that have different object names and object quantities, and thus, entirely different state spaces. We show that the learned, generalized Q-function can be utilized for zero-shot transfer to related problems without an explicit, hand-coded curriculum. Empirical evaluations on a range of problems show that our method facilitates efficient zero-shot transfer of learned knowledge to much larger problem instances containing many objects.
Cite
Text
Karia and Srivastava. "Relational Abstractions for Generalized Reinforcement Learning on Symbolic Problems." International Joint Conference on Artificial Intelligence, 2022. doi:10.24963/IJCAI.2022/435Markdown
[Karia and Srivastava. "Relational Abstractions for Generalized Reinforcement Learning on Symbolic Problems." International Joint Conference on Artificial Intelligence, 2022.](https://mlanthology.org/ijcai/2022/karia2022ijcai-relational/) doi:10.24963/IJCAI.2022/435BibTeX
@inproceedings{karia2022ijcai-relational,
title = {{Relational Abstractions for Generalized Reinforcement Learning on Symbolic Problems}},
author = {Karia, Rushang and Srivastava, Siddharth},
booktitle = {International Joint Conference on Artificial Intelligence},
year = {2022},
pages = {3135-3142},
doi = {10.24963/IJCAI.2022/435},
url = {https://mlanthology.org/ijcai/2022/karia2022ijcai-relational/}
}