Know Your Action Set: Learning Action Relations for Reinforcement Learning
Abstract
Intelligent agents can solve tasks in various ways depending on their available set of actions. However, conventional reinforcement learning (RL) assumes a fixed action set. This work asserts that tasks with varying action sets require reasoning of the relations between the available actions. For instance, taking a nail-action in a repair task is meaningful only if a hammer-action is also available. To learn and utilize such action relations, we propose a novel policy architecture consisting of a graph attention network over the available actions. We show that our model makes informed action decisions by correctly attending to other related actions in both value-based and policy-based RL. Consequently, it outperforms non-relational architectures on applications where the action space often varies, such as recommender systems and physical reasoning with tools and skills. Results and code at https://sites.google.com/view/varyingaction .
Cite
Text
Jain et al. "Know Your Action Set: Learning Action Relations for Reinforcement Learning." International Conference on Learning Representations, 2022.Markdown
[Jain et al. "Know Your Action Set: Learning Action Relations for Reinforcement Learning." International Conference on Learning Representations, 2022.](https://mlanthology.org/iclr/2022/jain2022iclr-know/)BibTeX
@inproceedings{jain2022iclr-know,
title = {{Know Your Action Set: Learning Action Relations for Reinforcement Learning}},
author = {Jain, Ayush and Kosaka, Norio and Kim, Kyung-Min and Lim, Joseph J},
booktitle = {International Conference on Learning Representations},
year = {2022},
url = {https://mlanthology.org/iclr/2022/jain2022iclr-know/}
}