Scaling Goal-Based Exploration via Pruning Proto-Goals
Abstract
One of the gnarliest challenges in reinforcement learning (RL) is exploration that scales to vast domains, where novelty-, or coverage-seeking behaviour falls short. Goal-directed, purposeful behaviours are able to overcome this, but rely on a good goal space. The core challenge in goal discovery is finding the right balance between generality (not hand-crafted) and tractability (useful, not too many). Our approach explicitly seeks the middle ground, enabling the human designer to specify a vast but meaningful proto-goal space, and an autonomous discovery process to refine this to a narrower space of controllable, reachable, novel, and relevant goals. The effectiveness of goal-conditioned exploration with the latter is then demonstrated in three challenging environments.
Cite
Text
Bagaria and Schaul. "Scaling Goal-Based Exploration via Pruning Proto-Goals." International Joint Conference on Artificial Intelligence, 2023. doi:10.24963/IJCAI.2023/384Markdown
[Bagaria and Schaul. "Scaling Goal-Based Exploration via Pruning Proto-Goals." International Joint Conference on Artificial Intelligence, 2023.](https://mlanthology.org/ijcai/2023/bagaria2023ijcai-scaling/) doi:10.24963/IJCAI.2023/384BibTeX
@inproceedings{bagaria2023ijcai-scaling,
title = {{Scaling Goal-Based Exploration via Pruning Proto-Goals}},
author = {Bagaria, Akhil and Schaul, Tom},
booktitle = {International Joint Conference on Artificial Intelligence},
year = {2023},
pages = {3451-3460},
doi = {10.24963/IJCAI.2023/384},
url = {https://mlanthology.org/ijcai/2023/bagaria2023ijcai-scaling/}
}