PAE: Reinforcement Learning from External Knowledge for Efficient Exploration
Abstract
Human intelligence is adept at absorbing valuable insights from external knowledge. This capability is equally crucial for artificial intelligence. In contrast, classical reinforcement learning agents lack such capabilities and often resort to extensive trial and error to explore the environment. This paper introduces $\textbf{PAE}$: $\textbf{P}$lanner-$\textbf{A}$ctor-$\textbf{E}$valuator, a novel framework for teaching agents to $\textit{learn to absorb external knowledge}$. PAE integrates the Planner's knowledge-state alignment mechanism, the Actor's mutual information skill control, and the Evaluator's adaptive intrinsic exploration reward to achieve 1) effective cross-modal information fusion, 2) enhanced linkage between knowledge and state, and 3) hierarchical mastery of complex tasks. Comprehensive experiments across 11 challenging tasks from the BabyAI and MiniHack environment suites demonstrate PAE's superior exploration efficiency with good interpretability.
Cite
Text
Wu et al. "PAE: Reinforcement Learning from External Knowledge for Efficient Exploration." International Conference on Learning Representations, 2024.Markdown
[Wu et al. "PAE: Reinforcement Learning from External Knowledge for Efficient Exploration." International Conference on Learning Representations, 2024.](https://mlanthology.org/iclr/2024/wu2024iclr-pae/)BibTeX
@inproceedings{wu2024iclr-pae,
title = {{PAE: Reinforcement Learning from External Knowledge for Efficient Exploration}},
author = {Wu, Zhe and Lu, Haofei and Xing, Junliang and Wu, You and Yan, Renye and Gan, Yaozhong and Shi, Yuanchun},
booktitle = {International Conference on Learning Representations},
year = {2024},
url = {https://mlanthology.org/iclr/2024/wu2024iclr-pae/}
}