Planning with Abstract Learned Models While Learning Transferable Subtasks

John Winder, Stephanie Milani, Matthew Landen, Erebus Oh, Shane Parr, Shawn Squire, Marie desJardins, Cynthia Matuszek

AAAI 2020 pp. 9992-10000

doi:10.1609/AAAI.V34I06.6555 /aaai/2020/winder2020aaai-planning/

Abstract

We introduce an algorithm for model-based hierarchical reinforcement learning to acquire self-contained transition and reward models suitable for probabilistic planning at multiple levels of abstraction. We call this framework Planning with Abstract Learned Models (PALM). By representing subtasks symbolically using a new formal structure, the lifted abstract Markov decision process (L-AMDP), PALM learns models that are independent and modular. Through our experiments, we show how PALM integrates planning and execution, facilitating a rapid and efficient learning of abstract, hierarchical models. We also demonstrate the increased potential for learned models to be transferred to new and related tasks.

PDF AAAI Semantic Scholar

Cite

Text

Winder et al. "Planning with Abstract Learned Models While Learning Transferable Subtasks." AAAI Conference on Artificial Intelligence, 2020. doi:10.1609/AAAI.V34I06.6555

Markdown

[Winder et al. "Planning with Abstract Learned Models While Learning Transferable Subtasks." AAAI Conference on Artificial Intelligence, 2020.](https://mlanthology.org/aaai/2020/winder2020aaai-planning/) doi:10.1609/AAAI.V34I06.6555

BibTeX

@inproceedings{winder2020aaai-planning,
  title     = {{Planning with Abstract Learned Models While Learning Transferable Subtasks}},
  author    = {Winder, John and Milani, Stephanie and Landen, Matthew and Oh, Erebus and Parr, Shane and Squire, Shawn and desJardins, Marie and Matuszek, Cynthia},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year      = {2020},
  pages     = {9992-10000},
  doi       = {10.1609/AAAI.V34I06.6555},
  url       = {https://mlanthology.org/aaai/2020/winder2020aaai-planning/}
}