ML Anthology

IR-VIC: Unsupervised Discovery of Sub-Goals for Transfer in RL

Nirbhay Modhe, Prithvijit Chattopadhyay, Mohit Sharma, Abhishek Das, Devi Parikh, Dhruv Batra, Ramakrishna Vedantam
IJCAI 2020 pp. 2022-2028
doi:10.24963/IJCAI.2020/280 /ijcai/2020/modhe2020ijcai-ir/

Abstract

We propose a novel framework to identify sub-goals useful for exploration in sequential decision making tasks under partial observability. We utilize the variational intrinsic control framework (Gregor et.al., 2016) which maximizes empowerment -- the ability to reliably reach a diverse set of states and show how to identify sub-goals as states with high necessary option information through an information theoretic regularizer. Despite being discovered without explicit goal supervision, our sub-goals provide better exploration and sample complexity on challenging grid-world navigation tasks compared to supervised counterparts in prior work.

PDF IJCAI Semantic Scholar

Cite

Text

Modhe et al. "IR-VIC: Unsupervised Discovery of Sub-Goals for Transfer in RL." International Joint Conference on Artificial Intelligence, 2020. doi:10.24963/IJCAI.2020/280

Markdown

[Modhe et al. "IR-VIC: Unsupervised Discovery of Sub-Goals for Transfer in RL." International Joint Conference on Artificial Intelligence, 2020.](https://mlanthology.org/ijcai/2020/modhe2020ijcai-ir/) doi:10.24963/IJCAI.2020/280

BibTeX

@inproceedings{modhe2020ijcai-ir,
  title     = {{IR-VIC: Unsupervised Discovery of Sub-Goals for Transfer in RL}},
  author    = {Modhe, Nirbhay and Chattopadhyay, Prithvijit and Sharma, Mohit and Das, Abhishek and Parikh, Devi and Batra, Dhruv and Vedantam, Ramakrishna},
  booktitle = {International Joint Conference on Artificial Intelligence},
  year      = {2020},
  pages     = {2022-2028},
  doi       = {10.24963/IJCAI.2020/280},
  url       = {https://mlanthology.org/ijcai/2020/modhe2020ijcai-ir/}
}