Option Encoder: A Framework for Discovering a Policy Basis in Reinforcement Learning
Abstract
Option discovery and skill acquisition frameworks are integral to the functioning of a Hierarchically organized Reinforcement learning agent. However, such techniques often yield a large number of options or skills, which can potentially be represented succinctly by filtering out any redundant information. Such a reduction can reduce the required computation while also improving the performance on a target task. In order to compress an array of option policies, we attempt to find a policy basis that accurately captures the set of all options. In this work, we propose Option Encoder, an auto-encoder based framework with intelligently constrained weights, that helps discover a collection of basis policies. The policy basis can be used as a proxy for the original set of skills in a suitable hierarchically organized framework. We demonstrate the efficacy of our method on a collection of grid-worlds and on the high-dimensional Fetch-Reach robotic manipulation task by evaluating the obtained policy basis on a set of downstream tasks.
Cite
Text
Manoharan et al. "Option Encoder: A Framework for Discovering a Policy Basis in Reinforcement Learning." European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases, 2020. doi:10.1007/978-3-030-67661-2_30Markdown
[Manoharan et al. "Option Encoder: A Framework for Discovering a Policy Basis in Reinforcement Learning." European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases, 2020.](https://mlanthology.org/ecmlpkdd/2020/manoharan2020ecmlpkdd-option/) doi:10.1007/978-3-030-67661-2_30BibTeX
@inproceedings{manoharan2020ecmlpkdd-option,
title = {{Option Encoder: A Framework for Discovering a Policy Basis in Reinforcement Learning}},
author = {Manoharan, Arjun and Ramesh, Rahul and Ravindran, Balaraman},
booktitle = {European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases},
year = {2020},
pages = {509-524},
doi = {10.1007/978-3-030-67661-2_30},
url = {https://mlanthology.org/ecmlpkdd/2020/manoharan2020ecmlpkdd-option/}
}