Active Policy Iteration: Efficient Exploration Through Active Learning for Value Function Approximation in Reinforcement Learning
Abstract
Appropriately designing sampling policies is highly important for obtaining better control policies in reinforcement learning. In this paper, we first show that the least-squares policy iteration (LSPI) framework allows us to employ statistical active learning methods for linear regression. Then we propose a design method of good sampling policies for efficient exploration, which is particularly useful when the sampling cost of immediate rewards is high. The effectiveness of the proposed method, which we call active policy iteration (API), is demonstrated through simulations with a batting robot.
Cite
Text
Akiyama et al. "Active Policy Iteration: Efficient Exploration Through Active Learning for Value Function Approximation in Reinforcement Learning." International Joint Conference on Artificial Intelligence, 2009. doi:10.1016/j.neunet.2009.12.010Markdown
[Akiyama et al. "Active Policy Iteration: Efficient Exploration Through Active Learning for Value Function Approximation in Reinforcement Learning." International Joint Conference on Artificial Intelligence, 2009.](https://mlanthology.org/ijcai/2009/akiyama2009ijcai-active/) doi:10.1016/j.neunet.2009.12.010BibTeX
@inproceedings{akiyama2009ijcai-active,
title = {{Active Policy Iteration: Efficient Exploration Through Active Learning for Value Function Approximation in Reinforcement Learning}},
author = {Akiyama, Takayuki and Hachiya, Hirotaka and Sugiyama, Masashi},
booktitle = {International Joint Conference on Artificial Intelligence},
year = {2009},
pages = {980-985},
doi = {10.1016/j.neunet.2009.12.010},
url = {https://mlanthology.org/ijcai/2009/akiyama2009ijcai-active/}
}