Reinforcement Learning with Limited Reinforcement: Using Bayes Risk for Active Learning in POMDPs
Abstract
Partially Observable Markov Decision Processes (POMDPs) have succeeded in planning domains because they optimally trade between actions that increase an agent's knowledge and actions that increase an agent's reward. Unfortunately, most POMDPs are defined with a large number of parameters which are difficult to specify only from domain knowledge. In this paper, we treat the POMDP model parameters as additional hidden state in a "model-uncertainty" POMDP and develop an approximate algorithm for planning in the this larger POMDP. The approximation, coupled with model-directed queries, allows the planner to actively learn good policies. We demonstrate our approach on several standard POMDP problems.
Cite
Text
Doshi et al. "Reinforcement Learning with Limited Reinforcement: Using Bayes Risk for Active Learning in POMDPs." International Conference on Machine Learning, 2008. doi:10.1145/1390156.1390189Markdown
[Doshi et al. "Reinforcement Learning with Limited Reinforcement: Using Bayes Risk for Active Learning in POMDPs." International Conference on Machine Learning, 2008.](https://mlanthology.org/icml/2008/doshi2008icml-reinforcement/) doi:10.1145/1390156.1390189BibTeX
@inproceedings{doshi2008icml-reinforcement,
title = {{Reinforcement Learning with Limited Reinforcement: Using Bayes Risk for Active Learning in POMDPs}},
author = {Doshi, Finale and Pineau, Joelle and Roy, Nicholas},
booktitle = {International Conference on Machine Learning},
year = {2008},
pages = {256-263},
doi = {10.1145/1390156.1390189},
url = {https://mlanthology.org/icml/2008/doshi2008icml-reinforcement/}
}