The Thing That We Tried Didn't Work Very Well: Deictic Representation in Reinforcement Learning
Abstract
Most reinforcement learning methods operate on propositional representations of the world state. Such representations are often intractably large and generalize poorly. Using a deictic representation is believed to be a viable alternative: they promise generalization while allowing the use of existing reinforcement-learning methods. Yet, there are few experiments on learning with deictic representations reported in the literature. In this paper we explore the effectiveness of two forms of deictic representation and a naïve propositional representation in a simple blocks-world domain. We find, empirically, that the deictic representations actually worsen learning performance. We conclude with a discussion of possible causes of these results and strategies for more effective learning in domains with objects.
Cite
Text
Finney et al. "The Thing That We Tried Didn't Work Very Well: Deictic Representation in Reinforcement Learning." Conference on Uncertainty in Artificial Intelligence, 2002.Markdown
[Finney et al. "The Thing That We Tried Didn't Work Very Well: Deictic Representation in Reinforcement Learning." Conference on Uncertainty in Artificial Intelligence, 2002.](https://mlanthology.org/uai/2002/finney2002uai-thing/)BibTeX
@inproceedings{finney2002uai-thing,
title = {{The Thing That We Tried Didn't Work Very Well: Deictic Representation in Reinforcement Learning}},
author = {Finney, Sarah and Gardiol, Natalia and Kaelbling, Leslie Pack and Oates, Tim},
booktitle = {Conference on Uncertainty in Artificial Intelligence},
year = {2002},
pages = {154-161},
url = {https://mlanthology.org/uai/2002/finney2002uai-thing/}
}