Overcoming Incomplete Perception with Utile Distinction Memory
Abstract
This paper presents a method by which a reinforcement learning agent can solve the incomplete perception problem using memory. The agent uses a hidden Markov model (HMM) to represent its internal state space and creates memory capacity by splitting states of the HMM. The key idea is a test to determine when and how a state should be split: the agent only splits a state when doing so will help the agent predict utility. Thus the agent can create only as much memory as needed to perform the task at hand—not as much as would be required to model all the perceivable world. I call the technique UDM, for Utile Distinction Memory.
Cite
Text
McCallum. "Overcoming Incomplete Perception with Utile Distinction Memory." International Conference on Machine Learning, 1993. doi:10.1016/B978-1-55860-307-3.50031-9Markdown
[McCallum. "Overcoming Incomplete Perception with Utile Distinction Memory." International Conference on Machine Learning, 1993.](https://mlanthology.org/icml/1993/mccallum1993icml-overcoming/) doi:10.1016/B978-1-55860-307-3.50031-9BibTeX
@inproceedings{mccallum1993icml-overcoming,
title = {{Overcoming Incomplete Perception with Utile Distinction Memory}},
author = {McCallum, R. Andrew},
booktitle = {International Conference on Machine Learning},
year = {1993},
pages = {190-196},
doi = {10.1016/B978-1-55860-307-3.50031-9},
url = {https://mlanthology.org/icml/1993/mccallum1993icml-overcoming/}
}