Characterizing Markov Decision Processes
Abstract
Problem characteristics often have a significant influence on the difficulty of solving optimization problems. In this paper, we propose attributes for characterizing Markov Decision Processes (MDPs), and discuss how they affect the performance of reinforcement learning algorithms that use function approximation. The attributes measure mainly the amount of randomness in the environment. Their values can be calculated from the MDP model or estimated on-line. We show empirically that two of the proposed attributes have a statistically significant effect on the quality of learning. We discuss how measurements of the proposed MDP attributes can be used to facilitate the design of reinforcement learning systems.
Cite
Text
Ratitch and Precup. "Characterizing Markov Decision Processes." European Conference on Machine Learning, 2002. doi:10.1007/3-540-36755-1_33Markdown
[Ratitch and Precup. "Characterizing Markov Decision Processes." European Conference on Machine Learning, 2002.](https://mlanthology.org/ecmlpkdd/2002/ratitch2002ecml-characterizing/) doi:10.1007/3-540-36755-1_33BibTeX
@inproceedings{ratitch2002ecml-characterizing,
title = {{Characterizing Markov Decision Processes}},
author = {Ratitch, Bohdana and Precup, Doina},
booktitle = {European Conference on Machine Learning},
year = {2002},
pages = {391-404},
doi = {10.1007/3-540-36755-1_33},
url = {https://mlanthology.org/ecmlpkdd/2002/ratitch2002ecml-characterizing/}
}