An Analysis of Linear Models, Linear Value-Function Approximation, and Feature Selection for Reinforcement Learning
Abstract
We show that linear value function approximation is equivalent to a form of linear model approximation. We derive a relationship between the model approximation error and the Bellman error, and show how this relationship can guide feature selection for model improvement and/or value function improvement. We also show how these results give insight into the behavior of existing feature-selection algorithms.
Cite
Text
Parr et al. "An Analysis of Linear Models, Linear Value-Function Approximation, and Feature Selection for Reinforcement Learning." International Conference on Machine Learning, 2008. doi:10.1145/1390156.1390251Markdown
[Parr et al. "An Analysis of Linear Models, Linear Value-Function Approximation, and Feature Selection for Reinforcement Learning." International Conference on Machine Learning, 2008.](https://mlanthology.org/icml/2008/parr2008icml-analysis/) doi:10.1145/1390156.1390251BibTeX
@inproceedings{parr2008icml-analysis,
title = {{An Analysis of Linear Models, Linear Value-Function Approximation, and Feature Selection for Reinforcement Learning}},
author = {Parr, Ronald and Li, Lihong and Taylor, Gavin and Painter-Wakefield, Christopher and Littman, Michael L.},
booktitle = {International Conference on Machine Learning},
year = {2008},
pages = {752-759},
doi = {10.1145/1390156.1390251},
url = {https://mlanthology.org/icml/2008/parr2008icml-analysis/}
}