Policy Gradients in Linearly-Solvable MDPs
Abstract
We present policy gradient results within the framework of linearly-solvable MDPs. For the first time, compatible function approximators and natural policy gradients are obtained by estimating the cost-to-go function, rather than the (much larger) state-action advantage function as is necessary in traditional MDPs. We also develop the first compatible function approximators and natural policy gradients for continuous-time stochastic systems.
Cite
Text
Todorov. "Policy Gradients in Linearly-Solvable MDPs." Neural Information Processing Systems, 2010.Markdown
[Todorov. "Policy Gradients in Linearly-Solvable MDPs." Neural Information Processing Systems, 2010.](https://mlanthology.org/neurips/2010/todorov2010neurips-policy/)BibTeX
@inproceedings{todorov2010neurips-policy,
title = {{Policy Gradients in Linearly-Solvable MDPs}},
author = {Todorov, Emanuel},
booktitle = {Neural Information Processing Systems},
year = {2010},
pages = {2298-2306},
url = {https://mlanthology.org/neurips/2010/todorov2010neurips-policy/}
}