ML Anthology

Solving Generalized Semi-Markov Decision Processes Using Continuous Phase-Type Distributions

Håkan L. S. Younes, Reid G. Simmons
AAAI 2004 pp. 742-748
doi:10.1073/pnas.2418097121 /aaai/2004/younes2004aaai-solving/

Abstract

We introduce the generalized semi-Markov decision process (GSMDP) as an extension of continuous-time MDPs and semi-Markov decision processes (SMDPs) for modeling stochastic decision processes with asynchronous events and actions. Using phase-type distributions and uniformization, we show how an arbitrary GSMDP can be approximated by a discrete-time MDP, which can then be solved using existing MDP techniques. The techniques we present can also be seen as an alternative approach for solving SMDPs, and we demonstrate that the introduction of phases allows us to generate higher quality policies than those obtained by standard SMDP solution techniques.

PDF AAAI Semantic Scholar

Cite

Text

Younes and Simmons. "Solving Generalized Semi-Markov Decision Processes Using Continuous Phase-Type Distributions." AAAI Conference on Artificial Intelligence, 2004. doi:10.1073/pnas.2418097121

Markdown

[Younes and Simmons. "Solving Generalized Semi-Markov Decision Processes Using Continuous Phase-Type Distributions." AAAI Conference on Artificial Intelligence, 2004.](https://mlanthology.org/aaai/2004/younes2004aaai-solving/) doi:10.1073/pnas.2418097121

BibTeX

@inproceedings{younes2004aaai-solving,
  title     = {{Solving Generalized Semi-Markov Decision Processes Using Continuous Phase-Type Distributions}},
  author    = {Younes, Håkan L. S. and Simmons, Reid G.},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year      = {2004},
  pages     = {742-748},
  doi       = {10.1073/pnas.2418097121},
  url       = {https://mlanthology.org/aaai/2004/younes2004aaai-solving/}
}