Regular Decision Processes: A Model for Non-Markovian Domains
Abstract
We introduce and study Regular Decision Processes (RDPs), a new, compact, factored model for domains with non-Markovian dynamics and rewards. In RDPs, transition and reward functions are specified using formulas in linear dynamic logic over finite traces, a language with the expressive power of regular expressions. This allows specifying complex dependence on the past using intuitive and compact formulas, and provides a model that generalizes MDPs and k-order MDPs. RDPs can also approximate POMDPs without having to postulate the existence of hidden variables, and, in principle, can be learned from observations only.
Cite
Text
Brafman and De Giacomo. "Regular Decision Processes: A Model for Non-Markovian Domains." International Joint Conference on Artificial Intelligence, 2019. doi:10.24963/IJCAI.2019/766Markdown
[Brafman and De Giacomo. "Regular Decision Processes: A Model for Non-Markovian Domains." International Joint Conference on Artificial Intelligence, 2019.](https://mlanthology.org/ijcai/2019/brafman2019ijcai-regular/) doi:10.24963/IJCAI.2019/766BibTeX
@inproceedings{brafman2019ijcai-regular,
title = {{Regular Decision Processes: A Model for Non-Markovian Domains}},
author = {Brafman, Ronen I. and De Giacomo, Giuseppe},
booktitle = {International Joint Conference on Artificial Intelligence},
year = {2019},
pages = {5516-5522},
doi = {10.24963/IJCAI.2019/766},
url = {https://mlanthology.org/ijcai/2019/brafman2019ijcai-regular/}
}