A Causal Approach to Hierarchical Decomposition of Factored MDPs
Abstract
We present Variable Influence Structure Analysis, an algorithm that dynamically performs hierarchical decomposition of factored Markov decision processes. Our algorithm determines causal relationships between state variables and introduces temporally-extended actions that cause the values of state variables to change. Each temporally-extended action corresponds to a subtask that is significantly easier to solve than the overall task. Results from experiments show great promise in scaling to larger tasks.
Cite
Text
Jonsson and Barto. "A Causal Approach to Hierarchical Decomposition of Factored MDPs." International Conference on Machine Learning, 2005. doi:10.1145/1102351.1102402Markdown
[Jonsson and Barto. "A Causal Approach to Hierarchical Decomposition of Factored MDPs." International Conference on Machine Learning, 2005.](https://mlanthology.org/icml/2005/jonsson2005icml-causal/) doi:10.1145/1102351.1102402BibTeX
@inproceedings{jonsson2005icml-causal,
title = {{A Causal Approach to Hierarchical Decomposition of Factored MDPs}},
author = {Jonsson, Anders and Barto, Andrew G.},
booktitle = {International Conference on Machine Learning},
year = {2005},
pages = {401-408},
doi = {10.1145/1102351.1102402},
url = {https://mlanthology.org/icml/2005/jonsson2005icml-causal/}
}