Inference and Learning in Dynamic Decision Networks Using Knowledge Compilation
Abstract
Decision making under uncertainty in dynamic environments is a fundamental AI problem in which agents need to determine which decisions (or actions) to make at each time step to maximise their expected utility. Dynamic decision networks (DDNs) are an extension of dynamic Bayesian networks with decisions and utilities. DDNs can be used to compactly represent Markov decision processes (MDPs). We propose a novel algorithm called mapl-cirup that leverages knowledge compilation techniques developed for (dynamic) Bayesian networks to perform inference and gradient-based learning in DDNs. Specifically, we knowledge-compile the Bellman update present in DDNs into dynamic decision circuits and evaluate them within an (algebraic) model counting framework. In contrast to other exact symbolic MDP approaches, we obtain differentiable circuits that enable gradient-based parameter learning.
Cite
Text
Venturato et al. "Inference and Learning in Dynamic Decision Networks Using Knowledge Compilation." AAAI Conference on Artificial Intelligence, 2024. doi:10.1609/AAAI.V38I18.30042Markdown
[Venturato et al. "Inference and Learning in Dynamic Decision Networks Using Knowledge Compilation." AAAI Conference on Artificial Intelligence, 2024.](https://mlanthology.org/aaai/2024/venturato2024aaai-inference/) doi:10.1609/AAAI.V38I18.30042BibTeX
@inproceedings{venturato2024aaai-inference,
title = {{Inference and Learning in Dynamic Decision Networks Using Knowledge Compilation}},
author = {Venturato, Gabriele and Derkinderen, Vincent and Dos Martires, Pedro Zuidberg and De Raedt, Luc},
booktitle = {AAAI Conference on Artificial Intelligence},
year = {2024},
pages = {20567-20576},
doi = {10.1609/AAAI.V38I18.30042},
url = {https://mlanthology.org/aaai/2024/venturato2024aaai-inference/}
}