Analyzing Intentional Behavior in Autonomous Agents Under Uncertainty
Abstract
Principled accountability for autonomous decision-making in uncertain environments requires distinguishing intentional outcomes from negligent designs from actual accidents. We propose analyzing the behavior of autonomous agents through a quantitative measure of the evidence of intentional behavior. We model an uncertain environment as a Markov Decision Process (MDP). For a given scenario, we rely on probabilistic model checking to compute the ability of the agent to influence reaching a certain event. We call this the scope of agency. We say that there is evidence of intentional behavior if the scope of agency is high and the decisions of the agent are close to being optimal for reaching the event. Our method applies counterfactual reasoning to automatically generate relevant scenarios that can be analyzed to increase the confidence of our assessment. In a case study, we show how our method can distinguish between 'intentional' and 'accidental' traffic collisions.
Cite
Text
Córdoba et al. "Analyzing Intentional Behavior in Autonomous Agents Under Uncertainty." International Joint Conference on Artificial Intelligence, 2023. doi:10.24963/IJCAI.2023/42Markdown
[Córdoba et al. "Analyzing Intentional Behavior in Autonomous Agents Under Uncertainty." International Joint Conference on Artificial Intelligence, 2023.](https://mlanthology.org/ijcai/2023/cordoba2023ijcai-analyzing/) doi:10.24963/IJCAI.2023/42BibTeX
@inproceedings{cordoba2023ijcai-analyzing,
title = {{Analyzing Intentional Behavior in Autonomous Agents Under Uncertainty}},
author = {Córdoba, Filip Cano and Judson, Samuel and Antonopoulos, Timos and Bjørner, Katrine and Shoemaker, Nicholas and Shapiro, Scott J. and Piskac, Ruzica and Könighofer, Bettina},
booktitle = {International Joint Conference on Artificial Intelligence},
year = {2023},
pages = {372-381},
doi = {10.24963/IJCAI.2023/42},
url = {https://mlanthology.org/ijcai/2023/cordoba2023ijcai-analyzing/}
}