On the Undecidability of Probabilistic Planning and Infinite-Horizon Partially Observable Markov Decision Problems

Abstract

We investigate the computability of problems in probabilistic planning and partially observable infinite-horizon Markov decision processes. The undecidability of the string-existence problem for probabilistic finite automata is adapted to show that the following problem of plan existence in probabilistic planning is undecidable: given a probabilistic planning problem, determine whether there exists a plan with success probability exceeding a desirable threshold. Analogous policy-existence problems for partially observable infinite-horizon Markov decision processes under discounted and undiscounted total reward models, average-reward models, and state-avoidance models are all shown to be undecidable. The results apply to corresponding approximation problems as well. 1 Introduction We show that problems in probabilistic planning (Kushmerick, Hanks, & Weld 1995; Boutilier, Dean, & Hanks 1999) and infinite-horizon partially observable Markov decision processes (POMDPs) (L...