Extending DTGOLOG with Options

Abstract

Introduction Recently Boutilier et al. (2000) proposed the language DTGOLOG which combines explicit agent programming with decision theory. The motivation is that a user often has some idea about how to go about solving a particular problem yet at the same time does not want to or cannot commit in advance to an exact course of action. Instead, certain choices are left to the agent running the program and determining an optimal action selection policy involves solving a Markov Decision Process (MDP) [Puterman, 1994] . In a sense, a DTGOLOG program can be thought of as a factored representation of an MDP. As an example, Boutilier et al. consider a mail-delivery scenario where the task of delivering mail to a particular person is hand-coded and fixed, while the agent chooses the order in which the various people are served according to some reward function. They note that this approach allows solving problems which are much larger than those solvable using the traditional dynamic-progra