Envelope-Based Planning in Relational MDPs

Abstract

A mobile robot acting in the world is faced with a large amount of sen- sory data and uncertainty in its action outcomes. Indeed, almost all in- teresting sequential decision-making domains involve large state spaces and large, stochastic action sets. We investigate a way to act intelli- gently as quickly as possible in domains where finding a complete policy would take a hopelessly long time. This approach, Relational Envelope- based Planning (REBP) tackles large, noisy problems along two axes. First, describing a domain as a relational MDP (instead of as an atomic or propositionally-factored MDP) allows problem structure and dynam- ics to be captured compactly with a small set of probabilistic, relational rules. Second, an envelope-based approach to planning lets an agent be- gin acting quickly within a restricted part of the full state space and to judiciously expand its envelope as resources permit.