On Becoming Decreasingly Reactive: Learning to Deliberate Minimally

Abstract

This paper describes the Minimal Deliberation (MD) approach, an integrated approach to learning and planning. In this approach, an agent acts reactively by default. However, in cases where shortsighted behavior causes failures, the system analyzes the failures to determine the goals for which deliberation is needed. The system then constructs a plan for these goals, leaving the other goals to be achieved by the reactive component. This process uses explanation–based learning, and thus from a single example, the system learns to avoid a general class of failures caused by the reactive component. The learning process continues until either a successful plan is generated or the system is unable to construct a classical plan for the current deliberative set of goals (i.e. the problem is unsolvable). By introducing deliberation minimally, the MD approach maintains some of the strengths of reactivity, including flexibility and reduced reasoning costs. At the same time, by allowing the system to fall back upon deliberative planning and use explanation–based learning, the system is able to avoid repeating errors caused by short–sighted reactivity.