A Role for Anticipation in Reactive Systems That Learn

Abstract

This chapter reviews the role of anticipation in reactive learning systems. Most work in intelligent robotics can be characterized as focusing either on rote behavior or thought behavior. During plan generation, the system reasons about the world to construct a plan that is followed during execution. Reactive systems focus on rote decisions and knowing what to do. Instead of relying on costly symbolic reasoning, reactive systems depend on precompiled knowledge about the way to behave in particular situations. Reactive systems rely on efficiently organized built in knowledge to obtain their performance. Plan-based controllers and reactive controllers represent endpoints on a spectrum, and for the most part, their respective architectures are quite dissimilar. More work is needed to explore the middle ground, and develop models capable of both reactive behavior and reasoning activity. Incremental learning systems in general, and systems using temporal difference methods in particular, are characterized by slow learning rates. Also, these systems typically learn by participating in their environments and receiving reinforcement commensurate with the appropriateness of their behavior.