Combining Learned Discrete and Continuous Action Models

Abstract

Action modeling is an important skill for agents that must perform tasks in novel domains. Previous work on action modeling has focused on learning STRIPS operators in discrete, relational domains. There has also been a separate vein of work in continuous function approximation for use in optimal control in robotics. Most real world domains are grounded in continuous dynamics but also exhibit emergent regularities at an abstract relational level of description. These two levels of regularity are often difficult to capture using a single action representation and learning method. In this paper we describe a system that combines discrete and continuous action modeling techniques in the Soar cognitive architecture. Our system accepts a continuous state representation from the environment and derives a relational state on top of it using spatial relations. The dynamics over each representation is learned separately using two simple instance-based algorithms. The predictions from the individual models are then combined in a way that takes advantage of the information captured by each representation. We empirically show that this combined model is more accurate and generalizable than each of the individual models in a spatial navigation domain.