Incremental Learning of Control Strategies with Genetic Algorithms

Abstract

This chapter reviews an approach to using genetic algorithms and other competition-based heuristics to learn reactive control rules given a simulation model of the environment implemented in a system called SAMUEL. SAMUEL learns rules expressed in a high-level rule language. The use of a symbolic rule language is intended to facilitate the incorporation of more traditional learning methods into the system where appropriate. SAMUEL consists of three major components: (1) a problem-specific module consisting of a World Model and its interfaces, (2) a performance module, and (3) a learning module. The Performance Module consists of CPS, a competition based production system that interacts with the World Model through the Sensor, Control, and Critic interfaces. CPS performs Matching, Conflict Resolution and Credit Assignment. The Learning Module uses a genetic algorithm to develop high performance strategies or reactive plans, expressed as a set of condition-action rules. Each strategy is evaluated by testing its performance in controlling the World Model through CPS. Genetic operators, such as crossover and mutation, produce plausible new strategies from high-performance precursors.