Active Exploration in Dynamic Environments

Abstract

\Vhenever an agent learns to control an unknown environment, two oppos(cid:173) ing principles have to be combined, namely: exploration (long-term opti(cid:173) mization) and exploitation (short-term optimization). Many real-valued connectionist approaches to learning control realize exploration by ran(cid:173) domness in action selection. This might be disadvantageous when costs are assigned to "negative experiences" . The basic idea presented in this paper is to make an agent explore unknown regions in a more directed manner. This is achieved by a so-called competence map, which is trained to predict the controller's accuracy, and is used for guiding exploration. Based on this, a bistable system enables smoothly switching attention between two behaviors - exploration and exploitation - depending on ex(cid:173) pected costs and knowledge gain. The appropriateness of this method is demonstrated by a simple robot navigation task.