Experience Selection and Problem Choice in an Exploratory Learning System

Abstract

A fully autonomous exploratory learning system must perform two tasks that are not required of supervised learning systems: experience selection and problem choice. Experience selection is the process of choosing informative training examples from the space of all possible examples. Problem choice is the process of identifying defects in the domain theory and determining which should be remedied next. These processes are closely related because the degree to which a specific experience is informative depends on the particular defects in the domain theory that the system is attempting to remedy. In this article we propose a general control structure for exploratory learning in which problem choice by an information-theoretic “curiosity” heuristic: the problem chosen then guides the selection of training examples. An implementation of an exploratory learning system based on this control structure is described, and a series of experimental results are presented.