Learning to Predict in Uncertain Continuous Tasks

Abstract

This paper addresses the automated learning of action effects by an autonomous agent in the physical world. The learning agent's physical sensors and effectors provide ordered feature values for the description of actions. Imperfections in the agent's sensor-effector system and characteristics of physical actions combine to generate noise and non-determinism in observed results of commanded actions. Successful learning of action effects requires a noise-tolerant learning algorithm. Successful goal-seeking ability further requires that the agent reason about its uncertainty in predicting the effects of its actions. Actions are represented by continuous operators called funnels. Funnels are computed by an empirical learning algorithm that is both noise-tolerant and biased away from making false positive prediction errors. This paper presents a learning algorithm for computing funnels and demonstrates the method's effectiveness with empirical tests of a robot on a physical manipulation task called tray-tilting.