Learning and Enforcement: Stabilizing Environments to Facilitate Activity

Abstract

One of the basic assumptions of work in machine learning is that the environments in which our learning systems are situated are stable enough to make learning useful. While this assumption is warranted in most domains, much of what we tend to think of as a natural regularity in the world has often been artificially imposed in order to make both learning and planning more tractable. This imposition is usually the product of a long-term manipulation of the physical structure, goals, and operators of these domains in the direction of maximal utility. Often, however, it is the product of an agent imposing stability on a domain in an effort to increase the utility of his own planning and learning. This paper examines the idea of what it would mean for an agent to strategically impose order on a domain in an effort to increase the effectiveness of its own learning. In particular, it outlines an initial taxonomy of classes of stability and presents the strategies for increasing overall stability that are associated with each class. Finally, it outlines the basic learning and planning trade-offs that have to be made when stability is optimized.