Reinforcement Learning in Dynamic Environments Using Instantiated Information

Abstract

We study using reinforcement learning in dynamic environments. Such environments may contain many dynamic objects which makes optimal planning hard. One way of using information about all dynamic objects is to expand the state description, but this results in a high dimensional policy space. Our approach is to instantiate information about dynamic objects in the model of the environment and to replan using model-based reinforcement learning whenever this information changes. Furthermore, our approach is combined with an a-priori model of the changing parts of the environment, which enables the agent to optimally plan a course of action. Results on a navigation task with multiple dynamic hostile agents show that our system is able to learn good solutions minimizing the risk of hitting hostile agents.