Discover Relevant Environment Feature Using Concurrent Reinforcement Learning

Abstract

In order to compare the policies more efficiently, we introduce a new reinforcement learning method called concurrent biased learning. This is a multi-thread learning method, in which each learning thread refers to one feature of the environment. If an agent intentionally focuses on part of these environmental features to learn a policy of a task, we call this method a biased learning; otherwise, if an agent uses all features that it perceives to learn a task, we call this unbiased learning. We present a method concerning the relevance of information in order to improve the learning of a reinforcement learning robot. We introduce a new concurrent online learning algorithm to calculate the contribution C(s) and relevance degree I(s) to quantify the relevancy of features with respect to a desired learning task. Our analysis shows that the correlation relationship of the environment features can be extracted and projected to concurrent learning threads. By comparing the contribution of these learning threads, we can evaluate the relevance degree of a feature when performing a particular learning task. ( ) i s π If the agent learns a policy with respect to one of the environmental features , we call this biased learning with respect to feature i . The biased Q-value can be denoted as . Then the Bellman update function of the biased Q-value is: i