Multiagent Reinforcement Learning: Theoretical Framework and an Algorithm

Abstract

In this paper, we adopt general-sum stochastic games as a framework for multiagent reinforcement learning. Our work extends previous work by Littman on zero-sum stochastic games to a broader framework. We design a multiagent Q-learning method under this framework, and prove that it converges to a Nash equilibrium under specified conditions. This algorithm is useful for finding the optimal strategy when there exists a unique Nash equilibrium in the game. When there exist multiple Nash equilibria in the game, this algorithm should be combined with other learning techniques to find optimal strategies. 1 Introduction Reinforcement learning has gained attention and extensive study in recent years [5, 12]. As a learning method that does not need a model of its environment and can be used online, reinforcement learning is wellsuited for multiagent systems, where agents know little about other agents, and the environment changes during learning. Applications of reinforcement learning in multi...