Analyzing Multi-Agent Reinforcement Learning Using Evolutionary Dynamics
Abstract
In this paper, we show how the dynamics of Q-learning can be visualized and analyzed from a perspective of Evolutionary Dynamics (ED). More specifically, we show how ED can be used as a model for Q-learning in stochastic games. Analysis of the evolutionary stable strategies and attractors of the derived ED from the Reinforcement Learning (RL) application then predict the desired parameters for RL in Multi-Agent Systems (MASs) to achieve Nash equilibriums with high utility. Secondly, we show how the derived fine tuning of parameter settings from the ED can support application of the COllective INtelligence (COIN) framework. COIN is a proved engineering approach for learning of cooperative tasks in MASs. We show that the derived link between ED and RL predicts performance of the COIN framework and visualizes the incentives provided in COIN toward cooperative behavior.
Cite
Text
Hoen and Tuyls. "Analyzing Multi-Agent Reinforcement Learning Using Evolutionary Dynamics." European Conference on Machine Learning, 2004. doi:10.1007/978-3-540-30115-8_18Markdown
[Hoen and Tuyls. "Analyzing Multi-Agent Reinforcement Learning Using Evolutionary Dynamics." European Conference on Machine Learning, 2004.](https://mlanthology.org/ecmlpkdd/2004/hoen2004ecml-analyzing/) doi:10.1007/978-3-540-30115-8_18BibTeX
@inproceedings{hoen2004ecml-analyzing,
title = {{Analyzing Multi-Agent Reinforcement Learning Using Evolutionary Dynamics}},
author = {Hoen, Pieter Jan't and Tuyls, Karl},
booktitle = {European Conference on Machine Learning},
year = {2004},
pages = {168-179},
doi = {10.1007/978-3-540-30115-8_18},
url = {https://mlanthology.org/ecmlpkdd/2004/hoen2004ecml-analyzing/}
}