Selectively Sharing Experiences Improves Multi-Agent Reinforcement Learning
Abstract
We present a novel multi-agent RL approach, Selective Multi-Agent Prioritized Experience Relay, in which agents share with other agents a limited number of transitions they observe during training. The intuition behind this is that even a small number of relevant experiences from other agents could help each agent learn. Unlike many other multi-agent RL algorithms, this approach allows for largely decentralized training, requiring only a limited communication channel between agents. We show that our approach outperforms baseline no-sharing decentralized training and state-of-the art multi-agent RL algorithms. Further, sharing only a small number of highly relevant experiences outperforms sharing all experiences between agents, and the performance uplift from selective experience sharing is robust across a range of hyperparameters and DQN variants.
Cite
Text
Gerstgrasser et al. "Selectively Sharing Experiences Improves Multi-Agent Reinforcement Learning." Neural Information Processing Systems, 2023.Markdown
[Gerstgrasser et al. "Selectively Sharing Experiences Improves Multi-Agent Reinforcement Learning." Neural Information Processing Systems, 2023.](https://mlanthology.org/neurips/2023/gerstgrasser2023neurips-selectively/)BibTeX
@inproceedings{gerstgrasser2023neurips-selectively,
title = {{Selectively Sharing Experiences Improves Multi-Agent Reinforcement Learning}},
author = {Gerstgrasser, Matthias and Danino, Tom and Keren, Sarah},
booktitle = {Neural Information Processing Systems},
year = {2023},
url = {https://mlanthology.org/neurips/2023/gerstgrasser2023neurips-selectively/}
}