Cheap Talk Discovery and Utilization in Multi-Agent Reinforcement Learning
Abstract
By enabling agents to communicate, recent cooperative multi-agent reinforcement learning (MARL) methods have demonstrated better task performance and more coordinated behavior. Most existing approaches facilitate inter-agent communication by allowing agents to send messages to each other through free communication channels, i.e., \emph{cheap talk channels}. Current methods require these channels to be constantly accessible and known to the agents a priori. In this work, we lift these requirements such that the agents must discover the cheap talk channels and learn how to use them. Hence, the problem has two main parts: \emph{cheap talk discovery} (CTD) and \emph{cheap talk utilization} (CTU). We introduce a novel conceptual framework for both parts and develop a new algorithm based on mutual information maximization that outperforms existing algorithms in CTD/CTU settings. We also release a novel benchmark suite to stimulate future research in CTD/CTU.
Cite
Text
Lo et al. "Cheap Talk Discovery and Utilization in Multi-Agent Reinforcement Learning." International Conference on Learning Representations, 2023.Markdown
[Lo et al. "Cheap Talk Discovery and Utilization in Multi-Agent Reinforcement Learning." International Conference on Learning Representations, 2023.](https://mlanthology.org/iclr/2023/lo2023iclr-cheap/)BibTeX
@inproceedings{lo2023iclr-cheap,
title = {{Cheap Talk Discovery and Utilization in Multi-Agent Reinforcement Learning}},
author = {Lo, Yat Long and de Witt, Christian Schroeder and Sokota, Samuel and Foerster, Jakob Nicolaus and Whiteson, Shimon},
booktitle = {International Conference on Learning Representations},
year = {2023},
url = {https://mlanthology.org/iclr/2023/lo2023iclr-cheap/}
}