Cooperative Multi-Agent Reinforcement Learning: Asynchronous Communication and Linear Function Approximation
Abstract
We study multi-agent reinforcement learning in the setting of episodic Markov decision processes, where many agents cooperate via communication through a central server. We propose a provably efficient algorithm based on value iteration that can simultaneously allow asynchronous communication and guarantee the benefit of cooperation with low communication complexity. Under linear function approximation, we prove that our algorithm enjoys a $\tilde{\mathcal{O}}(d^{3/2}H^2\sqrt{K})$ regret upper bound with $\tilde{\mathcal{O}}(dHM^2)$ communication complexity, where $d$ is the feature dimension, $H$ is the horizon length, $M$ is the total number of agents, and $K$ is the total number of episodes. We also provide a lower bound showing that an $\Omega(dM)$ communication complexity is necessary to improve the performance through collaboration.
Cite
Text
Min et al. "Cooperative Multi-Agent Reinforcement Learning: Asynchronous Communication and Linear Function Approximation." International Conference on Machine Learning, 2023.Markdown
[Min et al. "Cooperative Multi-Agent Reinforcement Learning: Asynchronous Communication and Linear Function Approximation." International Conference on Machine Learning, 2023.](https://mlanthology.org/icml/2023/min2023icml-cooperative/)BibTeX
@inproceedings{min2023icml-cooperative,
title = {{Cooperative Multi-Agent Reinforcement Learning: Asynchronous Communication and Linear Function Approximation}},
author = {Min, Yifei and He, Jiafan and Wang, Tianhao and Gu, Quanquan},
booktitle = {International Conference on Machine Learning},
year = {2023},
pages = {24785-24811},
volume = {202},
url = {https://mlanthology.org/icml/2023/min2023icml-cooperative/}
}