MARTI: A Framework for Multi-Agent LLM Systems Reinforced Training and Inference
Abstract
We present MARTI (Multi-Agent Reinforced Training and Inference), an open-source framework designed to facilitate scalable and efficient learning of multi-agent LLM systems. MARTI supports centralized multi-agent interactions and distributed policy training, with the added capability of multi-turn asynchronous rollouts to enhance training efficiency. The framework includes dynamic workflows for multi-agent interactions, which integrate both rule-based verifiable rewards and LLM-based generative rewards. We validate the effectiveness of MARTI through comprehensive experiments on diverse mathematical tasks, demonstrating that multi-agent LLM-based systems outperform single-agent systems within the same inference budget after convergence. Our contributions lay the foundation for exploring scalable collaborations within LLM-based multi-agent systems and advancing the capabilities of large reasoning models.
Cite
Text
Zhang et al. "MARTI: A Framework for Multi-Agent LLM Systems Reinforced Training and Inference." International Conference on Learning Representations, 2026.Markdown
[Zhang et al. "MARTI: A Framework for Multi-Agent LLM Systems Reinforced Training and Inference." International Conference on Learning Representations, 2026.](https://mlanthology.org/iclr/2026/zhang2026iclr-marti/)BibTeX
@inproceedings{zhang2026iclr-marti,
title = {{MARTI: A Framework for Multi-Agent LLM Systems Reinforced Training and Inference}},
author = {Zhang, Kaiyan and Tian, Kai and Liu, Runze and Zeng, Sihang and Zhu, Xuekai and Jia, Guoli and Fan, Yuchen and Lv, Xingtai and Zuo, Yuxin and Jiang, Che and Wang, Yuru and Wang, Jianyu and Hua, Ermo and Long, Xinwei and Gao, Junqi and Sun, Youbang and Ma, Zhiyuan and Cui, Ganqu and Ding, Ning and Qi, Biqing and Zhou, Bowen},
booktitle = {International Conference on Learning Representations},
year = {2026},
url = {https://mlanthology.org/iclr/2026/zhang2026iclr-marti/}
}