用多智能体强化学习提升大模型协作能力，显著提高复杂任务准确率。
Stronger-MAS: Multi-Agent Reinforcement Learning for Collaborative LLMs
- 设计针对多角色、多轮对话的分组强化学习算法
- 在长序列规划任务中准确率从14%提升至99.5%
- 适合研究大模型协同推理与智能体系统构建的学者
多智能体系统（MAS）和强化学习（RL）被广泛用于增强大语言模型（LLMs）的代理能力。MAS通过角色分工提升任务表现，而RL则利用环境奖励学习更强策略，如GRPO风格优化。然而，将在线策略强化学习应用于MAS仍缺乏探索，且面临独特挑战：算法上，标准GRPO的分组假设因角色和回合差异而失效；系统层面，训练架构需支持多智能体流程回放与单策略/多策略模型的在线更新。本文提出AT-GRPO，包含（i）面向多智能体的智能体与回合级分组强化学习算法，（ii）支持单策略与多策略模式的训练系统。在游戏、规划、编码和数学任务中，AT-GRPO均取得显著提升。在长时序规划任务中，准确率从单智能体强化学习基线的14.0%提升至96.0%~99.5%。同时在编码任务中平均提升3.87%~7.62%，数学任务提升9.0%~17.93%。代码与环境已公开于https://github.com/pettingllms-ai/PettingLLMs。
原文摘要 · Abstract (English)
Multi-agent systems (MAS) and reinforcement learning (RL) are widely used to enhance the agentic capabilities of large language models (LLMs). MAS improves task performance through role-based orchestration, while RL uses environmental rewards to learn stronger policies, such as GRPO-style optimization. However, applying on-policy RL to MAS remains underexplored and presents unique challenges. Algorithmically, standard GRPO grouping assumptions break down because prompts vary by role and by turn. System-wise, the training stack must support MAS-workflow rollouts and on-policy updates for both single-policy and multi-policy models. We propose AT-GRPO, which includes (i) an agent- and turn-wise grouped RL algorithm tailored to MAS and (ii) a training system that supports both single- and multi-policy regimes. Across game, planning, coding, and math tasks, AT-GRPO delivers substantial gains. On long-horizon planning, it increases accuracy from a 14.0 to 47.0 percent single-agent RL baseline to 96.0 to 99.5 percent. It also improves reasoning performance, with average gains of 3.87 to 7.62 percent on coding tasks and 9.0 to 17.93 percent on math. Code and environments are available at: https://github.com/pettingllms-ai/PettingLLMs.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。