让大模型通过协作强化学习高效分工，提升团队写作编码质量。
LLM Collaboration With Multi-Agent Reinforcement Learning
- 将大模型协作建模为合作式多智能体强化学习问题。
- 在写作与编程任务中实现高质量输出，响应效率显著提升。
- 适合研究多智能体协同、大模型训练的学者与开发者。
大量研究致力于多智能体系统（MAS）以建模和解决多个交互智能体的问题。然而，大多数大语言模型（LLM）是独立预训练的，并未针对协作进行专门优化。现有微调框架依赖个体奖励，需为每个智能体设计复杂的奖励函数以促进协作。为此，我们将大模型协作建模为合作式多智能体强化学习（MARL）问题，提出多智能体、多轮次算法——多智能体组相对策略优化（MAGRPO），结合当前大模型强化学习与多智能体强化学习技术。在大模型写作与编程协作任务上的实验表明，使用MAGRPO微调多智能体系统可使智能体通过有效协作生成高质量响应。该方法为其他多智能体强化学习方法应用于大模型开辟了道路，并揭示了相关挑战。代码已公开于 https://github.com/OpenMLRL/CoMLRL。
原文摘要 · Abstract (English)
A large amount of work has been done in Multi-Agent Systems (MAS) for modeling and solving problems with multiple interacting agents. However, most LLMs are pretrained independently and not specifically optimized for coordination. Existing LLM fine-tuning frameworks rely on individual rewards, which require complex reward designs for each agent to encourage collaboration. To address these challenges, we model LLM collaboration as a cooperative Multi-Agent Reinforcement Learning (MARL) problem. We develop a multi-agent, multi-turn algorithm, Multi-Agent Group Relative Policy Optimization (MAGRPO), to solve it, building on current RL approaches for LLMs as well as MARL techniques. Our experiments on LLM writing and coding collaboration demonstrate that fine-tuning MAS with MAGRPO enables agents to generate high-quality responses efficiently through effective cooperation. Our approach opens the door to using other MARL methods for LLMs and highlights the associated challenges. Our code is available at https://github.com/OpenMLRL/CoMLRL.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。