arXiv:2508.13167cs.AIcs.CL2025-08被引 70

让大模型像多智能体一样协作解题,且全程端到端训练。

Chain-of-Agents: End-to-End Agent Foundation Models via Multi-Agent Distillation and Agentic RL

  • 用多智能体蒸馏+智能体强化学习,让单模型学会动态调用工具和角色
  • 在网页与代码任务上刷新多个基准线,性能领先现有方法
  • 开源全部代码、数据和模型,支持后续智能体研究

大型语言模型(LLM)与多智能体系统在深度研究、编程和数学推理等复杂任务中展现出强大能力。然而,现有系统依赖人工提示/工作流设计,计算效率低、能力有限,且难以实现数据驱动学习。本文提出链式智能体(Chain-of-Agents, CoA),一种新型的LLM推理范式,使单一模型能以端到端方式模拟多智能体协作,完成多轮、多工具、多角色的任务求解。通过多智能体蒸馏框架,将先进多智能体系统的决策轨迹转化为链式智能体数据,用于智能体监督微调;再结合可验证任务上的智能体强化学习进一步提升性能。由此生成的模型称为智能体基础模型(Agent Foundation Models, AFMs)。实证研究表明,AFM在网页代理和代码代理等多个基准测试中达到新最佳表现。我们开源了全部研究资源,包括模型权重、训练与评估代码及训练数据,为智能体模型与智能体强化学习研究提供坚实起点。

原文摘要 · Abstract (English)

Recent advances in large language models (LLMs) and multi-agent systems have demonstrated remarkable capabilities in complex problem-solving tasks such as deep research, vibe coding, and mathematical reasoning. However, most existing multi-agent systems are built upon manual prompt/workflow engineering with sophisticated agent frameworks, making them computationally inefficient, less capable, and can not benefit from data-centric learning. In this work, we introduce Chain-of-Agents (CoA), a novel paradigm of LLM reasoning that enables native end-to-end complex problem-solving in the same way as a multi-agent system (i.e., multi-turn problem solving with multiple tools and multiple agents) within one model. In chain-of-agents problem-solving, the model dynamically activates different tool agents and role-playing agents to simulate multi-agent collaboration in an end-to-end fashion. To elicit end-to-end chain-of-agents problem-solving abilities in LLMs, we introduce a multi-agent distillation framework to distill state-of-the-art multi-agent systems into chain-of-agents trajectories for agentic supervised fine-tuning. We then use agentic reinforcement learning on verifiable agentic tasks to further improve the models' capabilities on chain-of-agents problem solving. We call the resulting models Agent Foundation Models (AFMs). Our empirical studies demonstrate that AFM establishes new state-of-the-art performance across diverse benchmarks in both web agent and code agent settings. We make the entire research, including the model weights, code for training and evaluation, and the training data, fully open-sourced, which offers a solid starting point for future research on agent models and agentic RL.

多智能体端到端强化学习大模型

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。