让大模型团队像蜂群一样自适应协作，动态调整成员分工。
Differentiable Mixture-of-Agents Incentivizes Swarm Intelligence of Large Language Models
- 用可微分路由机制动态决定每步该调用哪些模型角色
- 在9个基准测试中达到顶尖性能，且推理效率高、抗干扰强
- 适合需要灵活应变的复杂推理任务，如多步规划与开放问答
大语言模型的发展推动了多智能体系统在复杂推理任务中的应用。然而，现有系统通常依赖预定义或预先编译的通信拓扑，限制了其灵活性和对动态任务需求的适应能力。本文提出可微分混合智能体（DMoA），一种在推理过程中自我演化的多智能体框架，实现弹性且自适应的智能体协作。与静态构建工作流不同，DMoA在每个推理步骤动态路由并激活智能体，隐式模拟多种通信拓扑结构，并适应不断变化的需求。为此，我们设计了一种可微分的、上下文感知的路由机制，利用循环结构融合历史与上下文信息，以逐步稀疏方式生成智能体激活。此外，引入预测熵作为自监督信号优化路由过程，实现无需外部标注的高效测试时适应。在9个基准测试上的大量实验表明，DMoA不仅达到当前最优性能，还展现出强大的效率、鲁棒性和集成能力。
原文摘要 · Abstract (English)
Recent advances in Large Language Models (LLMs) have catalyzed the development of multi-agent systems (MAS) for complex reasoning tasks. However, existing MAS typically rely on pre-defined or pre-compiled communication topologies, which limits their flexibility and adaptability to dynamic task requirements. In this work, we propose Differentiable Mixture-of-Agents (DMoA), a self-evolving multi-agent framework that enables elastic and adaptive agent collaboration during inference. Instead of statically constructing workflows, DMoA dynamically routes and activates agents at each reasoning step, allowing the system to implicitly simulate diverse communication topologies and adapt to evolving demands. To achieve this, we design a differentiable, context-aware routing mechanism that leverages recurrent structures to incorporate historical and contextual information, producing sparse agent activations in a step-wise manner. Furthermore, we introduce predictive entropy as self-supervised signals to optimize the routing process, enabling efficient test-time adaptation without external annotations. Extensive experiments across 9 benchmarks demonstrate that DMoA achieves state-of-the-art performance while exhibiting strong efficiency, robustness, and ensembling capabilities.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。