提出统一框架与评测基准,让多智能体系统更高效、可验证地协作推理。
MAS-Orchestra: Understanding and Improving Multi-Agent Reasoning Through Holistic Orchestration and Controlled Benchmarks
- 将多智能体编排建模为函数调用的强化学习问题,实现全局结构化推理。
- 在数学推理等任务上优于单智能体,效率提升超10倍。
- 提供五维可控评测基准,揭示多智能体优势的适用条件。
尽管多智能体系统(MAS)通过智能体协同有望提升整体智能,但当前自动设计方法表现不佳。主要源于两方面:一是方法复杂性——现有编排依赖顺序代码执行,限制了全局系统级推理且难以扩展;二是效果不确定性——缺乏对多智能体相比单智能体是否真正有益的判断依据。本文提出MAS-Orchestra,一种训练时框架,将多智能体编排建模为函数调用的强化学习问题,实现一次性生成完整系统。复杂的目标导向子智能体被抽象为可调用函数,支持全局系统结构推理,同时隐藏内部执行细节。为系统研究多智能体何时何地有效,我们构建了MASBENCH,一个从深度、跨度、广度、并行性与鲁棒性五个维度刻画任务的受控基准。分析表明,多智能体收益高度依赖任务结构、验证协议及编排器与子智能体的能力,而非普遍成立。基于此洞察,MAS-Orchestra在数学推理、多跳问答和搜索型问答等公开基准上持续提升性能,效率超过强基线10倍以上。MAS-Orchestra与MASBENCH共同推动多智能体智能的更好训练与理解。
原文摘要 · Abstract (English)
While multi-agent systems (MAS) promise elevated intelligence through coordination of agents, current approaches to automatic MAS design under-deliver. Such shortcomings stem from two key factors: (1) methodological complexity - agent orchestration is performed using sequential, code-level execution that limits global system-level holistic reasoning and scales poorly with agent complexity - and (2) efficacy uncertainty - MAS are deployed without understanding if there are tangible benefits compared to single-agent systems (SAS). We propose MASOrchestra, a training-time framework that formulates MAS orchestration as a function-calling reinforcement learning problem with holistic orchestration, generating an entire MAS at once. In MAS-Orchestra, complex, goal-oriented subagents are abstracted as callable functions, enabling global reasoning over system structure while hiding internal execution details. To rigorously study when and why MAS are beneficial, we introduce MASBENCH, a controlled benchmark that characterizes tasks along five axes: Depth, Horizon, Breadth, Parallel, and Robustness. Our analysis reveals that MAS gains depend critically on task structure, verification protocols, and the capabilities of both orchestrator and subagents, rather than holding universally. Guided by these insights, MAS-Orchestra achieves consistent improvements on public benchmarks including mathematical reasoning, multi-hop QA, and search-based QA, while achieving more than 10x efficiency over strong baselines. Together, MAS-Orchestra and MASBENCH enable better training and understanding of MAS in the pursuit of multi-agent intelligence.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。