arXiv:2602.01797cs.AI2026-02被引 5

用确定性框架让多个AI模型协作推理,结果更可靠、可复现。

ORCH: many analyses, one merge-a deterministic multi-agent orchestrator for discrete-choice reasoning with EMA-guided routing

  • 多个模型独立分析问题,再由专门代理统一决策,流程固定可预测。
  • 在MMLU-Pro上比最强基线高10分以上,在GSM8K上提升超50分。
  • 适合需要可解释、可控的AI决策系统,如考试评测或严谨推理场景。

大型语言模型的进展使多智能体架构在复杂推理任务中备受关注。但现有系统多依赖随机路由或临时规则,行为难以复现且决策过程难解释。我们提出ORCH,一种用于离散选择推理的确定性协调框架,可调度异构语言模型。ORCH采用“多分析、一决策”范式:多个基础模型独立生成结构化分析,专用合并代理输出最终答案。该框架使用固定规则进行任务拆分与结果聚合,确保流程可预测、可复现且无需训练。确定性指在固定评估协议下路由与聚合规则不变,而非跨部署的比特级复现。为利用模型互补性,可选引入基于历史准确率、延迟或成本的EMA引导路由器;因其依赖答案反馈,主要适用于基准测试、受控评估或延迟反馈场景。在MMLU、MMLU-Pro和GSM8K上的实验表明,ORCH持续优于单模型基线和多数投票集成。在MMLU-Pro上,其准确率超过最强基线10分以上;在GSM8K上,提升超过50分;麦内马尔检验确认统计显著性。EMA路由器带来0.7至2.0分的额外增益,消融实验显示多智能体协作与路由机制均贡献显著。总体而言,ORCH为离散选择推理提供了可控制、可解释、可部署的实用路径。

原文摘要 · Abstract (English)

Recent advances in large-scale language models (LLMs) have made multi-agent architectures attractive for challenging reasoning tasks. However, many existing systems rely on stochastic routing or ad-hoc heuristics, making their behavior difficult to reproduce and their decision process hard to interpret. We propose ORCH, a deterministic coordination framework for discrete-choice reasoning that orchestrates heterogeneous LLMs. ORCH follows a ``many analyses, one decision'' paradigm: multiple base models independently produce structured analyses, and a dedicated merge agent outputs the final choice. The framework uses fixed rules for task decomposition and answer aggregation, keeping the pipeline predictable, reproducible, and training-free. Determinism here refers to fixed routing and aggregation rules under a fixed evaluation protocol, rather than strict bit-level reproducibility across deployments. To exploit model complementarity, we optionally introduce an EMA-guided router that updates agent selection using historical accuracy, latency, or cost; since it relies on answer-based feedback, it is mainly intended for benchmarking, controlled evaluation, or delayed-feedback settings. Experiments on MMLU, MMLU-Pro, and GSM8K show that ORCH consistently outperforms single-model baselines and a majority-vote ensemble. On MMLU-Pro, ORCH improves accuracy by over 10 points compared to the strongest baseline, and on GSM8K it yields gains exceeding 50 points; McNemar tests confirm statistical significance. The EMA router provides an additional 0.7--2.0 point accuracy boost, and ablations show that both multi-agent collaboration and routing contribute substantially. Overall, ORCH offers a practical path toward controllable, interpretable, and deployment-ready LLM-based agent systems for discrete-choice reasoning.

多智能体推理系统确定性可复现

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。