让多个大模型协作时能反复优化，提升推理准确率。
Iterative Critique-and-Routing Controller for Multi-Agent Systems with Heterogeneous LLMs
- 设计可迭代的评审与路由控制器，支持多轮修正。
- 在7个推理任务中优于现有方法，调用次数少于25%。
- 适合需要高精度推理的异构多模型系统使用。
多智能体大语言模型系统常依赖控制器协调异构模型，但现有控制器多为单次路由：仅选择一个模型并直接返回输出，缺乏对中间结果的评审或迭代优化机制。为此，本文提出一种评审-路由控制器，将多智能体协作建模为序列决策问题。每轮中，控制器评估当前草稿，决定是否停止或继续，并在必要时选择下一智能体进行进一步优化。该过程被形式化为带有显式模型使用约束的有限时域马尔可夫决策过程（MDP），设计复合奖励函数以优化跨轮次决策，并通过拉格朗日松弛下的策略梯度进行训练。在多个异构多智能体系统及七个推理基准上的实验表明，该方法持续优于现有先进基线，在调用总次数少于25%的情况下，显著缩小了与最强智能体之间的差距。
原文摘要 · Abstract (English)
Multi-agent large language model (LLM) systems often rely on a controller to coordinate a pool of heterogeneous models, yet existing controllers are typically limited to one-shot routing: they select a model once and return its output directly. Such routing-only designs provide no mechanism to critique intermediate drafts or support iterative refinement. To address this limitation, we propose a critique-and-routing controller that casts multi-agent coordination as a sequential decision problem. At each turn, the controller evaluates the current draft, decides whether to stop or continue, and, if needed, selects the next agent for further refinement. We formulate this process as a finite-horizon Markov Decision Process (MDP) with explicit agent-utilization constraints, design a composite reward for controller decisions across turns, and optimize the controller via policy gradients under a Lagrangian-relaxed objective. Extensive experiments across multiple heterogeneous multi-agent systems and seven reasoning benchmarks show that our method consistently outperforms state-of-the-art baselines and substantially narrows the gap to the strongest agent, while using it for fewer than 25% of total calls.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。