arXiv:2604.00344cs.CLstat.AP2026-04被引 1

用强化学习动态选配大模型多智能体协作方式,提升任务准确率与效率。

Agent Q-Mix: Selecting the Right Action for LLM Multi-Agent Systems through Reinforcement Learning

  • 将通信拓扑选择转为合作式强化学习问题,各智能体自主决策连接方式。
  • 在7个基准上平均准确率最高,令牌消耗更低,对智能体失效更鲁棒。
  • 适合需要高效多智能体协作的复杂推理任务,如编程、数学求解。

大型语言模型在完成各类任务中表现出色,但解决复杂问题常需多个智能体协同,核心挑战在于如何有效选择并连接这些智能体。本文提出 extbf{Agent Q-Mix},一种将拓扑选择重构为协作式多智能体强化学习(MARL)问题的框架。该方法利用QMIX值分解实现去中心化通信决策,每个智能体从一组通信动作中选择,共同生成每轮的通信图。其核心结合了拓扑感知的GNN编码器、GRU记忆单元和每智能体的Q头,在集中训练、去中心化执行(CTDE)范式下运行。框架优化一个兼顾任务准确率与令牌成本的奖励函数。在编码、推理和数学领域的七个核心基准上,Agent Q-Mix 的平均准确率优于现有方法,同时展现出更高的令牌效率和对智能体失效的鲁棒性。特别地,在使用 Gemini-3.1-Flash-Lite 作为主干模型的人类最后考试(HLE)中,取得20.8%的准确率,超越 Microsoft Agent Framework(19.2%)和 LangGraph(19.2%),随后为 AutoGen 与 OpenClaw 旗下的 Lobster。结果表明,通过学习实现的去中心化拓扑优化,能有效推动多智能体推理的边界。

原文摘要 · Abstract (English)

Large Language Models (LLMs) have shown remarkable performance in completing various tasks. However, solving complex problems often requires the coordination of multiple agents, raising a fundamental question: how to effectively select and interconnect these agents. In this paper, we propose \textbf{Agent Q-Mix}, a reinforcement learning framework that reformulates topology selection as a cooperative Multi-Agent Reinforcement Learning (MARL) problem. Our method learns decentralized communication decisions using QMIX value factorization, where each agent selects from a set of communication actions that jointly induce a round-wise communication graph. At its core, Agent Q-Mix combines a topology-aware GNN encoder, GRU memory, and per-agent Q-heads under a Centralized Training with Decentralized Execution (CTDE) paradigm. The framework optimizes a reward function that balances task accuracy with token cost. Across seven core benchmarks in coding, reasoning, and mathematics, Agent Q-Mix achieves the highest average accuracy compared to existing methods while demonstrating superior token efficiency and robustness against agent failure. Notably, on the challenging Humanity's Last Exam (HLE) using Gemini-3.1-Flash-Lite as a backbone, Agent Q-Mix achieves 20.8\% accuracy, outperforming Microsoft Agent Framework (19.2\%) and LangGraph (19.2\%), followed by AutoGen and Lobster by OpenClaw. These results underscore the effectiveness of learned, decentralized topology optimization in pushing the boundaries of multi-agent reasoning.

多智能体强化学习大模型协作优化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。