arXiv:2512.04695cs.LG2025-12被引 3

用轻量协调器让多个大模型分工协作,提升复杂任务表现。

TRINITY: An Evolved LLM Coordinator

  • 设计轻量协调器,动态分配思考、执行、验证角色给不同大模型。
  • 在多轮任务中表现超越单模型,LiveCodeBench达86.2%得分。
  • 适合需要多模型协同的高难度推理与编程场景。

融合多种基础模型前景广阔,但权重合并受限于架构不匹配和封闭API。Trinity通过一个轻量级协调器,实现多个大语言模型(LLMs)间的高效协作。该协调器由约0.6B参数的紧凑语言模型和约10K参数的轻量头组成,采用进化策略优化,实现高效自适应任务委派。在多轮交互中,协调器为每个回合选择一个LLM并分配思考者、执行者或验证者角色,将复杂技能学习从协调器本身剥离。实验表明,Trinity在编码、数学推理、常识知识等任务上持续优于单个模型及现有方法,并对分布外任务具有强泛化能力。在标准基准测试中达到顶尖性能,包括在LiveCodeBench上取得86.2%的分数。理论与实证分析揭示其优势源于:(1) 协调器隐状态提供丰富上下文表示;(2) 在高维与严格预算下,可分离协方差矩阵自适应进化策略优于强化学习、模仿学习与随机搜索,得益于潜在的块ε可分性。

原文摘要 · Abstract (English)

Combining diverse foundation models is promising, but weight-merging is limited by mismatched architectures and closed APIs. Trinity addresses this with a lightweight coordinator that orchestrates collaboration among large language models (LLMs). The coordinator, comprising a compact language model (approximately $0.6$B parameters) and a lightweight head (approximately $10$K parameters), is optimized with an evolutionary strategy for efficient and adaptive delegation. Trinity processes queries over multiple turns, where at each turn the coordinator assigns one of three roles (Thinker, Worker, or Verifier) to a selected LLM, effectively offloading complex skill acquisition from the coordinator itself. Experiments show that Trinity consistently outperforms individual models and existing methods across coding, math, reasoning, and domain knowledge tasks, and generalizes robustly to out-of-distribution tasks. On standard benchmarks, Trinity achieves state-of-the-art results, including a score of 86.2% on LiveCodeBench. Theoretical and empirical analyses identify two main factors behind this performance: (1) the coordinator's hidden-state representations provide rich contextualization of inputs, and (2) under high dimensionality and strict budget constraints, the separable Covariance Matrix Adaptation Evolution Strategy offers advantages over reinforcement learning, imitation learning, and random search by exploiting potential block-epsilon-separability.

大模型协同任务分配进化算法

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。