arXiv:2512.23340cs.LGcs.AI2025-12被引 3

提出多模型协作的缩放定律,揭示模型组合能突破单模型性能极限。

The Law of Multi-Model Collaboration: Scaling Limits of Model Ensembling for Large Language Models

  • 基于参数总量建立多模型集成的通用缩放定律
  • 多模型系统遵循幂律增长,理论误差下限更低
  • 异构模型组合比同构更优,多样性是关键优势

大型语言模型(LLMs)的进展主要依赖于单模型的缩放定律,即参数量和数据量增加可提升性能。然而单个模型能力存在固有上限。通过多个模型间的复杂交互,其集体表现可超越任一单一模型。尽管模型路由和后处理集成等多模型整合技术迅速发展,但缺乏统一的性能缩放理论。本文提出多模型协作定律,基于总参数预算预测LLM集成的性能极限。采用方法无关的公式化方法,假设理想集成预言机,每个样本的总交叉熵损失由模型池中任意模型的最小损失决定。实验表明,多模型系统随总参数量呈幂律缩放,改进趋势更显著,理论损失下限更低,优于单模型缩放。异构模型家族的集成性能优于同质模型,表明模型多样性是协作增益的主要驱动力。这些发现表明,模型协作是拓展大模型智能边界的关键路径。

原文摘要 · Abstract (English)

Recent advances in large language models (LLMs) have been largely driven by scaling laws for individual models, which predict performance improvements as model parameters and data volume increase. However, the capabilities of any single LLM are inherently bounded. One solution originates from intricate interactions among multiple LLMs, rendering their collective performance surpasses that of any constituent model. Despite the rapid proliferation of multi-model integration techniques such as model routing and post-hoc ensembling, a unifying theoretical framework of performance scaling for multi-model collaboration remains absent. In this work, we propose the Law of Multi-model Collaboration, a scaling law that predicts the performance limits of LLM ensembles based on their aggregated parameter budget. To quantify the intrinsic upper bound of multi-model collaboration, we adopt a method-agnostic formulation and assume an idealized integration oracle where the total cross-entropy loss of each sample is determined by the minimum loss of any model in the model pool. Experimental results reveal that multi-model systems follow a power-law scaling with respect to the total parameter count, exhibiting a more significant improvement trend and a lower theoretical loss floor compared to single model scaling. Moreover, ensembles of heterogeneous model families achieve better performance scaling than those formed within a single model family, indicating that model diversity is a primary driver of collaboration gains. These findings suggest that model collaboration represents a critical axis for extending the intelligence frontier of LLMs.

大模型多模型协作缩放定律集成学习

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。