让多个大模型协作时,选互补的而非最强的,提升回答质量。
Mixture of Complementary Agents for Robust LLM Ensemble

- 基于互补性筛选大模型,而非仅看性能强弱
- 在少量标注数据下实现高效选择,兼顾效果与成本
- 适合需要稳定高质输出的多模型协作场景
多大模型协作(如集成或辩论)是整合信息、提升性能的有前景范式。其核心步骤是将多个生成模型的回应输入汇总模型,由其合成更优答案。然而,如何选择参与的生成模型仍具挑战。现有方法多聚焦于准确率(选最强模型)或多样性(保证差异),常忽略生成模型间及与汇总模型间的交互关系。本文将生成模型选择重新建模为类似特征选择的组合优化问题,强调模型间的互补性。由于直接应用标准特征选择算法在大模型场景中计算开销过大,我们探索了一系列计算可行的贪心式选择算法,利用少量标注数据评估互补性。实验验证了互补性作为选择准则的有效性,并识别出实际表现最优、性价比最高的方法。
原文摘要 · Abstract (English)
Multi-AI collaboration, such as ensembling or debating large language models (LLMs), is a promising paradigm for aggregating information and boosting performance. A foundational step in these pipelines is to feed the responses of several proposer LLMs into a summarizer LLM, which synthesizes a better answer. However, choosing which proposers to include is non-trivial. Existing approaches primarily focus either on accuracy (picking the strongest models) or diversity (ensuring variety), and often overlook the interactions among proposers and with the summarizer. We reframe proposer selection as a combinatorial selection problem akin to feature selection, where the value of an LLM lies in its complementarity with others. However, directly applying standard feature-selection algorithms is impractical in the LLM setting due to prohibitive time complexity. Motivated by this limitation, we explore an extensive range of computationally feasible, greedy-style selection algorithms that assess complementarity using a small labeled set. Our experiments validate complementarity as a guiding principle for proposer selection and identify methods that achieve the best performance-cost trade-offs in practice.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。