arXiv:2510.00931cs.CL2025-10被引 2

不选最优,而是融合多个生成结果,提升大模型输出质量

Making, not Taking, the Best of N

  • 用通用大模型裁判融合N个生成样本的精华
  • 在11种语言、3类任务中均优于传统选优方法
  • 适合需要高质量生成的场景,如数据合成与推理优化

现代大模型生成高质量内容常被视作选择问题:从N个候选中选出最优解(最佳选择N,BoN)。然而该方法本质为零和博弈,丢弃了其余样本中的潜在信息。本文提出协作式融合方法FusioN,利用通用大模型裁判将多个样本的有用信息整合成最终答案。我们在两种场景下对比FusioN与BoN:(i)测试时扩展,从单个模型生成并聚合多样本;(ii)合成数据生成,融合多个教师模型的输出以改进学生模型。在11种语言、3类任务及不同模型规模下进行广泛评测,结果表明FusioN持续优于BoN,展现出跨任务、跨语言的稳健性。深入分析显示其在挑战性设置下仍具强大鲁棒性。这表明应从单一质量评估转向拥抱生成结果的多元特性,通过融合实现以往仅靠选择无法达到的性能提升。

原文摘要 · Abstract (English)

Obtaining high-quality generations in modern LLMs has largely been framed as a selection problem: identifying a single winning generation from a diverse pool of N samples, the Best-of-N (BoN). Yet, this approach is inherently zero-sum, discarding diverse and potentially useful information from the pool. Instead, we explore a collaborative setup, where all candidates can potentially contribute to the final winning generation. To this end, we propose Fusion-of-N (FusioN): a method that uses a general LLM judge to synthesize the most informative elements of each sample into a single final answer. We compare FusioN to BoN in two settings, (i) test-time scaling, where we sample and aggregate from a single model at test-time (ii) synthetic data generation, where we fuse samples from a pool of diverse teachers to improve a student model. We extensively benchmark both setups across 11 languages, 3 diverse tasks and varying model scales. Across the bench, FusioN consistently outperforms BoN showing versatility and robustness both in test-time scaling and in downstream gains from synthetic data generation. We also perform extensive analysis on FusioN, where it shows surprising strengths and robustness under challenging settings. These results show that we should shift how we think about evaluating and utilizing LLM generations from a monolithic measure of quality, to embracing their polylithic nature. This shift allows us to integrate diverse strengths, unlock latent potential, and achieve improvements that were previously inaccessible through selection alone.

大模型生成融合策略测试时扩展

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。