用多个模型轮流生成，自动选最优结果，无需训练就能提升推理准确率。
RoBoN: Routed Online Best-of-n for Test-Time Scaling with Multiple LLMs
- 通过奖励模型和响应一致性信号，动态分配不同模型生成任务。
- 在多个数学推理数据集上，最高比单模型 best-of-n 提升3.4%准确率。
- 适合需要高效利用多模型能力的部署场景，无需额外训练。
Best-of-$n$ 是主流的 LLM 推理测试时扩展方法。然而，尽管不同 LLM 在任务间表现出互补优势，传统 best-of-$n$ 仅依赖单一模型生成回复。本文提出 RoBoN（Routed Online Best-of-$n$），一种基于多个模型的序列化多模型替代方案。给定一组模型 $\\[m_i\\_\{i=1\}^M$，RoBoN 根据奖励模型评分和预测结果的一致性信号，逐个将生成任务路由至不同模型。该在线路由无需额外训练，保持计算量一致，且可适配任意插件式奖励模型。在 MATH500、OlympiadBench、MinervaMath、GSM8K、MMLU 等推理基准上，当 $n$ 较大时，RoBoN 持续优于对每个模型单独应用 standard best-of-$n$，最高绝对准确率提升达 3.4%，并优于均匀多模型组合基线。结果表明，推理时可利用模型间的多样性，显著提升 best-of-$n$ 性能，为多模型测试时扩展提供一种简单、免训练的新路径。
原文摘要 · Abstract (English)
Best-of-$n$ is a widely used test-time scaling approach for LLM inference. Yet despite evidence that LLMs exhibit complementary strengths across tasks, traditionally best-of-$n$ relies on a single model to generate responses. We propose RoBoN (Routed Online Best-of-$n$), a sequential multi-LLM alternative to the prevailing single-model best-of-$n$. Given a suite of models $\{m_i\}_{i=1}^M$, RoBoN sequentially routes generations one-by-one across models, based on scores computed using a reward model and an agreement signal on the predicted responses. This online routing requires no additional training, keeps compute parity, and works with any plug-in reward model. Across reasoning benchmarks (MATH500, OlympiadBench, MinervaMath, GSM8K, MMLU), RoBoN consistently outperforms standard best-of-$n$ applied to each individual model for larger $n$, with gains of up to 3.4\% in absolute accuracy, and also improves over a uniform multi-model portfolio baseline. Our results indicate that diversity across models can be exploited at inference to improve best-of-$n$ performance over any constituent model alone, providing a simple, training-free path to test-time scaling with multiple LLMs.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。