用多个独立模型的共识选出最优推理链,效果优于传统方法。
LLMs as a Jury: Cross-Model Consensus Can Outperform Process Reward Models for LLM Reasoning
- 用多个独立训练模型的投票结果决定答案,无需额外标注数据。
- 在数学竞赛题上接近理想选择器表现,自评方法几乎无效。
- 可预测何时可信,适合解决跨领域推理问题。
从候选推理路径中选出正确答案是测试时扩展的核心机制,但现有方法各有代价:自一致性继承单个模型的错误,而训练好的奖励模型需要标注数据且在分布外表现差。本文研究一种免费的推理信号——跨模型共识,即多个独立训练的模型各自解题一次后对最终答案的一致性。将这一组合视为‘大模型陪审团’,其验证信号来自答案一致性的结构本身,无需模型评价彼此工作。在七个基准上,该方法选出正确答案的表现优于自一致性,远胜于模型自评;在竞赛数学题上,几乎消除了与理想选择器的差距,而自评方法几乎未缩小差距。其原理是错误去相关:独立模型的错误各不相同,错误答案分散,正确答案则积累共识。我们推导出一个无参数的闭式规律,仅需三个面板统计量即可预测共识准确率,平均绝对误差为0.03,并揭示了方法上限:共享错误底限,数学任务接近零,科学类任务则不可忽视。相比四种训练好的验证器(判别式、结果导向、生成式奖励模型),该自由陪审团在数学训练域内匹配最强,在域外则为最优选择。因此,跨模型共识是一种可提前刻画的验证机制:有规律说明何时可信,有底限标明何时失效。
原文摘要 · Abstract (English)
Selecting the correct answer from a pool of candidate reasoning chains is the engine of test-time scaling, yet the standard selectors each carry a cost: self-consistency inherits the errors of the single model it resamples, and trained reward models need labeled data and transfer poorly off-distribution. We study a third signal, free at inference time: cross-model consensus, the degree to which independently trained models, each solving the problem once, agree on a final answer. We treat the panel as an LLM-jury, in which the verification signal is the structure of agreement itself, with no model scoring another's work. Across seven benchmarks it selects correct answers better than self-consistency and far better than a model scoring its own candidates: on competition math it closes the entire gap to an oracle selector, while self-scoring closes almost none. The mechanism is error decorrelation: independently trained models err differently, so their wrong answers scatter while the correct one accumulates agreement. We make this precise with a parameter-free law, derived in closed form, that predicts consensus accuracy from three measured panel statistics to a mean absolute error of $0.03$ and exposes the method's ceiling: a shared-error floor where models share a misconception, near zero on math but non-trivial on science. Against four trained verifiers spanning discriminative, outcome, and generative reward models, the free LLM-jury matches the strongest inside their math training domain and is the top selector outside it. Cross-model consensus is thus a verifier we can characterize in advance: a law that says when to trust it, and a floor that marks where it cannot.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。