弱模型靠重采样无法超越强模型，因验证器误判会限制上限。
The Limits of Inference Scaling Through Resampling
- 用重采样提升推理性能受限于验证器误报率
- 单次采样准确率与误报率强相关，重采样无法突破上限
- 多数情况下10次以内采样最优，过量反而有害
近期研究认为，通过重采样直至通过验证器（如单元测试）的推理放大，可使弱模型逼近强模型性能。但本文指出，当验证器存在非零误报概率时，该方法存在根本局限：重采样无法降低误报率，从而在任意计算开销下均存在性能上限。在HumanEval和MBPP数据集上，模型单次采样准确率与误报率呈强相关性，因此弱模型无法通过重采样达到强模型的单次准确率。实验显示，最优采样次数通常少于10次，因误报带来的负收益超过收益，导致推理放大曲线向下弯曲。此外，误报结果可能还存在编码风格不符等不良特性。
原文摘要 · Abstract (English)
Recent research has generated hope that inference scaling, such as resampling solutions until they pass verifiers like unit tests, could allow weaker models to match stronger ones. Beyond inference, this approach also enables training reasoning models, where data is curated using rejection sampling against a verifier. However, we show that this approach is fundamentally limited when verifiers are imperfect and have a non-zero probability of producing false positives. Resampling cannot decrease this probability, so it imposes an upper bound to the accuracy of resampling-based inference scaling, regardless of compute budget. Our analysis shows that there is a strong correlation between the model's single-sample accuracy and its false positive rate on HumanEval and MBPP, whose unit tests have limited coverage. Therefore, no amount of inference scaling of weaker models can enable them to match the single-sample accuracy of a sufficiently strong model. Empirical results show that optimal sampling attempts are often fewer than 10, as the negative utility of false positives outweighs benefits, bending inference scaling curves downward. Finally, false positives may have other undesirable qualities, like poor adherence to coding style conventions.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。