用运行结果共识选出最优代码，提升大模型生成质量。
Majority Voting for Code Generation
- 通过代码运行结果一致性筛选最佳解
- 在LiveCodeBench上显著提升准确率
- 适合无标签训练场景的推理优化
我们研究了基于功能共识的函数多数投票（FMV）方法，该方法利用多个生成代码在测试输入上的运行时执行签名，从中识别出代表性解决方案。实验表明，FMV是一种高效的测试时推理策略，在不增加大量计算开销的情况下，显著提升了LiveCodeBench上的性能。此外，我们将功能共识扩展应用于无标签测试时强化学习的聚合策略，证明其可提高保留任务上的pass@1指标，但未发现其能突破基础模型的性能上限。
原文摘要 · Abstract (English)
We investigate Functional Majority Voting (FMV), a method based on functional consensus for code generation with Large Language Models, which identifies a representative solution from multiple generations using their runtime execution signatures on test inputs. We find that FMV is an effective test-time inference strategy, substantially boosting performance on LiveCodeBench without a large compute overhead. Furthermore, we extend the utility of functional consensus and apply it as an aggregation strategy for label-free Test-Time Reinforcement Learning. We demonstrate that this increases pass@1 on holdout tasks, but find no evidence of self-improvement beyond the base model's performance ceiling.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。