合成验证限制代码大模型训练质量,需调整验证策略提升性能。
Verification Limits Code LLM Training
- 用复杂多样的测试用例提升代码生成能力,平均提升3分pass@1。
- 放宽通过标准或引入语言模型软验证,可提升2-4分pass@1。
- 保持正确解的多样性有助于模型泛化,适合优化代码生成模型。
代码生成的大语言模型越来越多依赖合成数据,其中问题解和验证测试均由模型生成。这种做法虽可规模化生成数据,却带来新瓶颈:验证天花板,即训练数据的质量与多样性受限于合成验证器的能力。本文系统研究验证设计与策略对模型性能的影响。我们发现:(i) 测试复杂度与数量影响生成能力,更丰富的测试集平均提升3分pass@1,仅增加数量则收益递减;(ii) 验证方式上,严格100%通过标准过于严苛,采用宽松阈值或基于LLM的软验证可恢复有价值数据,带来2-4分的pass@1提升,但效果依赖测试用例强度与多样性;(iii) 验证必要性通过对比形式正确与错误解及人工评估确认:保留每题多样正确的解能持续提升泛化能力。结果表明,当前验证过于僵化,过滤了宝贵多样性,但不能取消,只能校准。结合校准验证与多样、挑战性的题目-解对,可突破验证天花板,推动更强代码生成模型发展。
原文摘要 · Abstract (English)
Large language models for code generation increasingly rely on synthetic data, where both problem solutions and verification tests are generated by models. While this enables scalable data creation, it introduces a previously unexplored bottleneck: the verification ceiling, in which the quality and diversity of training data are fundamentally constrained by the capabilities of synthetic verifiers. In this work, we systematically study how verification design and strategies influence model performance. We investigate (i) what we verify by analyzing the impact of test complexity and quantity: richer test suites improve code generation capabilities (on average +3 pass@1), while quantity alone yields diminishing returns, (ii) how we verify by exploring relaxed pass thresholds: rigid 100% pass criteria can be overly restrictive. By allowing for relaxed thresholds or incorporating LLM-based soft verification, we can recover valuable training data, leading to a 2-4 point improvement in pass@1 performance. However, this benefit is contingent upon the strength and diversity of the test cases used, and (iii) why verification remains necessary through controlled comparisons of formally correct versus incorrect solutions and human evaluation: retaining diverse correct solutions per problem yields consistent generalization gains. Our results show that Verification as currently practiced is too rigid, filtering out valuable diversity. But it cannot be discarded, only recalibrated. By combining calibrated verification with diverse, challenging problem-solution pairs, we outline a path to break the verification ceiling and unlock stronger code generation models.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。