用错误代码生成测试用例，提升大模型代码生成的鲁棒性。
Robust Code RL via Faulty-Code-Driven Test case Synthesis and Dense Reward Shaping
- 基于近似正确的错误代码生成测试用例，挖掘潜在逻辑缺陷。
- 引入分步密集奖励机制，使模型在测试通过率上表现更稳定。
- 适合想提升代码生成模型泛化能力的研究者和开发者。
基于可验证奖励的强化学习（RLVR）对提升大语言模型代码生成能力至关重要，但其效果常因测试用例覆盖不足而受限，导致奖励欺骗和策略退化。为此，我们提出RobustTests框架，采用故障代码驱动的测试用例合成策略。通过利用“近似正确”的错误代码，该方法捕捉隐含逻辑差异，并借助带有行为特征聚类的验证代理过滤无效或冗余测试用例。此外，引入基于通过率的分步密集奖励函数，缓解误判问题并增强训练鲁棒性。基于此流程，我们构建了改进版CodeContests+数据集，具备更强诊断能力。实验表明，使用RobustTests对Qwen3-32B进行强化学习微调，在LiveCodeBench上取得3%的绝对性能提升，验证了其在提升大模型代码生成能力方面的有效性。代码与数据可在https://huggingface.co/datasets/sid6/RobustTests获取。
原文摘要 · Abstract (English)
Reinforcement Learning from Verifiable Rewards (RLVR) is pivotal for enhancing LLM code generation, yet its efficacy is often hindered by insufficient test case coverage, leading to reward hacking and policy degradation. To address this, we propose RobustTests, a framework featuring a faulty-code-driven test case synthesis strategy. By leveraging "near-correct" faulty codes, RobustTests captures latent logical discrepancies and employs validator agents with behavioral feature clustering to filter invalid or redundant test cases. Additionally, a stepwise dense reward function based on pass rates is introduced to mitigate false negatives and enhance training robustness. Using this pipeline, we construct an augmented version of the CodeContests+ dataset with superior diagnostic utility. Experimental results show that RL fine-tuning of Qwen3-32B via RobustTests achieves a 3% absolute gain on LiveCodeBench, demonstrating its effectiveness in advancing LLM code generation proficiency. Codes and data are available at https://huggingface.co/datasets/sid6/RobustTests.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。