用自洽生成与修复机制自动构建代码修复训练数据，显著提升修复效果。
Anchored Self-Play for Code Repair
- 单模型通过强化学习交替生成漏洞和修复，形成自适应训练课程。
- 在真实代码来源上修复率提升7.0个百分点，相对提升24%。
- 引入参考数据锚定训练，有效避免生成不现实的复杂漏洞。
代码修复是语言模型的重要能力：给定含错程序及单元测试，模型需生成能通过测试的正确代码。由于真实修复数据有限，我们提出利用语言模型自动生成带错代码与修复任务。本文设计生成-修复自博弈框架，单一模型通过强化学习交替生成漏洞并修复，随修复能力提升，生成方逐渐产生更难的漏洞，形成自动课程。为验证该课程泛化性，我们构建了覆盖真实漏洞来源的基准 BugSourceBench，包含人工编写、模型生成及人工修改的模型生成代码中的漏洞。实验发现，标准自博弈趋向于难以且不现实的漏洞，虽提升对合成漏洞的修复表现，却损害对人类编写代码的修复能力。为此，我们提出锚定式自博弈（Anchored Self-Play, ASP），通过添加代码嵌入相似性奖励约束生成，并在修复器训练中混合少量参考漏洞数据。在多源漏洞上，ASP均取得最佳修复率，相较标准自博弈平均修复率提升7.0个百分点（相对提升24%），在模型与人类生成的漏洞上均有显著增益。
原文摘要 · Abstract (English)
Code repair is an important capability for language models (LMs): given a buggy program and unit tests, an LM must produce a fixed program that passes the tests. Because code repair data is limited, we aim to scale supervision by using an LM to generate bug--fix tasks. We propose __generator--fixer self-play__, in which a single model is trained with reinforcement learning to generate bugs and fix them. As the fixer improves, the generator adapts to produce more difficult bugs, yielding an automatic curriculum. To test whether this curriculum generalizes, we introduce BugSourceBench, a repair benchmark spanning realistic bug sources: bugs in human-written code, LM-generated code, and human-edited LM-generated code. On BugSourceBench, we find that self-play drifts toward difficult but unrealistic bugs, improving on synthetic bugs but degrading on human-authored ones. We propose Anchored Self-Play (ASP), which anchors self-play with a small reference set by adding a code-embedding similarity reward for generation and mixing reference bugs into fixer training. Across bug sources, ASP achieves the best fix rates, improving average fix rate over standard self-play by $+24\%$ relative / $+7.0$ pp absolute, with gains on bugs from both LMs and humans.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。