用真实代码合并请求训练模型,让其学会跨文件精准修改代码。
Pull Requests as a Training Signal for Repo-Level Code Editing
- 用真实 GitHub 合并请求生成可执行的代码修改指令
- 在 SWE-bench Lite 上提升 13.6%,Verified 上提升 12.3%
- 无需复杂推理框架,模型可直接内化代码编辑能力
仓库级代码编辑要求模型理解复杂依赖并精确执行跨多文件的修改。尽管 SWE-bench 的进展依赖于复杂的智能体架构,但其能力是否可通过高质量训练信号内化仍不明确。为此,我们提出 Clean Pull Request(Clean-PR),一种利用真实 GitHub 合并请求作为训练信号的中段训练范式。我们构建了可扩展的流水线,通过重建与验证将噪声合并请求差异转化为搜索/替换编辑块,形成目前最大公开的 200 万条合并请求语料库,覆盖 12 种编程语言。基于此训练信号,我们进行中段训练,并采用无智能体对齐的监督微调与错误驱动数据增强。在 SWE-bench 测试中,模型显著优于指令微调基线,在 SWE-bench Lite 上实现 13.6% 的绝对提升,在 SWE-bench Verified 上提升 12.3%。结果表明,仓库级代码理解与编辑能力可在简化、无智能体的协议下有效内化至模型权重中,无需依赖复杂的推理时架构。
原文摘要 · Abstract (English)
Repository-level code editing requires models to understand complex dependencies and execute precise multi-file modifications across a large codebase. While recent gains on SWE-bench rely heavily on complex agent scaffolding, it remains unclear how much of this capability can be internalised via high-quality training signals. To address this, we propose Clean Pull Request (Clean-PR), a mid-training paradigm that leverages real-world GitHub pull requests as a training signal for repository-level editing. We introduce a scalable pipeline that converts noisy pull request diffs into Search/Replace edit blocks through reconstruction and validation, resulting in the largest publicly available corpus of 2 million pull requests spanning 12 programming languages. Using this training signal, we perform a mid-training stage followed by an agentless-aligned supervised fine-tuning process with error-driven data augmentation. On SWE-bench, our model significantly outperforms the instruction-tuned baseline, achieving absolute improvements of 13.6% on SWE-bench Lite and 12.3% on SWE-bench Verified. These results demonstrate that repository-level code understanding and editing capabilities can be effectively internalised into model weights under a simplified, agentless protocol, without relying on heavy inference-time scaffolding.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。