用强化学习训练真实代码修复模型，提升稳定性和效果。
Agentic Reinforcement Learning for Real-World Code Repair
- 构建可验证的修复流程，通过固定依赖和禁用自动升级提高可复现性。
- 在近1000个真实问题上，强化学习比监督微调提升7-20%修复成功率。
- 模型需匹配训练测试环境，否则无法跨环境泛化，适合实操型开发团队。
我们解决在真实代码库中训练可靠代码修复智能体的挑战，因复杂构建和动态依赖导致评估不稳定。为此，我们设计了一个可验证的流水线，以修复后构建成功作为成功标准，并通过锁定依赖项和关闭自动升级，在约1000个真实问题上提升了可复现性。在此基础上，我们引入一种可扩展的简化流水线，用于大规模强化学习（RL）。利用该设置，我们在完整流水线上对Qwen3-32B进行监督微调，并在简化环境中对SFT模型应用强化学习。从GPT-4.1轨迹蒸馏出的SFT模型体积仅为原模型的1/56，性能相当，而强化学习在相同训练-测试条件下带来7-20%的绝对提升。实验显示，“思考模式”表现持平或更差。无论是SFT还是强化学习模型，均未能跨环境泛化，凸显了训练与测试环境一致对构建可靠真实代码修复智能体的重要性。
原文摘要 · Abstract (English)
We tackle the challenge of training reliable code-fixing agents in real repositories, where complex builds and shifting dependencies make evaluation unstable. We developed a verifiable pipeline with success defined as post-fix build validation and improved reproducibility across ~1K real issues by pinning dependencies and disabling automatic upgrades. Building on this, we introduced a scalable simplified pipeline for large-scale reinforcement learning (RL). Using this setup, we supervised fine-tuned Qwen3-32B in the full pipeline and applied RL on top of the SFT model in the simplified environment. The SFT model distilled from GPT-4.1 trajectories performs on par while being 56x smaller, and RL added 7-20% absolute gains under matched train-test conditions. "Thinking mode" was on par or worse in our experiments. Both SFT and RL models failed to generalize across environments, highlighting the importance of matching train-test environments for building reliable real-world code-fixing agents.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。