用推理代替测试，让代码智能体高效可靠地自我监督。
Scalable Supervision for Software Agents via Patch Reasoning
- 通过组内对比验证多个代码修改，无需执行测试即可生成密集奖励
- 在SWE-bench上达72.2%准确率，接近商业模型表现
- 支持无执行环境的强化学习训练，适合大规模代码生成任务
尽管语言模型智能体已推动软件工程发展，但现有的基于测试的监督方式在真实问题上难以扩展。原因有二：(1) 现实中高覆盖率测试本就稀少；(2) 构建和运行测试沙箱成本高且脆弱。为突破这一瓶颈，我们提出R4P——一种基于推理的监督方法，可提供无沙箱依赖的奖励。R4P采用组内训练目标，使多个代码补丁能相互验证修改内容，从而在不执行测试、不依赖特定智能体轨迹的情况下获得密集奖励。R4P在SWE-bench上的补丁验证准确率达72.2%，与专有模型相当。为进一步展示其下游实用性，我们设计并训练了仅靠R4P实现纯强化学习的无执行沙箱Mini-SE。Mini-SE在Pass@1上达到26.2%，比原Qwen3-32B提升10.0%，通过R4P在测试时优化补丁选择，可进一步提升至32.8%。稳定增长曲线表明，即使存在不足，R4P仍能可靠支撑下游任务规模化应用。
原文摘要 · Abstract (English)
While language model agents have advanced software engineering, existing test-based supervision is limiting its scalability on real-world issues. The reason is twofold: (1) high-coverage tests are naturally rare in the wild, and (2) building and running test sandbox is heavy and fragile. To unlock supervision scaling, we propose R4P, a reasoning-based method that provides scaffold-agnostic rewards. R4P uses a group-wise training objective, enabling it to verify multiple patches against each other's modification and gain a dense reward for supervising agents without executing tests or relying on specific agent trajectories. R4P achieves 72.2% Acc. for verifying patches from SWE-bench, competitive with proprietary models. To show the downstream practical utility of R4P, we design and train an execution-free scaffold, Mini-SE, with pure RL via R4P. Mini-SE achieves 26.2% Pass@1, showing a 10.0% improvement over the original Qwen3-32B, and can be further improved to 32.8% with R4P for test-time scaling on patch selection. The stable scaling curves illustrate that though imperfect, R4P can still reliably support downstream tasks at scale.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。