arXiv:2605.21996cs.SEcs.AI2026-05被引 1

用修复补丁指导训练,让编程代理更高效精准地解决问题。

From Patches to Trajectories: Privileged Process Supervision for Software-Engineering Agents

论文配图:From Patches to Trajectories: Privileged Process Supervision for Software-Engineering Agents
图 1 · 摘自论文原文
  • 利用正确修复补丁构建上下文图谱,引导轨迹筛选
  • 在1.8k数据上提升通过率10.8点,推理成本降15%
  • 适合追求高精度与低耗时的软件工程自动化研究者

在长教师轨迹上进行监督微调是赋予开源软件工程代理探究与推理能力的主要方式。然而,每一步保留的响应都会成为模仿目标,导致学生继承最终结果及中间缺陷,如无根据的跳跃和冗余循环。高质量训练数据需兼具有效性(每步都有据可依并缩小认知差距)与高效性(每步信息量充足,不冗余或重复)。现有方法仅用二元终态验证器过滤或重标注教师轨迹,无法直接优化这两个维度,且对教师失败的情况无监督。多数真实问题附有开发者提供的参考补丁 $p^ ext{⋆}$,揭示了文件路径、运行行为和编码规范等关键信息,但标准流程将其丢弃。本文提出P2T(Patches-to-Trajectories),将 $p^ ext{⋆}$ 作为特权信息,在数据筛选中构建双目标优化:每步有效性与轨迹长度。反向阶段从 $p^ ext{⋆}$ 提炼出隐式过程图 $G^ ext{⋆}$,包含上下文事实与解题里程碑;正向阶段通过遮蔽教师续写,基于 $G^ ext{⋆}$ 评分每步进展,并通过防泄露的接地性检查,保留最短有效片段。仅用1.8k经筛选的SWE-Gym实例,P2T在效果与效率上均优于结果过滤的SFT及其工具错误掩码变体。在SWE-bench Verified上,Pass@1最高提升10.8点,单实例推理成本降低约15%,SWE-bench Lite上亦具持续优势。规模匹配的消融实验与定性分析进一步将轨迹质量提升归因于方法而非数据量。

原文摘要 · Abstract (English)

Supervised fine-tuning (SFT) on long teacher trajectories is the dominant way to instill investigation and reasoning in open software-engineering (SWE) agents. Since every retained response becomes an imitation target, the student inherits the final outcome and intermediate flaws, including ungrounded leaps and redundant loops. High-quality training data must be effective(each step is grounded and narrows the agent's epistemic gap to the correct fix) and efficient(each step is information-bearing rather than redundant or looping). Existing recipes filter or relabel teacher rollouts using only a binary terminal verifier, which does not directly target these axes and provides no supervision on instances where the teacher fails. Most real issue includes a developer-authored reference patch, $p^\star$, revealing the file paths, runtime behaviors, and coding conventions presupposed by the correct fix, yet standard pipelines discard it. We propose Patches-to-Trajectories (P2T), which uses $p^\star$ as privileged information during curation and formulates trajectory construction as bi-objective optimization over per-step effectiveness and trajectory length. A reverse phase distills $p^\star$ into a latent process graph, $G^\star$, of contextual facts and solution milestones. A forward phase curates trajectories from blinded teacher continuations by scoring per-step progress against $G^\star$ under a leakage-blocking groundedness check and retaining the shortest effective segments. Using only 1.8k curated SWE-Gym instances, P2T improves effectiveness and efficiency over outcome-filtered SFT and its tool-error-masking variant. On SWE-bench Verified, it raises Pass@1 by up to 10.8 points while reducing per-instance inference cost by ~15%, with consistent gains on SWE-bench Lite. Size-matched ablations and qualitative analysis further isolate trajectory quality from data scale.

程序生成强化学习代码修复智能代理

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。