arXiv:2608.08570cs.AI2026-08

让代码模型从失败中学习,提升解决难题能力。

FailForge: Distilling Procedural Competence from Persistent Failures into Code Agents

论文配图:FailForge: Distilling Procedural Competence from Persistent Failures into Code Agents
图 1 · 摘自论文原文
  • 从失败轨迹中诊断错误,提炼可复用的修复技能
  • 成功重试的轨迹回流训练集,使解决率提升6.6点
  • 适合需要攻克复杂编程难题的代码生成场景

拒绝采样微调(RFT)通过保留通过测试的代码执行轨迹来训练代码代理,但强模型仍会在大量任务上失败,而传统RFT直接丢弃这些失败样本。这些被丢弃的失败实例正是最难且最具信息量的,且其数据集构建成本高昂。我们提出FailForge,一种智能体框架,将失败轨迹转化为训练信号:针对每个失败实例,智能体基于错误反馈与执行轨迹诊断问题,提炼出简洁可操作的修复技能,并注入代理上下文进行有指导的第二次尝试。成功完成的轨迹被回流至RFT数据集。关键在于,训练时移除该技能,迫使模型内化修复行为而非依赖外部提示。实验表明,FailForge在仅增加少量成本的情况下,恢复了超过26%的先前失败实例;在Qwen3.5-4B上使用增强后的数据集训练,SWE-bench Verified的解决率比强基线提升6.6点,且增益集中于最困难的问题。

原文摘要 · Abstract (English)

Rejection sampling fine-tuning (RFT) is widely used to train code agents by generating trajectories on verifiable software engineering tasks, retaining those that pass the tests, and fine-tuning on the successful rollouts. However, even strong code agents repeatedly fail on a substantial fraction of such tasks, and standard RFT simply discards these failures. The discarded samples are precisely the hardest and most informative ones, drawn from verifiable instances that are costly to curate. Stronger base models may reduce the number of failures, but the remaining hard cases still define the frontier for further improvement. We propose FailForge, an agentic framework that converts failed rollouts into training signal. For each failed instance, an agent diagnoses the failure from error feedback and execution traces, distills the diagnosis into a concise and actionable skill, and injects the skill into the agent context for a guided second attempt. Trajectories that succeed under skill guidance are folded back into the RFT corpus. Crucially, the skill is removed at training time, so the model internalizes the recovered behavior rather than relying on external hints at inference. FailForge recovers over 26% of previously failed instances at marginal additional cost, and training Qwen3.5-4B on the augmented corpus improves the SWE-bench Verified resolve rate by 6.6 points over a strong RFT baseline, with gains concentrated on the hardest problems.

代码生成智能体强化学习

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。