用错误回溯生成引导提示，让代码代理无需思维链也能高效推理。
Hindsight Hint Distillation: Scaffolded Reasoning for SWE Agents from CoT-free Answers
- 从模型自失败的执行中提取事后提示，指导成功路径探索。
- 在SWE-bench Verified上提升8%，远超其他基线约2%的改进。
- 能泛化到未见过的任务类型，尤其擅长多语言场景。
解决复杂长周期任务需要强大的规划与推理能力。尽管带有显式思维链（CoT）标注的数据集能显著促进学习，但其获取成本高昂。为此，我们提出事后提示蒸馏（HHD），仅需易于获得的问题-答案对，无需CoT标注。受人类教师利用学生错误提供针对性指导的启发，HHD从模型自身的失败自执行轨迹中合成事后提示，并用于引导成功完成任务的在线轨迹。模型随后对这些引导轨迹进行自蒸馏，实现对新问题的泛化，且无需提示支持。实验表明，HHD显著优于迭代RFT和轨迹合成基线，在SWE-bench Verified上实现8%的绝对提升，而所有基线仅提升约2%。值得注意的是，由HHD诱导的推理策略能有效泛化至分布外任务，在无多语言训练数据的情况下，于SWE-bench Multilingual上取得最大收益。结果表明，HHD能从无思维链数据中有效合成类专家推理，显著提升长周期任务表现。
原文摘要 · Abstract (English)
Solving complex long-horizon tasks requires strong planning and reasoning capabilities. Although datasets with explicit chain-of-thought (CoT) rationales can substantially benefit learning, they are costly to obtain. To address this challenge, we propose Hindsight Hint Distillation (HHD), which only requires easy-to-obtain question-answer pairs without CoT annotations. Inspired by how human teachers use student mistakes to provide targeted guidance, HHD synthesizes hindsight hints from the model's own failed self-rollouts and uses them to scaffold on-policy rollouts that successfully complete the tasks. The model then self-distills these scaffolded trajectories and generalizes to new problems without hint guidance. Experiments show that HHD significantly outperforms iterative RFT and trajectory-synthesis baselines, achieving an absolute improvement of 8\% on SWE-bench Verified, while all baselines improve by only around 2\%. Notably, the reasoning strategies induced by HHD generalize effectively to out-of-distribution tasks, yielding the largest gains on SWE-bench Multilingual despite no training on multilingual data. These results demonstrate that HHD can effectively synthesize expert-like reasoning from CoT-free data and substantially improve long-horizon performance.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。