用推理时奖励模型纠正代码生成智能体的低效路径，提升成功率10.6个百分点。
When Agents go Astray: Course-Correcting SWE Agents with PRMs
- 引入SWE-PRM，在执行中检测并修正代码生成过程中的冗余探索、死循环等问题。
- 在SWE-bench Verified上将解决率从40.0%提升至50.6%，中高难度任务提升显著。
- 基于错误类型分类的反馈优于盲目或直接动作指导，且额外推理成本仅0.2美元。
大型语言模型（LLM）智能体正被广泛用于复杂的多步骤软件工程任务，但其执行轨迹常出现冗余探索、死循环及过早终止等低效问题。以往工作多采用事后诊断，而本文提出SWE-PRM——一种推理时的流程奖励模型，在执行过程中实时检测并纠正轨迹级错误。该模型基于常见效率问题的分类体系，提供轻量、可解释的反馈，不修改底层策略。在SWE-bench Verified数据集上，闭源PRM将解决率从40.0%提升至50.6%（+10.6个百分点），尤其在中等和困难任务上收益最大。相较而言，基于分类的反馈策略优于无指导或显式动作引导方案，既提高成功率又缩短轨迹长度。额外推理成本低至0.2美元，具备实际应用与扩展性。
原文摘要 · Abstract (English)
Large Language Model (LLM) agents are increasingly deployed for complex, multi-step software engineering (SWE) tasks. However, their trajectories often contain costly inefficiencies, such as redundant exploration, looping, and failure to terminate once a solution is reached. Prior work has largely treated these errors in a post-hoc manner, diagnosing failures only after execution. In this paper, we introduce SWE-PRM, an inference-time Process Reward Model (PRM) that intervenes during execution to detect and course-correct trajectory-level errors. Our PRM design leverages a taxonomy of common inefficiencies and delivers lightweight, interpretable feedback without modifying the underlying policy. On SWE-bench Verified, closed-source PRMs improve resolution from 40.0% to 50.6% (+10.6 p.p.), with the largest gains on medium and hard tasks. Among feedback strategies, taxonomy-guided PRMs outperform unguided or explicit action-prescriptive variants, increasing success rate while reducing trajectory length. These benefits come at an acceptable added inference cost of as low as $0.2, making PRMs a practical and scalable mechanism for improving SWE agents' reliability and efficiency.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。