用多面测试诊断引导大模型修复代码，显著提升准确率。
SWE-Doctor: Guiding Software Engineering Agents with Runtime Diagnosis from Multi-Faceted Bug Reproduction Tests
- 通过多面错误重现测试生成运行时诊断，指导补丁生成
- 在SWE-bench上平均修复率达75.7%（验证集）和59.4%（专业版）
- 适合需要高精度自动修复的开发者与自动化测试研究者
基于大语言模型（LLM）的软件工程智能体正被用于从问题报告和代码库中生成补丁。错误重现测试（BRTs）是此类智能体的重要组成部分，已被证明对补丁验证有帮助。然而，现有研究尚未明确其是否能有效辅助更核心的补丁生成阶段。我们初步研究表明，直接使用先进BRT生成器指导补丁生成效果不佳：失败-失败型BRT会误导智能体，而即使成功-失败型BRT也仅带来有限或负向收益。分析发现，原因在于成功-失败型BRT可能仅覆盖问题的一种表现形式，导致部分补丁；而失败-失败型BRT作为补丁生成目标不可靠。基于此洞察，我们提出SWE-Doctor，一种利用多面BRT执行后的运行时诊断来引导补丁生成的智能体。该方法先为问题中描述的不同行为需求生成多面BRT，再执行并调试这些测试以构建运行时诊断记录，并结合生成过程中推断出的定位信息，指导补丁生成并减少部分补丁。我们在广泛使用的SWE-bench Verified和SWE-bench Pro数据集上，针对五种LLM后端进行评估。SWE-Doctor在所有10组实验组合中均优于现有智能体，在SWE-bench Verified上平均修复率达75.7%，在更具挑战性的SWE-bench Pro上达59.4%。尤其在后者上，相比基线平均提升8.0至8.9个百分点。
原文摘要 · Abstract (English)
Large language model (LLM)-based software engineering agents are increasingly developed to resolve software issues by generating patches from issue reports and code repositories. Bug reproduction tests (BRTs) are an important building block for such agents and have been shown useful for patch validation. However, it remains unclear whether BRTs can also help the more central stage of patch generation. We first conduct a preliminary study and find that directly using advanced BRT generators to guide patch generation is not beneficial: fail-to-fail BRTs can mislead agents, while even fail-to-pass BRTs bring limited or negative gains. Our analysis reveals two reasons: fail-to-pass BRTs may cover only one manifestation of the reported issue, leading to partial patches, whereas fail-to-fail BRTs are unreliable as direct patch-generation targets. Motivated by these insights, we propose SWE-Doctor, a software issue resolution agent that guides patch generation with runtime diagnoses derived from multi-faceted BRT executions. SWE-Doctor first generates multi-faceted BRTs for different behavioral requirements stated in the issue, then executes and debugs these BRTs to construct runtime-grounded diagnosis records, and finally uses the diagnoses together with localization information inferred during BRT generation to guide patch generation and reduce partial patches. We evaluate SWE-Doctor on Python bug-fixing issues from the widely adopted SWE-bench Verified and SWE-bench Pro across five LLM backends. SWE-Doctor consistently outperforms existing agents across all 10 LLM-benchmark combinations, achieving average resolution rates of 75.7% on SWE-bench Verified and 59.4% on SWE-bench Pro. In particular, on the more challenging SWE-bench Pro, SWE-Doctor improves the average resolution rate by 8.0-8.9 percentage points over the baseline agents.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。