TDFlow用分步代理实现测试驱动的代码修复，性能逼近人类。
TDFlow: Agentic Workflows for Test Driven Development
- 拆解修复任务为四阶段子流程，各由专用代理负责。
- 在SWE-Bench Lite上通过率达88.8%，优于现有系统27.8%。
- 适合研究自动化代码修复与人机协同开发的开发者。
我们提出TDFlow，一种面向测试驱动开发的新型智能体工作流，将仓库级软件工程视为测试求解任务，专为解决人工编写的测试而设计。给定一组测试后，TDFlow通过精心设计的子代理和严格约束的工具，反复提出、修正并调试仓库级补丁。该流程将程序修复分解为四个由对应子代理管理的组件：（1）降低单个子代理的长上下文负担；（2）使每个子代理聚焦于预定义的具体子任务；（3）支持对特定子任务的性能专项优化。在提供人工编写测试的情况下，TDFlow在SWE-Bench Lite上达到88.8%的通过率（相比次优系统绝对提升27.8%），在SWE-Bench Verified上达94.3%。对SWE-Bench Lite和Verified中800次运行的手动检查仅发现7例测试作弊情况，均已计为失败。此外，我们表明当前实现人类级软件工程能力的主要障碍在于生成有效的复现测试。我们设想一个基于TDFlow的人机交互系统，由人类编写测试，由大模型系统求解。这些结果表明，现代大模型若嵌入精心设计的、以测试驱动的工作流中，已可实现人类级别的测试求解能力——而完全自治的仓库级修复的最后挑战，是准确生成有效复现测试。
原文摘要 · Abstract (English)
We introduce TDFlow, a novel test-driven agentic workflow that frames repository-scale software engineering as a test-resolution task, specifically designed to solve human-written tests. Given a set of tests, TDFlow repeatedly proposes, revises, and debugs repository-scale patches using precisely engineered sub-agents and tightly constrained tools. The workflow decomposes software engineering program repair into four components governed by respective sub-agents. This simple, forced decoupling of patch proposing, debugging, patch revision, and optional test generation (1) reduces long-context burden on any individual sub-agent, (2) focuses each sub-agent on specific, pre-defined sub-tasks, and (3) allows for specialized performance improvement on specific sub-tasks. When provided human-written tests, TDFlow attains 88.8% pass rate on SWE-Bench Lite (an absolute improvement of 27.8% over the next best system) and 94.3% on SWE-Bench Verified. Manual inspection of the 800 TDFlow runs within SWE-Bench Lite and Verified uncover only 7 instances of test hacking, which were subsequently counted as failures. Furthermore, we show that the primary obstacle to human-level software engineering performance lies within writing successful reproduction tests. We envision a human-LLM interactive system powered by TDFlow where human developers write tests solved by LLM systems. Together, these results indicate that modern LLMs, when embedded in a narrowly engineered, test-driven workflow, already achieve human-level test resolution -- with the final frontier for fully autonomous repository repair being the accurate generation of valid reproduction tests.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。