测试发现机器写的代码更难维护，后续修改成功率低13.1%
Is Agent Code Less Maintainable Than Human Code?
- 构建CodeThread框架，控制实验环境对比人与机器代码的可维护性
- 基于4个前沿模型和4个基准测试，机器代码后续任务解决率下降最多13.1%
- 传统可维护性指标无法解释差异，行为细节如错误处理更关键
可维护性是软件工程的核心维度，影响代码的编写、审查与长期演化。尽管编码代理在单任务上表现优异，但其代码在后续由其他代理迭代时的可维护性尚不明确，可能引发连锁性下游问题。本文提出CodeThread框架，从仓库级编码基准中构建受控实验，评估代理代码与人类代码在维护场景下的表现。对四个前沿编码代理和四个基准的测试显示，代理在基于代理代码的任务中解决率低于基于人类代码，最高下降13.1%。回归分析表明，传统可维护性度量无法解释这一差距。最显著的信号来自代理代码的细微行为差异，如输入验证与错误处理方式的变化，以及下游代码规模和任务难度的不同。研究强调需不仅关注即时任务完成，还需评估代码可维护性，并揭示了代理代码可能引入的下游错误来源。
原文摘要 · Abstract (English)
Maintainability is a core dimension of software engineering, shaping how code is written, reviewed, and developed over time. While coding agents have demonstrated strong performance on single-issue tasks, it remains unclear how maintainable their code is when future agents build on top of it, potentially leading to compounding downstream effects. We investigate how agent code compares to human code in these maintenance settings, presenting CodeThread, a framework to construct controlled experiments from repository-level coding benchmarks. Applying CodeThread to four frontier coding agents and four benchmarks, we find that agents are less effective at resolving tasks when building on agent code compared to human code, with task resolve rate drops of up to 13.1%. Regression analysis reveals that many traditional software engineering maintainability metrics do not explain this difference. Instead, the clearest signals are subtler behavioral differences in agent code, such as changes to input validation and error handling, along with differences in downstream code size and task difficulty. These findings highlight the need to evaluate these systems not only by immediate task resolution but also by code maintainability, and point to potential sources of downstream errors introduced by agent code.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。