AI生成代码比预想更持久,但维护方式与人类不同。
Will It Survive? Deciphering the Fate of AI-Generated Code in Open Source
- 通过生存分析对比200000+代码单元,发现AI代码修改率更低
- AI代码修改率低15.8个百分点,但纠错需求略高
- 代码修改时机难预测,组织流程是关键瓶颈
将AI代理作为编程助手融入软件开发,引发对其生成代码长期可行性的关注。社区普遍认为这类代码是‘一次性’的——快速合并后很快被丢弃。若属实,组织将面临从生成转向后期修复的维护负担。本文通过对201个开源项目、超过20万段由AI代理和人类编写的代码单元进行生存分析检验该假设。结果与‘一次性’观点相反:在行级层面,AI生成代码的修改率低15.8个百分点,修改风险降低16%(危险比HR = 0.842,p < 0.001)。然而修改模式存在差异:AI代码的修正率稍高(26.3% vs. 23.0%),而人类代码适应性修改更多。效应量较小(Cramér's V = 0.116),且单个代理间差异大于代理与人类之间的差距。文本特征可识别易修改代码(AUC-ROC = 0.671),但预测修改时间困难(宏平均F1 = 0.285),表明时间取决于外部组织动态。因此,AI代码的瓶颈可能不在生成质量,而在决定其长期演化的组织实践。
原文摘要 · Abstract (English)
The integration of AI agents as coding assistants into software development has raised questions about the long-term viability of AI agent-generated code. A prevailing hypothesis within the software engineering community suggests this code is "disposable", meaning it is merged quickly but discarded shortly thereafter. If true, organizations risk shifting maintenance burden from generation to post-deployment remediation. We investigate this hypothesis through survival analysis of 201 open-source projects, tracking over 200,000 code units authored by AI agents versus humans. Contrary to the disposable code narrative, agent-authored code survives significantly longer: at the line level, it exhibits a 15.8 percentage-point lower modification rate and 16% lower hazard of modification (HR = 0.842, p < 0.001). However, modification profiles differ. Agent-authored code shows modestly elevated corrective rates (26.3% vs. 23.0%), while human code shows higher adaptive rates. However, the effect sizes are small (Cramér's V = 0.116), and per-agent variation exceeds the agent-human gap. Turning to prediction, textual features can identify modification-prone code (AUC-ROC = 0.671), but predicting when modifications occur remains challenging (Macro F1 = 0.285), suggesting timing depends on external organizational dynamics. The bottleneck for agent-generated code may not be generation quality, but the organizational practices that govern its long-term evolution.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。