arXiv:2507.00788cs.SEcs.AI2025-07中稿 · ance

测试AI编程助手对代码可维护性的影响,发现后续开发者演化代码无明显差异。

Echoes of AI: Investigating the Downstream Effects of AI Assistants on Software Maintainability

  • 通过控制实验对比人工与AI辅助开发的代码,由新开发者无辅助演化
  • 后续演化时间与代码质量无显著差异,AI提升效果小且不确定
  • 适合关注AI对长期开发影响的研究者和团队管理者

AI助手(如GitHub Copilot、Cursor)正改变软件工程。尽管已有研究显示生产力提升,但其对代码可维护性的影响仍需深入探究。本研究通过两阶段控制实验,招募151名参与者(95%为专业开发者),第一阶段中参与者在有或无AI辅助下为Java Web应用添加功能;第二阶段随机分配新开发者在无AI情况下演化这些代码。结果表明,后续演化在完成时间与代码质量上无显著差异。贝叶斯分析显示,使用AI带来的速度或质量提升最多为微小且高度不确定。第一阶段观察到,使用AI可使完成时间中位数减少30.7%,习惯使用者速度提升约55.9%。结论:在本研究任务与度量范围内,未发现代码可维护性系统性优劣。未来需关注过度生成导致的代码膨胀及认知负担累积风险。

原文摘要 · Abstract (English)

[Context] AI assistants, like GitHub Copilot and Cursor, are transforming software engineering. While several studies highlight productivity improvements, their impact on maintainability requires further investigation. [Objective] This study investigates whether co-development with AI assistants affects software maintainability, specifically how easily other developers can evolve the resulting source code. [Method] We conducted a two-phase controlled experiment involving 151 participants, 95% of whom were professional developers. In Phase 1, participants added a new feature to a Java web application, with or without AI assistance. In Phase 2, a randomized controlled trial, new participants evolved these solutions without AI assistance. [Results] Phase 2 revealed no significant differences in subsequent evolution with respect to completion time or code quality. Bayesian analysis suggests that any speed or quality improvements from AI use were at most small and highly uncertain. Observational results from Phase 1 corroborate prior research: using an AI assistant yielded a 30.7% median reduction in completion time, and habitual AI users showed an estimated 55.9% speedup. [Conclusions] Overall, we did not detect systematic maintainability advantages or disadvantages when other developers evolved code co-developed with AI assistants. Within the scope of our tasks and measures, we observed no consistent warning signs of degraded code-level maintainability. Future work should examine risks such as code bloat from excessive code generation and cognitive debt as developers offload more mental effort to assistants.

AI编程代码可维护性软件工程

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。