AI IDE能生成大型可运行代码，但存在设计缺陷需人工把关。
Beyond Functional Correctness: Design Issues in AI IDE-Generated Large-Scale Projects
- 用人类引导框架生成项目，平均16965行代码、114个文件。
- 功能正确率91%，但静态分析发现超4500个设计问题。
- 主要问题为代码重复、复杂度高，影响长期维护，适合开发者评估。
新一代AI编程工具，包括具备代理能力的AI IDE，可在项目上下文中生成代码。尽管这些工具被认为能大规模生成项目级代码，但其实际能力与生成系统的设计质量仍缺乏实证研究。为此，我们通过特征驱动的人类在环（FD-HITL）框架，使用Cursor生成了10个跨三个应用领域、多种技术的大规模项目。手动评估显示平均功能正确率达91%。利用CodeScene和SonarQube进行静态分析，分别检测到1,305个和3,193个设计问题，涵盖9类与11类问题。主要问题包括代码重复、高复杂度、大方法、框架最佳实践违反、异常处理不当及可访问性问题，违反单职责原则（SRP）、关注点分离（SoC）和不要重复自己（DRY）。这些问题可能带来长期可维护性与可演化风险，需由经验开发者审慎审查。
原文摘要 · Abstract (English)
New generation of AI coding tools, including AI-powered IDEs equipped with agentic capabilities, can generate code within the context of the project. These AI IDEs are increasingly perceived as capable of producing project-level code at scale. However, there is limited empirical evidence on the extent to which they can generate large-scale software systems and what design issues such systems may exhibit. To address this gap, we conducted a study to explore the capability of Cursor in generating large-scale projects and to evaluate the design quality of projects generated by Cursor. First, we propose a Feature-Driven Human-In-The-Loop (FD-HITL) framework that systematically guides project generation from curated project descriptions. We generated 10 projects using Cursor with the FD-HITL framework across three application domains and multiple technologies. We assessed the functional correctness of these projects through manual evaluation, obtaining an average functional correctness score of 91%. Next, we analyzed the generated projects using two static analysis tools, CodeScene and SonarQube, to detect design issues. We identified 1,305 design issues categorized into 9 categories by CodeScene and 3,193 issues in 11 categories by SonarQube. Our findings show that (1) when used with the FD-HITL framework, Cursor can generate functional large-scale projects averaging 16,965 LoC and 114 files; (2) the generated projects nevertheless contain design issues that may pose long-term maintainability and evolvability risks, requiring careful review by experienced developers; (3) the most prevalent issues include Code Duplication, high Code Complexity, Large Methods, Framework Best-Practice Violations, Exception-Handling Issues and Accessibility Issues; (4) these design issues violate design principles such as SRP, SoC, and DRY. The replication package is at https://github.com/Kashifraz/DIinAGP
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。