arXiv:2511.10271cs.SEcs.AI2025-11

评估大模型生成代码的非功能性质量,发现学术与工业界关注点不一致。

Quality Assurance of LLM-generated Code: Addressing Non-Functional Quality Characteristics

  • 基于ISO/IEC 25010模型,结合文献、实践访谈与实证分析
  • 生成代码在安全、可维护性、性能上表现不稳定,易积累技术债
  • 适合关注代码质量保障的开发者与研究者参考

近年来,大语言模型被广泛应用于软件工程流程,支持代码生成等任务。尽管以往评估多聚焦功能正确性,对生成代码的非功能性质量特征仍缺乏深入理解。本研究基于ISO/IEC 25010质量模型,采用三重互补方法:对109篇论文的文献综述、来自多家企业的两场行业研讨会,以及使用三种LLM修复真实软件问题的实证分析。结合文献与从业者洞察,实证研究重点考察了生成补丁在安全性、可维护性和性能效率方面的质量,这些被识别为关键代码级质量属性。结果表明,现有研究主要关注安全、性能和可维护性,而其他质量属性仍被忽视;相比之下,从业者更重视可维护性与可读性,警示生成代码可能加速技术债积累。实证评估显示,在实际工程场景中,仅靠提示词优化非功能性质量特性存在显著不稳定性。整体而言,研究揭示了学术关注、行业优先级与模型行为之间的错配,强调需在大模型代码生成流程中嵌入质量保障机制,确保生成代码不仅通过测试,更具备高质量。

原文摘要 · Abstract (English)

In recent years, large language models have been widely integrated into software engineering workflows, supporting tasks like code generation. While prior evaluations focus on functional correctness, there is still a limited understanding of the non-functional quality characteristics of generated code. Guided by the ISO/IEC 25010 quality model, this study adopts a multi-methods approach comprising three complementary elements: a literature review of 109 papers, two industry workshops with practitioners from multiple organizations, and an empirical analysis of patching real-world software issues using three LLMs. Motivated by insights from both the literature and practitioners, the empirical study examined the quality of generated patches regarding security, maintainability, and performance efficiency, which were identified as critical code-level quality attributes. Our results indicate that existing research primarily emphasizes security, performance efficiency, and maintainability, while other quality attributes are understudied. In contrast, practitioners prioritize maintainability and readability, warning that generated code may accelerate the accumulation of technical debt. The empirical evaluation demonstrates the instability of optimizing NFQCs through prompts in practical software engineering settings. Overall, our findings expose a misalignment between academic focus, industry priorities, and observed model behavior, highlighting the need to integrate quality assurance mechanisms into LLM code generation pipelines to ensure that future generated code not only passes tests but truly passes with quality.

代码生成质量评估LLM技术债

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。