分析大模型生成代码的可读性问题,发现提示词设计能有效改善但仍有局限。
Characterizing Readability Issue Patterns and the Role of Prompt Design in LLM-Generated Code

- 构建融合文本、结构、程序和视觉特征的可读性评估模型。
- 2735个场景下,大模型生成代码整体可读性接近人工代码,但存在复杂度高、注释冗余等问题。
- 函数签名、约束条件和风格描述是影响可读性的关键提示要素。
大型语言模型(LLMs)正改变代码生成方式,但生成代码仍需人工审查才能用于实际项目,因此可读性成为关键问题。现有研究多关注功能正确性和任务完成率,却忽视了代码是否可读、可读性失败的具体模式,以及提示词设计的影响。本文构建了一个整合文本、结构、程序和视觉特征的可读性评估模型,对比了2735个来自World of Code(WoC)和LeetCode的场景中,代表性前沿大模型生成代码与人工编写代码的可读性差异。通过主题分析刻画可读性问题模式,并开展受控提示变体实验探究提示设计关联。结果表明:当前大模型生成代码在整体可读性上接近人工代码,但隐藏着过度复杂、注释冗余和未知API使用等典型问题。提示词分析显示,函数签名、约束条件和风格描述对可读性影响最显著,但提示设计的整体作用仍有限。研究揭示了人工智能辅助编程中的潜在可读性债务,指出提示设计是提升可读性的轻量级切入点,并呼吁未来开发工作流中引入自动化工具以检测和缓解可读性问题。
原文摘要 · Abstract (English)
Large Language Models (LLMs) are increasingly changing how code is produced, but generated code still requires human review and validation before it can be adapted or integrated into real-world projects. This makes the readability of LLM-generated code a critical concern. Existing studies have mainly focused on functional correctness and task completion of LLM generated code, leaving open questions about whether it is readable, how its readability fails, and to what extent prompt design can improve it. We therefore investigate the readability of LLM-generated code. We first construct a readability assessment model that integrates textual, structural, program, and visual features. Using this model, we compare human-written code with code generated by representative frontier LLMs across 2,735 scenarios derived from World of Code (WoC) and LeetCode. We further characterize readability issue patterns using thematic analysis and examine prompt design associations through controlled prompt-variant experiments. Our results show that current LLMs produce code that is comparable to human-written code in overall readability. However, this aggregate similarity masks distinct issue patterns, including excessive complexity, redundant comments, and unknown API usage. Prompt analysis further shows that function signature, constraints, and style description are most strongly associated with code readability, although the overall role of prompt design remains bounded. These findings reveal latent readability debt in AI-assisted programming, identify prompt design as a lightweight starting point for improving generated-code readability, and motivate automated support for detecting and mitigating readability issues in future development workflows.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。