arXiv:2506.15453cs.SEcs.AI2025-06被引 1

用大模型生成代码注释,发现多数原注释是示例用法。

Uncovering Intention through LLM-Driven Code Snippet Description Generation

  • 基于18.5万包的代码片段,分析开发者常用注释类型。
  • 大模型识别出79.75%为示例类注释,与人工判断一致。
  • 生成注释平均相似度0.7173,虽相关但仍有改进空间。

代码片段文档化对开发者和用户关注重点至关重要,尤其在第三方库中。本文基于包含185,412个包、共1,024,579个代码片段的NPM数据集,选取400个样本进行分析。手动分类发现,55.5%的原始描述聚焦于示例用法,凸显清晰文档的重要性;部分描述缺乏足够细节以传达意图。使用Llama大模型生成注释,结果显示其正确识别出79.75%的原描述为“示例”类型,与人工结果一致,体现模型良好的泛化能力。生成描述与原描述的平均相似度为0.7173,表明内容相关但存在偏差,低于0.9说明仍有一定无关性。研究指出,代码片段意图可能因任务而异,涵盖使用说明、安装指引或学习示例等不同方向。

原文摘要 · Abstract (English)

Documenting code snippets is essential to pinpoint key areas where both developers and users should pay attention. Examples include usage examples and other Application Programming Interfaces (APIs), which are especially important for third-party libraries. With the rise of Large Language Models (LLMs), the key goal is to investigate the kinds of description developers commonly use and evaluate how well an LLM, in this case Llama, can support description generation. We use NPM Code Snippets, consisting of 185,412 packages with 1,024,579 code snippets. From there, we use 400 code snippets (and their descriptions) as samples. First, our manual classification found that the majority of original descriptions (55.5%) highlight example-based usage. This finding emphasizes the importance of clear documentation, as some descriptions lacked sufficient detail to convey intent. Second, the LLM correctly identified the majority of original descriptions as "Example" (79.75%), which is identical to our manual finding, showing a propensity for generalization. Third, compared to the originals, the produced description had an average similarity score of 0.7173, suggesting relevance but room for improvement. Scores below 0.9 indicate some irrelevance. Our results show that depending on the task of the code snippet, the intention of the document may differ from being instructions for usage, installations, or descriptive learning examples for any user of a library.

代码生成LLM应用文档自动化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。