代码注释能显著提升AI修Bug准确率，最多提高三倍。
On the Impact of Code Comments for Automated Bug-Fixing: An Empirical Study
- 保留注释训练和推理，可让AI修Bug更准
- 有注释时修复准确率最高提升三倍
- 尤其解释实现逻辑的注释最有效，适合做代码修复研究
大型语言模型（LLMs）在软件工程中应用日益广泛，自动化缺陷修复（ABF）是其关键场景之一。当前主流做法是在训练前移除代码注释，但我们假设注释可能对修复特定类型缺陷具有关键作用，因其蕴含设计与实现洞察。本研究通过实证评估两种模型族，在训练与推理阶段是否保留注释的所有组合，重新审视注释删除的常规做法。针对现有数据集注释稀缺的问题，我们使用大模型自动为无注释方法生成注释。结果表明：若训练与推理阶段均保留注释，修复准确率最高可提升三倍；而训练时即使缺少注释，也不会降低模型性能。可解释性分析进一步发现，描述方法实现细节的注释对提升修复准确性尤为有效。
原文摘要 · Abstract (English)
Large Language Models (LLMs) are increasingly relevant in Software Engineering research and practice, with Automated Bug Fixing (ABF) being one of their key applications. ABF involves transforming a buggy method into its fixed equivalent. A common preprocessing step in ABF involves removing comments from code prior to training. However, we hypothesize that comments may play a critical role in fixing certain types of bugs by providing valuable design and implementation insights. In this study, we investigate how the presence or absence of comments, both during training and at inference time, impacts the bug-fixing capabilities of LLMs. We conduct an empirical evaluation comparing two model families, each evaluated under all combinations of training and inference conditions (with and without comments), and thereby revisiting the common practice of removing comments during training. To address the limited availability of comments in state-of-the-art datasets, we use an LLM to automatically generate comments for methods lacking them. Our findings show that comments improve ABF accuracy by up to threefold when present in both phases, while training with comments does not degrade performance when instances lack them. Additionally, an interpretability analysis identifies that comments detailing method implementation are particularly effective in aiding LLMs to fix bugs accurately.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。