研究注释对大模型理解旧代码的影响，发现注释显著提升模型理解力。
Impact of Comments on LLM Comprehension of Legacy Code
- 用多选题评测大模型对旧代码的理解能力。
- 有注释时模型准确率显著提高，错误注释则产生误导。
- 适合关注代码可维护性与AI辅助开发的研究者。
大型语言模型（LLMs）在软件工程和维护任务中表现优异，尤其在现代编程语言理解方面。然而，其对遗留语言代码的理解能力仍存在研究空白，主要因真实遗留系统常缺乏或包含错误文档，影响模型理解。为客观评估该能力，需高效、量化的评价方法。本文采用新兴的多选题问答（MCQA）方法，评估大模型对遗留代码的理解水平及其受注释数量与准确性的影响。研究呈现初步结果，揭示注释对模型理解的关键作用，并提出未来研究的战略目标。
原文摘要 · Abstract (English)
Large language models (LLMs) have been increasingly integrated into software engineering and maintenance tasks due to their high performance with software engineering tasks and robust understanding of modern programming languages. However, the ability of LLMs to comprehend code written with legacy languages remains a research gap challenged by real-world legacy systems lacking or containing inaccurate documentation that may impact LLM comprehension. To assess LLM comprehension of legacy languages, there is a need for objective LLM evaluation. In order to objectively measure LLM comprehension of legacy languages, we need an efficient, quantitative evaluation method. We leverage multiple-choice question answering (MCQA), an emerging LLM evaluation methodology, to evaluate LLM comprehension of legacy code and the impact of comment prevalence and inaccurate comments. In this work, we present preliminary findings on the impact of documentation on LLM comprehension of legacy code and outline strategic objectives for future work.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。