LLMs能替代人类完成遗留代码分块，生成更准确的文档。
Can LLMs Replace Humans During Code Chunking?
- 用LLM自动划分超长遗留代码，解决上下文长度限制问题。
- LLM分块生成的注释比人工分块更准确，事实性高20%，更有用10%。
- 适合政府遗留系统现代化，尤其对MUMPS、ALC等老语言有效。
大型语言模型（LLMs）在计算机科学中已成为代码理解与生成的关键工具。然而，现有研究未充分应对政府应用代码特有的挑战：这类代码常使用MUMPS或汇编语言（ALC），整体令牌长度超过当前商用LLM的上下文窗口。此外，LLMs主要基于现代编程语言训练，对遗留语言的理解能力尚不明确。本文研究了在现代化过程中使用LLM处理ALC和MUMPS遗留代码的分块方法，评估不同分块策略对生成模块注释质量的影响。实验对比了GPT-4o、Claude 3 Sonnet、Mixtral和Llama 3等模型的表现。结果表明，LLMs选择的分割点与人类专家高度一致。分块方式显著影响下游任务表现：由LLM生成的分块所产注释，事实性最高提升20%，实用性最高提升10%。因此，我们得出结论：在LLM辅助现代化过程中，LLMs可作为人类分块的有效替代方案。
原文摘要 · Abstract (English)
Large language models (LLMs) have become essential tools in computer science, especially for tasks involving code understanding and generation. However, existing work does not address many of the unique challenges presented by code written for government applications. In particular, government enterprise software is often written in legacy languages like MUMPS or assembly language code (ALC) and the overall token lengths of these systems exceed the context window size for current commercially available LLMs. Additionally, LLMs are primarily trained on modern software languages and have undergone limited testing with legacy languages, making their ability to understand legacy languages unknown and, hence, an area for empirical study. This paper examines the application of LLMs in the modernization of legacy government code written in ALC and MUMPS, addressing the challenges of input limitations. We investigate various code-chunking methods to optimize the generation of summary module comments for legacy code files, evaluating the impact of code-chunking methods on the quality of documentation produced by different LLMs, including GPT-4o, Claude 3 Sonnet, Mixtral, and Llama 3. Our results indicate that LLMs can select partition points closely aligned with human expert partitioning. We also find that chunking approaches have significant impact on downstream tasks such as documentation generation. LLM-created partitions produce comments that are up to 20% more factual and up to 10% more useful than when humans create partitions. Therefore, we conclude that LLMs can be used as suitable replacements for human partitioning of large codebases during LLM-aided modernization.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。