arXiv:2409.00921cs.PLcs.AI2024-09被引 21

让大模型生成代码时用上类型上下文,减少错误。

Statically Contextualizing Large Language Models with Typed Holes

  • 用语言服务器提供类型和作用域信息,增强代码补全上下文。
  • 在MVUBench数据集上,类型上下文使生成代码正确率显著提升。
  • 适合需要精准代码生成的开发者,尤其对复杂类型系统有效。

大型语言模型(LLMs)已重塑程序合成领域。然而,当前基于LLM的代码补全系统常因缺乏适当上下文而生成错误代码,尤其是在处理训练数据中未出现或远离光标位置的定义时。本文表明,通过与语言服务器提供的类型和绑定结构紧密集成,可在不增加大量标记的情况下有效解决这一问题。我们提出将LLM代码生成集成到Hazel实时程序草图环境中,该环境可识别待填充的‘孔洞’的类型与类型上下文,即使存在错误也能保证生成有意义的程序草图。这使得提示词能包含整个代码库的上下文信息,而不仅是光标附近或同一文件的内容,但与开发者目标语义相关。随后,利用与语言服务器的进一步交互迭代优化生成结果。为评估这些方法,我们引入了MVUBench——一个用于模型-视图-更新(MVU)Web应用的数据集,其依赖特定应用数据结构,构成挑战性任务。实验发现,引入类型定义进行上下文化尤为有效。在Hazel中验证后,我们将方法复现并移植至TypeScript,以验证其在高资源语言中的适用性。最后,我们提出了ChatLSP,作为语言服务器协议(LSP)的保守扩展,允许语言服务器暴露能力,供不同设计的AI代码补全系统使用,从而在生成提示时融入静态上下文。

原文摘要 · Abstract (English)

Large language models (LLMs) have reshaped the landscape of program synthesis. However, contemporary LLM-based code completion systems often hallucinate broken code because they lack appropriate context, particularly when working with definitions not in the training data nor near the cursor. This paper demonstrates that tight integration with the type and binding structure of a language, as exposed by its language server, can address this contextualization problem in a token-efficient manner. In short, we contend that AIs need IDEs, too! In particular, we integrate LLM code generation into the Hazel live program sketching environment. The Hazel Language Server identifies the type and typing context of the hole being filled, even in the presence of errors, ensuring that a meaningful program sketch is always available. This allows prompting with codebase-wide contextual information not lexically local to the cursor, nor necessarily in the same file, but that is likely to be semantically local to the developer's goal. Completions synthesized by the LLM are then iteratively refined via further dialog with the language server. To evaluate these techniques, we introduce MVUBench, a dataset of model-view-update (MVU) web applications. These applications serve as challenge problems due to their reliance on application-specific data structures. We find that contextualization with type definitions is particularly impactful. After introducing our ideas in the context of Hazel we duplicate our techniques and port MVUBench to TypeScript in order to validate the applicability of these methods to higher-resource languages. Finally, we outline ChatLSP, a conservative extension to the Language Server Protocol (LSP) that language servers can implement to expose capabilities that AI code completion systems of various designs can use to incorporate static context when generating prompts for an LLM.

代码生成类型上下文IDE集成LLM

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。