用编程知识图谱精准检索代码片段,提升复杂问题生成准确率
Context-Augmented Code Generation Using Programming Knowledge Graphs
- 构建编程知识图谱,按模块和函数粒度检索代码
- 树剪枝与重排序降低无关信息干扰,pass@1提升20%
- 自动补全注释文档,适合需要高质量代码的开发者
大型语言模型(LLMs)和代码专用模型(CLLMs)在代码生成上取得显著进展,但在处理复杂问题时仍常出错。检索增强生成(RAG)通过推理时引入外部知识缓解此问题,但现有检索模型难以找到最相关上下文,而生成模型因上下文容量有限,易在引入无关数据时产生幻觉。本文提出一种新框架,利用编程知识图谱(PKG)对代码进行语义表示与检索。该方法通过聚焦最相关代码片段并采用树剪枝技术减少冗余上下文,实现细粒度检索。PKG结合重排序机制,进一步抑制幻觉,选择更优解。提出两种基于PKG的检索方式:块级与函数级,优化上下文粒度。在HumanEval和MBPP基准测试中,本方法将pass@1准确率提升最高达20%,在MBPP上超越最先进模型34%。贡献包括基于PKG的检索、树剪枝提升精度、重排序机制实现鲁棒解选择,以及填空中间(FIM)增强模块,可自动添加相关注释与文档字符串。
原文摘要 · Abstract (English)
Large Language Models (LLMs) and Code-LLMs (CLLMs) have significantly improved code generation, but, they frequently face difficulties when dealing with challenging and complex problems. Retrieval-Augmented Generation (RAG) addresses this issue by retrieving and integrating external knowledge at the inference time. However, retrieval models often fail to find most relevant context, and generation models, with limited context capacity, can hallucinate when given irrelevant data. We present a novel framework that leverages a Programming Knowledge Graph (PKG) to semantically represent and retrieve code. This approach enables fine-grained code retrieval by focusing on the most relevant segments while reducing irrelevant context through a tree-pruning technique. PKG is coupled with a re-ranking mechanism to reduce even more hallucinations by selectively integrating non-RAG solutions. We propose two retrieval approaches-block-wise and function-wise-based on the PKG, optimizing context granularity. Evaluations on the HumanEval and MBPP benchmarks show our method improves pass@1 accuracy by up to 20%, and outperforms state-of-the-art models by up to 34% on MBPP. Our contributions include PKG-based retrieval, tree pruning to enhance retrieval precision, a re-ranking method for robust solution selection and a Fill-in-the-Middle (FIM) enhancer module for automatic code augmentation with relevant comments and docstrings.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。