用混合检索与代码图结构防止大模型编造代码引用,准确率达92%且无幻觉。
Citation-Grounded Code Comprehension: Preventing LLM Hallucination Through Hybrid Retrieval and Graph-Augmented Context
- 融合关键词匹配、语义向量与代码导入关系图进行混合检索
- 在30个仓库上实现92%引用准确率,零幻觉,跨文件证据发现率提升62%
- 适合需要高可靠性的代码理解系统开发者或研究者使用
大语言模型已成为代码理解的重要工具,使开发者可通过自然语言查询陌生代码库。但模型幻觉——生成看似合理却错误的代码引用——仍是可靠开发辅助的关键障碍。本文通过混合检索与轻量结构推理,解决可验证的引用扎根代码理解问题。在30个Python仓库、180个开发者查询上的系统评估表明,引用准确性受稀疏词法匹配、密集语义相似性及跨文件架构依赖的相互作用影响。其中,跨文件证据发现是引用完整性最大贡献者,但现有系统因仅依赖文本相似性而忽略此点。本文提出将引用扎根生成作为代码理解系统的架构原则,并实现92%引用准确率且零幻觉。具体构建了结合BM25稀疏匹配、BGE稠密嵌入与基于Neo4j导入关系图扩展的混合检索系统,在跨文件证据发现上比纯文本相似性高出62%,性能优于单一模式基线14至18个百分点。
原文摘要 · Abstract (English)
Large language models have become essential tools for code comprehension, enabling developers to query unfamiliar codebases through natural language interfaces. However, LLM hallucination, generating plausible but factually incorrect citations to source code, remains a critical barrier to reliable developer assistance. This paper addresses the challenges of achieving verifiable, citation grounded code comprehension through hybrid retrieval and lightweight structural reasoning. Our work is grounded in systematic evaluation across 30 Python repositories with 180 developer queries, comparing retrieval modalities, graph expansion strategies, and citation verification mechanisms. We find that challenges of citation accuracy arise from the interplay between sparse lexical matching, dense semantic similarity, and cross file architectural dependencies. Among these, cross file evidence discovery is the largest contributor to citation completeness, but it is largely overlooked because existing systems rely on pure textual similarity without leveraging code structure. We advocate for citation grounded generation as an architectural principle for code comprehension systems and demonstrate this need by achieving 92 percent citation accuracy with zero hallucinations. Specifically, we develop a hybrid retrieval system combining BM25 sparse matching, BGE dense embeddings, and Neo4j graph expansion via import relationships, which outperforms single mode baselines by 14 to 18 percentage points while discovering cross file evidence missed by pure text similarity in 62 percent of architectural queries.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。