用静态分析检测并修复大模型生成代码中的虚构错误
Detecting and Correcting Hallucinations in LLM-Generated Code via Deterministic AST Analysis
- 通过解析代码为抽象语法树，对照动态构建的知识库验证
- 在200个样本上实现100%精准率、87.6%召回率、77%自动修复率
- 适合需要高可靠性代码生成的开发者和工业级应用
用于代码生成的大语言模型虽提升效率，但常引入知识冲突型幻觉（KCHs），即如不存在的API参数等细微语义错误，逃过静态检查工具并导致运行时失败。现有方法如约束解码或非确定性模型内修复对这类错误效果不佳。本文探究是否可通过确定性的静态分析框架可靠地检测并自动修复KCHs。提出一种后处理框架：将生成代码解析为抽象语法树（AST），并基于库内省动态构建的知识库进行验证。该非执行方法使用确定性规则识别并修复API与标识符层级的冲突。在人工标注的200个Python片段数据集上，框架实现了100%精确率、87.6%召回率（F1-score 0.934），成功自动修复了77.0%的已识别幻觉。结果表明，这种确定性后处理方法是概率修复的有效且可靠的替代方案，为可信代码生成提供明确路径。
原文摘要 · Abstract (English)
Large Language Models (LLMs) for code generation boost productivity but frequently introduce Knowledge Conflicting Hallucinations (KCHs), subtle, semantic errors, such as non-existent API parameters, that evade linters and cause runtime failures. Existing mitigations like constrained decoding or non-deterministic LLM-in-the-loop repair are often unreliable for these errors. This paper investigates whether a deterministic, static-analysis framework can reliably detect \textit{and} auto-correct KCHs. We propose a post-processing framework that parses generated code into an Abstract Syntax Tree (AST) and validates it against a dynamically-generated Knowledge Base (KB) built via library introspection. This non-executing approach uses deterministic rules to find and fix both API and identifier-level conflicts. On a manually-curated dataset of 200 Python snippets, our framework detected KCHs with 100\% precision and 87.6\% recall (0.934 F1-score), and successfully auto-corrected 77.0\% of all identified hallucinations. Our findings demonstrate that this deterministic post-processing approach is a viable and reliable alternative to probabilistic repair, offering a clear path toward trustworthy code generation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。