通过分析模型内部结构,判断生成代码是否正确。
CodeCircuit: Toward Inferring LLM-Generated Code Correctness via Attribution Graphs
- 用归因图追踪模型生成代码的内在逻辑路径。
- 内部图结构能比表面特征更准确预测代码正确性。
- 适用于需要自检生成代码质量的开发者与研究者。
当前代码验证严重依赖外部机制,如基于执行的单元测试或辅助的大语言模型评判,这些方法往往耗时且受限于评判模型自身能力。这引出一个根本但未被探索的问题:能否仅凭大模型内部计算结构评估其生成代码的功能正确性?本文旨在探究模型神经动态是否包含可解码的、预示代码逻辑有效性的信号。受机械可解释性启发,我们将代码验证视为机械诊断任务,将模型显式算法轨迹映射为行级归因图。通过分解复杂的残差流,我们试图识别区分正确推理与逻辑错误的内部结构特征。在 Python、C++ 和 Java 上的分析表明,内在正确性信号在不同语法中均具有鲁棒性。这些内部图的拓扑特征比表面启发式方法更能可靠预测正确性,并支持针对性的因果干预修复错误逻辑。研究结果确立了内部自省作为生成代码验证的可解码属性。代码已开源:https://github.com/bruno686/CodeCircuit。
原文摘要 · Abstract (English)
Current paradigms for code verification rely heavily on external mechanisms-such as execution-based unit tests or auxiliary LLM judges-which are often labor-intensive or limited by the judging model's own capabilities. This raises a fundamental, yet unexplored question: Can an LLM's functional correctness be assessed purely from its internal computational structure? Our primary objective is to investigate whether the model's neural dynamics encode internally decodable signals that are predictive of logical validity during code generation. Inspired by mechanistic interpretability, we propose to treat code verification as a mechanistic diagnostic task, mapping the model's explicit algorithmic trajectory into line-level attribution graphs. By decomposing complex residual flows, we aim to identify the structural signatures that distinguish sound reasoning from logical failure within the model's internal circuits. Analysis across Python, C++, and Java confirms that intrinsic correctness signals are robust across diverse syntaxes. Topological features from these internal graphs predict correctness more reliably than surface heuristics and enable targeted causal interventions to fix erroneous logic. These findings establish internal introspection as a decodable property for verifying generated code. Our code is at https:// github.com/bruno686/CodeCircuit.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。