通过内外一致性检测,高效识别大模型推理中的不忠实现象。
Detecting Unfaithful Chain-of-Thought via Circuit-Guided Internal-External Discrepancy

- 基于内部计算路径与外部推理文本的差异度量
- 在四个数据集上达到当前最优检测效果
- 适合关注模型可解释性与推理可信度的研究者
链式思维(CoT)推理提升了大语言模型的问题求解能力,但生成的推理过程可能并不真实反映模型的实际决策流程。现有检测方法主要依赖生成推理文本的外部信号,如逻辑合理性或答案一致性,忽视了模型内部计算证据。尽管近期的电路追踪方法可通过追踪信息流动获取内部证据,但构建长序列推理的完整电路成本高、难扩展。为此,我们提出电路引导的内外不一致评分器(CIE-Scorer),用于实例级的CoT不忠实检测。核心思想是:忠实的推理应与模型计算过程一致,不忠实推理则存在偏差。CIE-Scorer从关键推理词元高效追踪紧凑的句子级电路,构建内部与外部推理图,并使用融合格罗莫夫-沃瑟斯坦距离衡量其差异。在FaithCoT-Bench的四个数据集上的实验表明,CIE-Scorer在显著降低电路构建成本的同时,达到当前最佳性能,验证了结合机制可解释性信号与外部推理痕迹的有效性。
原文摘要 · Abstract (English)
Chain-of-thought (CoT) reasoning improves the problem-solving ability of large language models (LLMs), but generated reasoning traces may not faithfully reflect the model's actual decision process. Existing CoT unfaithfulness detectors mainly rely on external signals from generated rationales, such as textual plausibility or answer consistency, while overlooking evidence from the model's internal computation. Although recent circuit tracing methods provide a way to obtain model-internal evidence by tracing how information flows through model components during reasoning, constructing full reasoning circuits for long CoTs is costly and difficult to scale. To address these challenges, we propose Circuit-guided Internal-External Discrepancy Scorer (CIE-Scorer), a framework for instance-level CoT unfaithfulness detection. The key idea is that faithful reasoning traces should align with the model's computational process, whereas unfaithful traces may diverge from it. CIE-Scorer efficiently traces compact sentence-level circuits from informative reasoning tokens, constructs internal and external reasoning graphs, and measures their discrepancy using Fused Gromov--Wasserstein distance. Experiments on four datasets from FaithCoT-Bench show that CIE-Scorer achieves state-of-the-art performance while reducing the cost of circuit construction, demonstrating the effectiveness of combining mechanistic interpretability signals with external reasoning traces for CoT unfaithfulness detection.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。