用因果模型检测大模型推理是否真实驱动决策,发现很多推理只是事后编的。
Project Ariadne: A Structural Causal Framework for Auditing Faithfulness in LLM Agents
- 通过干预推理节点,用因果敏感度衡量推理对结论的影响。
- 实测显示推理与结论脱节率高达0.77,存在严重不一致。
- 适合关注AI可解释性与安全性的研究人员和开发者。
随着大语言模型(LLM)代理被赋予高风险自主决策任务,其推理过程的透明性已成为关键安全问题。尽管链式思维(CoT)提示能生成人类可读的推理轨迹,但这些轨迹究竟是模型输出的真实驱动因素,还是仅是事后合理化,尚不明确。我们提出项目阿里阿德涅(Project Ariadne),一个基于结构因果模型(SCMs)和反事实逻辑的XAI框架,用于审计代理推理的因果完整性。不同于依赖表面文本相似度的现有方法,Project Ariadne对中间推理节点施加硬干预(do-计算),系统性反转逻辑、否定前提、颠倒事实陈述,以测量终端答案的因果敏感度(ϕ)。对前沿模型的实证评估揭示了持续存在的‘忠实性缺口’。我们定义并检测到一种广泛存在的失效模式——因果解耦,其在事实与科学领域中的违反密度(ρ)高达0.77。在此类情况下,即使内部逻辑相互矛盾,代理仍得出相同结论,证明其推理轨迹实为‘推理剧场’,而决策实际由潜在参数先验驱动。研究结果表明,当前代理架构本质上倾向于不可靠的解释,我们提议以阿里阿德涅得分作为对齐表述逻辑与模型行为的新基准。
原文摘要 · Abstract (English)
As Large Language Model (LLM) agents are increasingly tasked with high-stakes autonomous decision-making, the transparency of their reasoning processes has become a critical safety concern. While \textit{Chain-of-Thought} (CoT) prompting allows agents to generate human-readable reasoning traces, it remains unclear whether these traces are \textbf{faithful} generative drivers of the model's output or merely \textbf{post-hoc rationalizations}. We introduce \textbf{Project Ariadne}, a novel XAI framework that utilizes Structural Causal Models (SCMs) and counterfactual logic to audit the causal integrity of agentic reasoning. Unlike existing interpretability methods that rely on surface-level textual similarity, Project Ariadne performs \textbf{hard interventions} ($do$-calculus) on intermediate reasoning nodes -- systematically inverting logic, negating premises, and reversing factual claims -- to measure the \textbf{Causal Sensitivity} ($ϕ$) of the terminal answer. Our empirical evaluation of state-of-the-art models reveals a persistent \textit{Faithfulness Gap}. We define and detect a widespread failure mode termed \textbf{Causal Decoupling}, where agents exhibit a violation density ($ρ$) of up to $0.77$ in factual and scientific domains. In these instances, agents arrive at identical conclusions despite contradictory internal logic, proving that their reasoning traces function as "Reasoning Theater" while decision-making is governed by latent parametric priors. Our findings suggest that current agentic architectures are inherently prone to unfaithful explanation, and we propose the Ariadne Score as a new benchmark for aligning stated logic with model action.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。