用因果图解析大模型推理逻辑，让决策过程透明可懂。
LLM Explainability with Counterfactual Chains and Causal Graphs
- 用因果图建模大模型内部概念关系，揭示其推理机制
- 通过反事实链扩充数据，提升因果发现稳定性与解释力
- 适用于医疗诊断、情感分析等场景，适合关注模型可信度的研究者
因果图提供了一种高阶语言来揭示机制的透明性。近期研究利用大语言模型（LLMs）恢复外部世界过程的因果图。本文则反其道而行之，用因果图建模 LLM 推理本身，为利益相关者提供模型如何感知和组织高层概念以生成预测的透明视图。我们提出四阶段方法：给定目标 LLM 和一组文本样例，该方法发现具有类别区分性且人类可理解的概念，并将每个输入映射到模型感知的概念状态。随后引入受 MCMC 启发的反事实增强流程，通过反事实链扩展稀疏观测数据。这使得 σ-CG 能稳定地进行因果发现，生成信息丰富且可解释的因果图。我们在三个 LLM 上应用于疾病诊断、情感分析及 LLM-as-a-judge 分类任务。评估了所学图的预测保真度和结构稳定性，以及反事实增强的收敛性和下游实用性。结果表明，所发现的因果图捕捉到了与模型推理一致的有意义依赖关系。本工作为大模型的概念级可解释性奠定了基础。
原文摘要 · Abstract (English)
Causal graphs provide a high-level language for making mechanisms transparent. Recent work uses Large Language Models (LLMs) to recover causal graphs of external-world processes. Instead, in this paper, we use causal graphs to model LLM inference itself, providing stakeholders with a transparent view of how the model perceives and organizes high-level concepts to produce a prediction. We propose a four-phase method for constructing such graphs. Given a target LLM and a set of textual examples, our method discovers class-discriminative, human-interpretable concepts and maps each input to LLM-perceived concept states. We then introduce an MCMC-inspired counterfactual augmentation procedure that expands the sparse observational data through chains of counterfactuals. This enables stable causal discovery with $σ$-CG, yielding informative, interpretable graphs. We apply our method to three LLMs across disease diagnosis, sentiment analysis, and LLM-as-a-judge classification tasks. We evaluate the learned graphs for predictive fidelity and structural stability, and the MCMC-inspired augmentation for convergence and downstream utility. Our results show that the discovered causal graphs capture meaningful dependencies consistent with LLMs' reasoning. Together, this paper provides a foundation for concept-level explainability of LLMs.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。