arXiv:2508.12495cs.CLcs.AI2025-08被引 15

通过显式建模因果图,让大模型更少说谎。

Mitigating Hallucinations in Large Language Models via Causal Reasoning

  • 训练模型先画出变量间的因果图,再基于图推理
  • 在CLADDER任务上达95.33%准确率,首次超过人类
  • 适用于需要逻辑严谨性的问答、决策类场景

大型语言模型(LLMs)常出现看似合理却违反逻辑的幻觉。现有推理方法如思维链(CoT)仅在文本标记层面操作,无法建模变量间的因果关系,缺乏对条件独立性及因果识别假设的支持。为此,我们提出因果-DAG构建与推理(CDCR-SFT)框架,通过监督微调使模型显式构建变量级有向无环图(DAG),并在此基础上进行推理。我们还构建了包含25,368个样本的因果推理数据集(CausalDR),每条数据包含问题、显式因果图、图推理轨迹和验证答案。在四个大模型上跨八项任务的实验表明,该方法将因果推理准确率提升至CLADDER任务的95.33%(首次超越人类94.8%),并在HaluEval上将幻觉减少10%。结果证明,显式建模因果结构能有效缓解大模型输出中的逻辑不一致问题。代码已开源。

原文摘要 · Abstract (English)

Large language models (LLMs) exhibit logically inconsistent hallucinations that appear coherent yet violate reasoning principles, with recent research suggesting an inverse relationship between causal reasoning capabilities and such hallucinations. However, existing reasoning approaches in LLMs, such as Chain-of-Thought (CoT) and its graph-based variants, operate at the linguistic token level rather than modeling the underlying causal relationships between variables, lacking the ability to represent conditional independencies or satisfy causal identification assumptions. To bridge this gap, we introduce causal-DAG construction and reasoning (CDCR-SFT), a supervised fine-tuning framework that trains LLMs to explicitly construct variable-level directed acyclic graph (DAG) and then perform reasoning over it. Moreover, we present a dataset comprising 25,368 samples (CausalDR), where each sample includes an input question, explicit causal DAG, graph-based reasoning trace, and validated answer. Experiments on four LLMs across eight tasks show that CDCR-SFT improves the causal reasoning capability with the state-of-the-art 95.33% accuracy on CLADDER (surpassing human performance of 94.8% for the first time) and reduces the hallucination on HaluEval with 10% improvements. It demonstrates that explicit causal structure modeling in LLMs can effectively mitigate logical inconsistencies in LLM outputs. Code is available at https://github.com/MrLYG/CDCR-SFT.

因果推理幻觉抑制大模型

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。