arXiv:2607.18820cs.CL2026-07

让大模型推理过程真正支撑答案,解决‘答非所问’问题。

CASE: Causal Alignment and Structural Enforcement for Improving Chain-of-Thought Faithfulness

论文配图:CASE: Causal Alignment and Structural Enforcement for Improving Chain-of-Thought Faithfulness
图 1 · 摘自论文原文
  • 训练时构建反事实数据并优化损失,强化推理链对答案的影响。
  • 推理时屏蔽指令到答案的直接注意力,强制走完整推理路径。
  • 在4个基准上提升37%推理忠实度,适合追求可解释AI的场景。

链式思维(CoT)推理广泛用于提升大语言模型(LLMs)的性能与可解释性,但生成的推理过程未必真实支撑最终答案。本文从因果视角分析,理想的推理应遵循链条 $Z\rightarrow X\rightarrow Y$,即指令 $Z$ 只能通过推理链 $X$ 影响答案 $Y$。然而,传统自回归模型在生成答案时同时依赖指令和推理链,允许指令到答案的直接捷径。为此,本文提出CASE框架,结合训练时的因果对齐与推理时的结构约束:训练阶段构建反事实-推理、偏见指令、空指令数据集,并采用选择性损失微调,增强推理链对答案的依赖,抑制指令捷径;推理阶段屏蔽指令词到答案词的直接注意力,防止模型绕过生成的推理链。信息论分析表明该设计促进忠实推理。在三个模型、四个基准上的实验显示,CASE相比最强基线平均提升37%的推理忠实度,表现出更强跨数据集迁移能力,且保持竞争力的平均准确率。代码已开源。

原文摘要 · Abstract (English)

Chain-of-thought (CoT) reasoning is widely used to improve both the performance and interpretability of large language models (LLMs), yet the generated reasoning may not faithfully support the final answer. We study this problem from a causal perspective, where a faithful CoT process should follow the chain $Z\rightarrow X\rightarrow Y$, with $Z$, $X$, and $Y$ denoting the instruction, reasoning chain, and final answer, respectively. In this process, the instruction should affect the answer only through the reasoning chain. However, conventional autoregressive LLMs condition answer generation on both the instruction and the CoT, which still allows a direct instruction-to-answer shortcut. To address this issue, we propose CASE, a framework that combines training-time causal alignment and inference-time structural enforcement. During training, CASE builds counterfactual-CoT, biased-instruction, and empty-instruction datasets, and applies selective-loss fine-tuning to strengthen CoT-to-answer dependence while suppressing instruction shortcuts. During inference, CASE masks direct attention from instruction tokens to answer tokens, preventing the model from bypassing the generated CoT. We provide an information-theoretic analysis showing how these components promote faithful chains. Experiments on three models and four benchmarks show that CASE achieves a 37\% average per-setting relative improvement in overall CoT faithfulness over the strongest baselines, exhibits stronger cross-dataset faithfulness transfer, and maintains competitive average accuracy. Code is available at https://github.com/oddwang/CASE.

链式思维因果推理忠实度提升大模型可解释性

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。