对比大模型与强化学习训练模型的因果推理能力,发现强化训练能有效减少虚假关联。
Correlation or Causation: Analyzing the Causal Structures of LLM and LRM Reasoning Process
- 构建四变量因果模型,分析问题指令、思考过程等间的因果关系
- 强化训练模型在因果结构上更接近理想状态,偏差和不一致显著降低
- 适合关注AI可解释性与可靠性的研究者与开发者
大型语言模型(LLMs)因缺乏坚实的因果基础,常表现出不忠实、偏见和不一致等问题,可能依赖表面相关而非真实理解。近年来,基于强化学习(RL)与蒸馏等技术的推理模型(LRMs)展现出更高任务准确率,但其对因果性的影响仍不清楚。本研究系统分析了四种关键变量——问题指令(Z)、思考过程(T)、推理步骤(X)与答案(Y)——的结构因果模型(SCMs)。结果表明,经过RLVR训练的LRMs具备更强的因果推理能力,更接近理想因果结构;而传统LLMs及蒸馏型LRMs未能解决因果缺陷。进一步分析显示,RLVR能有效减少虚假相关性,增强真实因果模式,从而缓解不忠实与偏见。对训练过程动态观察发现,虚假特征减少与因果结构改善高度相关,且因果关系随训练持续优化。本研究深化了对推理模型中因果性的理解,揭示了RLVR在提升因果推理中的关键作用,并为未来具强因果基础的AI系统设计提供参考。代码与数据已开源:https://github.com/Harryking1999/CoT_Causal_Analysis。
原文摘要 · Abstract (English)
LLMs suffer from critical reasoning issues such as unfaithfulness, bias, and inconsistency, since they lack robust causal underpinnings and may rely on superficial correlations rather than genuine understanding. Successive LRMs have emerged as a promising alternative, leveraging advanced training techniques such as reinforcement learning (RL) and distillation to improve task accuracy. However, the impact of these training methods on causality remains largely unexplored. In this study, we conduct a systematic causal analysis on LLMs and LRMs, examining structural causal models (SCMs) of four key variables: problem instruction (Z), thinking process (T), reasoning steps (X), and answer (Y). Our findings reveal that RLVR-trained LRMs exhibit enhanced causal reasoning capabilities, aligning more closely with ideal causal structures, while LLMs and distilled LRMs fail to address causality-related deficiencies. Our further investigation indicates that RLVR reduces spurious correlations and strengthens genuine causal patterns, thereby mitigating unfaithfulness and bias. In addition, our inspection on the dynamics of the RLVR training process observes a high correlation between reduced spurious features and improved causal structures, where the causal relationships consistently improve in the training process. This study contributes to the understanding of causality in reasoning models, highlights the critical role of RLVR in enhancing causal reasoning, and provides insights for designing future AI systems with stronger causal foundations. We release our code and data at https://github.com/Harryking1999/CoT_Causal_Analysis.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。