arXiv:2509.13334cs.AIcs.LG2025-09被引 7

通过因果干预训练,让大模型推理更可信

FRIT: Using Causal Importance to Improve Chain-of-Thought Faithfulness

  • 用人为干扰推理步骤生成真假对比数据,训练模型识别因果关系
  • 在GSM8K上使Mistral模型可信推理提升3.4个百分点,准确率升7.6%
  • 无需人工标注,可大规模应用,适合追求模型可解释性的研究者

链式思维(CoT)虽能提升大模型在复杂任务上的表现,但近期研究发现其推理步骤常无法真正影响最终答案,导致输出不可靠。现有方法多聚焦于衡量可信度,缺乏系统性提升手段。本文提出可信推理干预训练(FRIT),一种可扩展的对齐方法:通过干预模型生成的CoT中单个推理步骤,构造出具有因果一致性的“真”与“假”推理对,形成合成训练数据;再使用直接偏好优化(DPO)训练模型偏好因果一致的推理路径。在Qwen3-8B和Mistral-7B-v0.1上评估,FRIT使Mistral在GSM8K上的可信推理提升3.4个百分点,准确率提高7.6个百分点。该方法是首个无需监督、可规模化训练语言模型生成更可靠、可解释推理的方案,填补了推理性能与可信度之间的关键空白。代码已开源。

原文摘要 · Abstract (English)

Chain-of-thought (CoT) reasoning has emerged as a powerful tool for improving large language model performance on complex tasks, but recent work shows that reasoning steps often fail to causally influence the final answer, creating brittle and untrustworthy outputs. Prior approaches focus primarily on measuring faithfulness, while methods for systematically improving it remain limited. We introduce Faithful Reasoning via Intervention Training (FRIT), a scalable alignment method that trains models to produce causally consistent reasoning by learning from systematically corrupted examples. FRIT generates synthetic training data by intervening on individual reasoning steps in model-generated CoTs, creating faithful/unfaithful pairs that highlight when reasoning breaks down. We then apply Direct Preference Optimization to teach models to prefer causally consistent reasoning paths. Evaluating on Qwen3-8B and Mistral-7B-v0.1 across factual and symbolic reasoning tasks, FRIT increases faithful reasoning by $3.4$ percentage points for Mistral on GSM8K while improving accuracy by $7.6$ percentage points. Our approach provides the first scalable, supervision-free method for training language models to produce more reliable and interpretable reasoning, addressing a critical gap between reasoning performance and trustworthiness. We release our code at \href{https://github.com/Anut-py/frit}.

链式思维因果推理可信度模型对齐

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。