让大模型推理更可信：通过反事实测试强制逻辑一致性。
Causal Consistency Regularization: Training Verifiably Sensitive Reasoning in Large Language Models
- 用反事实扰动（如加减互换）检验推理链，惩罚错误路径仍得正确答案的行为。
- 在数学、逻辑等任务上，推理可信度提升最高达70个百分点，准确率不降反而提高。
- 适用于数学、编程等可验证领域，适合追求高可靠推理的场景。
大语言模型常能给出正确答案，却依赖错误的推理过程，因训练目标仅关注最终答案正确性而非中间推理真实性，影响高风险场景的信任度。本文提出反事实敏感性正则化（CSR），通过在操作层面施加微小扰动（如将“+”替换为“-”），生成逻辑无效但形式相近的反事实推理链，并惩罚模型在这些错误路径下仍得出原答案的情况，从而强化推理过程的因果一致性。该方法训练开销仅增加约9%，采用预热课程与词元子集优化实现高效部署。使用反事实结果敏感性（COS）评估可信度，在算术（GSM8K）、逻辑推导（ProofWriter）、多跳问答（HotpotQA）和代码生成（MBPP）任务中，CSR实现了更高的准确率-可信度权衡，建立新的帕累托前沿。相比标准微调与过程监督，可信度提升最多达70个百分点，跨模型家族迁移成功率达94.2%至96.7%。该方法还可与自洽性等推理时技术结合。总体而言，CSR为数学、形式逻辑、代码等操作明确且可验证的结构化领域提供了一条实用的高可靠性推理路径，覆盖估计40%至60%的高风险推理应用。
原文摘要 · Abstract (English)
Large language models can produce correct answers while relying on flawed reasoning traces, partly because common training objectives reward final-answer correctness rather than faithful intermediate reasoning. This undermines trustworthiness in high-stakes settings. We propose Counterfactual Sensitivity Regularization (CSR), a training paradigm that improves reasoning faithfulness by enforcing causal consistency between reasoning steps and outcomes. CSR automatically applies operator-level interventions to reasoning traces, such as swapping "+" with "-", to generate minimally perturbed counterfactual rationales, and penalizes the model when these logically invalid traces still lead to the original answer. Our implementation is efficient, adding about 9 percent training overhead via a warm-start curriculum and token-subset optimization. We evaluate faithfulness using Counterfactual Outcome Sensitivity (COS), which measures how appropriately answers change under logical perturbations. Across arithmetic (GSM8K), logical deduction (ProofWriter), multi-hop question answering (HotpotQA), and code generation (MBPP), CSR yields improved accuracy versus faithfulness trade-offs, establishing a new Pareto frontier. CSR improves faithfulness over standard fine-tuning and process supervision by up to 70 percentage points, and transfers across model families with 94.2 to 96.7 percent success in structured domains. CSR also complements inference-time methods such as self-consistency. Overall, CSR offers a practical route to more reliable reasoning in structured domains, including mathematics, formal logic, and code, where operators are well-defined and verifiable, covering an estimated 40 to 60 percent of high-stakes reasoning deployments.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。