构建通用因果推理数据集,让大模型学会真正理解因果关系。
Towards a Universal Causal Reasoner

- 设计统一框架生成18类因果问题数据,覆盖从相关到干预的完整因果层级。
- 训练后模型在18类任务上平均提升22.9%,跨领域基准上领先8.1%。
- 适合医疗、法律等需严谨推理场景,显著提升模型决策可信度。
尽管因果推理至关重要,但训练大语言模型进行因果推理仍处于探索阶段。现有数据集多聚焦于因果性的特定方面,难以用于训练具备泛化能力的因果推理模型。为此,我们提出UniCo数据生成框架,该框架涵盖佩尔的因果阶梯中的18类因果查询,并将原生符号化示例转化为代码与自然语言形式,以模拟现实场景中因果术语未被明确指定的情况。为保证数据质量,UniCo基于精确的因果推断生成答案,并过滤存在推理捷径的样本。使用66.6K条UniCo生成实例进行监督微调后,Qwen3-4B、Qwen3-8B和Olmo-3-7B-Instruct在所有18个分布内查询类型上平均提升22.9%,在7个外部已建立的因果基准测试中相较现有最优数据生成框架提升8.1%。更重要的是,在真实医疗理解、法律决策和表格推理任务中,经UniCo训练的模型展现出更忠实的推理过程,其在可信度指标上相比基础模型平均提升20.2%。这表明以因果为中心的训练不仅能增强因果推理能力,还能使大模型在一般推理任务中形成因果思维模式。
原文摘要 · Abstract (English)
Despite the importance of causal reasoning, training LLMs to reason causally remains underexplored. Existing data efforts mostly focus on benchmarking LLMs on specific aspects of causality, making them less suitable for training generalizable causal reasoners. To address this, we propose UniCo, a data generation framework that both (1) addresses 18 causal query types across Pearl's Causal Ladder and (2) translates natively symbolic examples into code and natural language forms to simulate real-world use cases where causal terms are not explicitly specified. To ensure data quality, UniCo grounds answers with exact causal inference and filters cases with reasoning shortcuts. Upon supervised finetuning with 66.6K UniCo-generated instances, Qwen3-4B, Qwen3-8B and Olmo-3-7B-Instruct achieve an average of 22.9% improvements across all 18 in-distribution query types, and 8.1% over state-of-the-art causal data generation frameworks on 7 established causal benchmarks outside the training distribution. More importantly, in real-world medical understanding, legal decision, and tabular reasoning, UniCo-trained models consistently display more faithful reasoning traces, outperforming the base models by an average of 20.2% in faithfulness metrics. These suggest that causality-centered training not only strengthens causal reasoning, but also equips LLMs with a causal mindset in general reasoning tasks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。