用可执行的因果模拟器,让大模型学会复杂因果推理
CauSim: Scaling Causal Reasoning with Increasingly Complex Causal Simulators

- 用大模型逐步构建可运行的因果模型,实现复杂系统建模
- 在多个任务上实现因果推理性能提升,且数据越多效果越好
- 适合研究因果推理、大模型自我改进的学者使用
尽管大语言模型在数学、编程等知识密集型任务上已超越人类,但在因果推理方面仍表现不佳。核心挑战在于:因果系统本身复杂,常以不可执行的形式表达,且因果问题的真实答案稀缺。我们提出CauSim框架,将因果推理从稀缺标签问题转化为可扩展的监督学习问题。CauSim通过大模型逐步构建越来越复杂的可执行结构因果模型(SCMs),能够扩展到全局复杂系统,同时保证因果查询答案可验证。该框架跨表示运作:将非可执行因果知识形式化为代码,实现数据增强;并将可执行的SCMs翻译为自然语言,实现以往难以监督的表示上的标注。研究分为两部分:(1) 如何构建日益复杂的因果模拟器;(2) 系统性研究CauSim的能力,证明其在跨表示泛化、课程式规模扩展和数据量增加下的持续增益、大模型通过自生成模拟器实现自我改进,以及通过对已有领域知识的形式化实现数据增强。
原文摘要 · Abstract (English)
Despite surpassing human performance across mathematics, coding, and other knowledge-intensive tasks, large language models (LLMs) continue to struggle with causal reasoning. A core obstacle is the target data itself: causal systems are complex and often expressed in non-executable forms, while ground-truth answers to causal queries are inherently scarce. We introduce CauSim, a framework that turns causal reasoning from a scarce-label problem into a scalable supervised one. CauSim constructs increasingly complex causal simulators: executable structural causal models (SCMs), incrementally built by LLMs, that scale to globally complex systems while maintaining verifiable answers to causal queries. CauSim operates across representations by formalizing non-executable causal knowledge into code, enabling data augmentation, and translating executable SCMs into natural language, enabling supervision in previously difficult-to-supervise representations. We structure our research into two parts: (1) how to construct increasingly complex causal simulators, and (2) a systematic study of what CauSim enables, demonstrating generalization across representations, consistent gains from curriculum scaling and data volume, LLM self-improvement through self-generated simulators, and data augmentation via formalization of existing domain knowledge.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。