用闭环框架自动选大模型生成的因果先验,显著提升因果推断准确率。
When and Why LLM Causal Priors Help: Closed-Loop Prior Selection for Amortized Causal Inference

- 构建闭环优化框架,从候选先验中选出最优者
- 在主域上效果提升2.75倍,误差低于基线模型
- 适合需高精度因果推断的研究者和工业应用
因果效应估计旨在回答干预下结果如何变化,是医学、经济与公共政策的基础任务。先验-数据拟合网络(PFNs)可加速该任务:在大量程序生成的合成因果任务上训练后,模型仅需一次前向传播即可根据观测数据输出干预效应估计。模型性能主要取决于合成训练先验,但当前仍依赖人工设计,成为瓶颈。大语言模型(LLMs)可为特定领域生成合理因果图,提示其可作为先验材料。然而,注入这些图是否有效、为何有效、何时有效仍不明确,实践中依赖手动试错。本文提出一种闭环先验选择框架,将先验注入视为预算约束下的候选先验池优化问题。候选先验经低成本微调后,由以真实域泛化为主导的综合指标评分,优胜者接受完整训练并进行统计验证。在734万参数的Do-PFN上,该框架选出的最佳先验在主评估域实现2.75倍的显著提升,且误差低于未注入的官方基线。邻近监控域的泛化能力显著改善,且无任何监控能力退化。机制实验表明,增益来自蒸馏图的语义内容,而非结构多样性本身(方向性证据)。此框架使LLM因果先验的使用从试错变为可实证验证的选择过程。
原文摘要 · Abstract (English)
Causal effect estimation asks how an outcome would change under an intervention, and medicine, economics, and public policy all treat it as a foundational task. Prior-data fitted networks (PFNs) amortize the task: a model trained on large numbers of programmatically generated synthetic causal tasks reads a new problem's observational data into context and returns an interventional-effect estimate in a single forward pass. The capability of such models is largely determined by the synthetic training prior, which is currently designed by hand, a bottleneck acknowledged by both Do-PFN and CausalPFN. Large language models (LLMs) can now ``draw'' plausible causal graphs for a given domain, suggesting that LLM-distilled graphs could serve as prior material. Whether injecting such graphs helps at all, where any gain comes from, and when injection helps. Practice has so far relied on manual trial and error. We propose a \emph{closed-loop prior selection framework} that casts prior injection as a budget-constrained optimization over a candidate prior pool. Candidates undergo cheap post-training and are scored by a composite metric dominated by real-domain generalization; the winner then receives full training and paired statistical validation. On a 7.34M-parameter Do-PFN, the framework's winner attains a formally significant $2.75\times$ gain on the primary evaluation domain, and its error falls below that of the uninjected official base. Generalization on an adjacent monitoring domain improves significantly, and no monitored capability degrades. Mechanism experiments show that the gain depends on the semantic content of the distilled graph rather than its structural diversity alone does not produce it (directional evidence). With this framework and this regularity in hand, the use of LLM causal priors stops being manual trial and error and becomes an empirically verifiable selection problem.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。