arXiv:2605.27567cs.AIcs.CL2026-05被引 1

大模型无法可靠发现因果关系,新方法通过外部干预机制突破瓶颈。

Why LLMs Fail at Causal Discovery and How Interventional Agents Escape

论文配图:Why LLMs Fail at Causal Discovery and How Interventional Agents Escape
图 1 · 摘自论文原文
  • 用外部贝叶斯循环替代内部学习,避免因果混淆陷阱
  • 在24变量数据集上超越微调与偏好优化方法
  • 无需训练即可达到基线性能,适合高复杂度因果推断

因果发现是科学推理的核心,但大语言模型能否可靠完成仍存疑。现有基准显示,即使经过微调,模型在简单因果图上表现受限,且复杂度升高时性能下降。我们证明这种失败具有根本性:监督微调、直接偏好优化和上下文学习均产生无法区分生成相似观测数据的因果图的预测器;任何试图突破此限制都需模型内部表示无界增长,违背这些方法的前提。我们提出代理式因果贝叶斯优化(A-CBO),其中冻结的语言模型作为干预查询的应答器,外部贝叶斯循环在对数轮次内聚焦于候选图信念。由于决策发生在障碍不适用的空间外,A-CBO可证明收敛,而底层模型保持不变。在Corr2Cause上,A-CBO无需训练即匹配微调基线;在扩展版Corr2Cause(24变量,18,000测试样本)上,显著优于微调与偏好优化,优势随复杂度提升而扩大。

原文摘要 · Abstract (English)

Causal discovery is a cornerstone of scientific reasoning, yet whether large language models can perform it reliably remains an open question. Recent benchmarks show that even fine-tuned models plateau on simple causal graphs and degrade as complexity grows, but why they fail has not been established. We prove the failure is fundamental: supervised fine-tuning, direct preference optimization, and in-context learning all produce predictors that cannot distinguish between causal graphs generating similar observational data, and any attempt to do so requires the model's internal representations to grow unboundedly, violating the very conditions under which these methods work. We formalize this as a kernel obstruction theorem, establishing that the limitation is intrinsic to the learning paradigm, \emph{not any particular model or dataset}. We propose Agentic Causal Bayesian Optimization (A-CBO), wherein a frozen language model serves as an interventional oracle answering targeted queries about intervention effects, while an external Bayesian loop concentrates beliefs over candidate graphs in logarithmically many rounds. Because the decision operates outside the space where the obstruction applies, A-CBO provably converges while the underlying model remains unchanged. On Corr2Cause, A-CBO matches fine-tuned baselines without any training. On Extended Corr2Cause, a new benchmark scaling to 24 variables with 18K test samples, A-CBO significantly outperforms both fine-tuning and preference optimization, with the advantage growing

因果发现大模型局限代理系统贝叶斯优化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。