LLM能从观测数据中发现因果关系,提升推理准确率。
Can LLMs Leverage Observational Data? Towards Data-Driven Causal Discovery with LLMs
- 将观测数据直接嵌入提示词,让LLM基于数据推理因果关系。
- 在基准数据集上F1得分最高提升0.11,优于传统统计方法0.52点。
- 适合想用LLM做因果推断的研究者,尤其关注数据驱动的场景。
因果发现传统依赖统计方法处理观测数据,常需大规模数据及对因果结构的假设。大型语言模型(LLMs)因具备领域知识,为因果发现带来新可能,但其是否能有效利用观测数据尚不明确。本文探索将观测数据融入LLM推理过程,以实现数据驱动的因果发现。通过成对提示与广度优先搜索(BFS)提示两种策略,将观测数据直接输入提示词,评估LLM从数据中推断因果关系的能力。在基准数据集上的实验表明,引入观测数据使因果发现性能提升,两策略下F1分数最高增加0.11,且相比传统统计因果发现基线最高提升0.52分。结果揭示了LLM在数据驱动因果发现中的潜力与局限,证明其可超越文本元数据,有效解析并利用观测数据进行更精准的因果推理。本研究为迈向全链路LLM驱动的因果发现奠定基础。
原文摘要 · Abstract (English)
Causal discovery traditionally relies on statistical methods applied to observational data, often requiring large datasets and assumptions about underlying causal structures. Recent advancements in Large Language Models (LLMs) have introduced new possibilities for causal discovery by providing domain expert knowledge. However, it remains unclear whether LLMs can effectively process observational data for causal discovery. In this work, we explore the potential of LLMs for data-driven causal discovery by integrating observational data for LLM-based reasoning. Specifically, we examine whether LLMs can effectively utilize observational data through two prompting strategies: pairwise prompting and breadth first search (BFS)-based prompting. In both approaches, we incorporate the observational data directly into the prompt to assess LLMs' ability to infer causal relationships from such data. Experiments on benchmark datasets show that incorporating observational data enhances causal discovery, boosting F1 scores by up to 0.11 point using both pairwise and BFS LLM-based prompting, while outperforming traditional statistical causal discovery baseline by up to 0.52 points. Our findings highlight the potential and limitations of LLMs for data-driven causal discovery, demonstrating their ability to move beyond textual metadata and effectively interpret and utilize observational data for more informed causal reasoning. Our studies lays the groundwork for future advancements toward fully LLM-driven causal discovery.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。