用一致性评分引导LLM推断因果顺序,提升真实场景下因果推理可靠性。
Retrieving Classes of Causal Orders with Inconsistent Knowledge Bases
- 基于变量对的一致性评分构建半完整有向图,抽象出因果顺序
- 在科学文献数据集上准确恢复正确因果顺序,效果优于传统方法
- 适合需要可信因果分析的医学、公共卫生领域研究者使用
传统因果发现方法依赖强且不可验证的假设,难以应用于现实场景。大型语言模型(LLMs)可从文本元数据中提取因果知识,整合领域专家经验,但存在幻觉问题。为此,我们提出一种基于一致性评分的因果顺序抽象方法。通过计算变量间的成对一致性分数,构建半完整部分有向图,并在此基础上识别最大化一致性的最优部分有向无环图与无环锦标赛集合。该方法能有效区分直接与间接因果关系,避免因模糊表述导致的歧义。我们在流行病学与公共卫生领域的大量因果DAG上进行评估,结果表明该方法能准确恢复真实因果顺序,为基于LLM的因果推断提供了可靠、实用的新框架。
原文摘要 · Abstract (English)
Traditional causal discovery methods often depend on strong, untestable assumptions, making them unreliable in real-world applications. In this context, Large Language Models (LLMs) have emerged as a promising alternative for extracting causal knowledge from text-based metadata, effectively consolidating domain expertise. However, LLMs are prone to hallucinations, necessitating strategies that account for these limitations. One effective approach is to use a consistency measure as a proxy of reliability. Moreover, LLMs do not clearly distinguish direct from indirect causal relationships, complicating the discovery of causal Directed Acyclic Graphs (DAGs), which are often sparse. This ambiguity is evident in the way informal sentences are formulated in various domains. For this reason, focusing on causal orders provides a more practical and direct task for LLMs. We propose a new method for deriving abstractions of causal orders that maximizes a consistency score obtained from an LLM. Our approach begins by computing pairwise consistency scores between variables, from which we construct a semi-complete partially directed graph that consolidates these scores into an abstraction. Using this structure, we identify both a maximally oriented partially directed acyclic graph and an optimal set of acyclic tournaments that maximize consistency across all configurations. We further demonstrate how both the abstraction and the class of causal orders can be used to estimate causal effects. We evaluate our method on a wide set of causal DAGs extracted from scientific literature in epidemiology and public health. Our results show that the proposed approach can effectively recover the correct causal order, providing a reliable and practical LLM-assisted causal framework.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。