用大模型从新闻中挖掘因果关系并做反事实推理,提升判断准确性。
Counterfactual Causal Inference in Natural Language with Large Language Models
- 通过大模型提取文本中的因果变量,构建可整合的因果图。
- 在真实新闻数据上验证方法,发现模型偏差源于预测误差。
- 适合研究因果推理、可解释AI的学者与工程师参考。
因果结构发现方法通常用于已知变量的结构化数据,依赖统计检验判断因果关系。然而,从新闻等非结构化自然语言中恢复因果结构面临变量未知、缺乏反事实数据等挑战。大语言模型虽展现潜力,但仍存局限。本文提出一种端到端的自然语言因果结构发现与因果推断方法:首先利用大模型从文本中提取实例化因果变量并构建因果图;通过融合多个数据源的因果图,获得尽可能全面的因果集合;随后在估计的因果图上进行反事实推理。因果图条件化有助于减少大模型偏见,更准确表达因果估计量。实验表明,大模型在反事实因果推理中的局限主要来自预测误差,并提出了缓解方向。方法在真实新闻文章上得到验证,展示了其应用可行性。
原文摘要 · Abstract (English)
Causal structure discovery methods are commonly applied to structured data where the causal variables are known and where statistical testing can be used to assess the causal relationships. By contrast, recovering a causal structure from unstructured natural language data such as news articles contains numerous challenges due to the absence of known variables or counterfactual data to estimate the causal links. Large Language Models (LLMs) have shown promising results in this direction but also exhibit limitations. This work investigates LLM's abilities to build causal graphs from text documents and perform counterfactual causal inference. We propose an end-to-end causal structure discovery and causal inference method from natural language: we first use an LLM to extract the instantiated causal variables from text data and build a causal graph. We merge causal graphs from multiple data sources to represent the most exhaustive set of causes possible. We then conduct counterfactual inference on the estimated graph. The causal graph conditioning allows reduction of LLM biases and better represents the causal estimands. We use our method to show that the limitations of LLMs in counterfactual causal reasoning come from prediction errors and propose directions to mitigate them. We demonstrate the applicability of our method on real-world news articles.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。