用领域知识提升从非结构化数据中发现因果关系的准确性
DKCD: Domain Knowledge-Enhanced Causal Discovery from Unstructured Data

- 引入领域知识增强的因果发现框架,分三步完成
- 在两个专业数据集上显著提升因果因子识别与图构建效果
- 适合医疗、金融等高专业度领域的研究人员使用
从非结构化数据中进行因果发现是医疗、金融、教育等高专业度领域中一个具有挑战性但研究不足的任务。现有方法通常利用大语言模型的通用知识从非结构化数据中识别因果因素,并将其标注为结构化数据以构建因果图。然而,这些方法仍面临两大挑战:(CH1) 隐含因素识别不足,因缺乏领域特定知识,而这些因素对因果发现至关重要;(CH2) 因子标注不可靠,源于缺乏领域支撑的推理,导致错误传播至最终因果图。为此,我们提出一种新的领域知识增强型因果发现框架(DKCD),包含三个相互关联的组件:(1) 知识挖掘:基于可观测因素检索相关领域知识,支持后续因果推理;(2) 知识引导的因果推理:结合相关知识,发现隐含因果因素以应对CH1,生成关键因果线索以提高标注准确性以应对CH2;(3) 因果结构发现:基于更完整的因子集和准确标注构建最终因果图。在两个领域专用数据集上的实验表明,DKCD显著提升了因果因子识别与因果图构建的效果。
原文摘要 · Abstract (English)
Causal discovery from unstructured data is a challenging yet underexplored task in high-expertise domains such as healthcare, finance, and education. Existing methods typically leverage the general knowledge of large language models (LLMs) to identify causal factors from unstructured data and annotate them into structured data for causal graph construction. However, they remain limited by two key challenges (CHs): (CH1) insufficient identification of latent factors, which are implicit in the data yet essential for causal discovery, due to the lack of domain-specific knowledge; and (CH2) unreliable factor annotation, caused by the lack of domain-grounded reasoning, which propagates errors to the resulting causal graphs. To address these challenges, we introduce a novel Domain Knowledge-enhanced Causal Discovery framework (DKCD) for causal discovery from unstructured data in high-expertise domains with three interconnected components: (1) Knowledge Mining: It retrieves relevant domain knowledge based on observable factors to support subsequent causal reasoning. (2) Knowledge-guided Causal Reasoning: Reasoning with relevant knowledge, it discovers latent causal factors to address CH1 and generates key causal clues for more accurate data annotation to address CH2. (3) Causal Structure Discovery: It constructs the final causal graphs based on a more complete factor set and accurate annotations. Experiments on two domain-specific datasets show that DKCD significantly improves both causal factor identification and causal graph construction.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。