让模型学会判断何时信任专家,提升因果发现的准确性。
Learning to Defer for Causal Discovery with Imperfect Experts
- 引入可学习的拒决函数,动态决定用数据还是专家意见
- 在图灵对数据集上表现优于单独使用专家或传统方法
- 能识别专家在哪些领域可靠,适合需要融合知识的因果研究
将专家知识(如大语言模型)融入因果发现算法时,若知识不可靠则易引发问题。专家建议可能与数据结果冲突,其可信度随领域和具体问题变化。现有基于软约束或不一致性的方法无法应对这种差异。为此,我们提出 L2D-CD,一种用于评估专家建议正确性并最优融合数据驱动因果发现结果的方法。通过将学习拒决(L2D)算法适配到成对因果发现任务,我们训练一个拒决函数,根据文本元数据决定是否依赖经典数据方法或专家建议。在标准图灵对数据集上的实验表明,该方法性能优于单独使用因果发现方法或专家。此外,该方法能识别专家表现强或弱的领域。最后,我们提出了扩展至多变量图因果发现的策略,为该方向研究铺路。
原文摘要 · Abstract (English)
Integrating expert knowledge, e.g. from large language models, into causal discovery algorithms can be challenging when the knowledge is not guaranteed to be correct. Expert recommendations may contradict data-driven results, and their reliability can vary significantly depending on the domain or specific query. Existing methods based on soft constraints or inconsistencies in predicted causal relationships fail to account for these variations in expertise. To remedy this, we propose L2D-CD, a method for gauging the correctness of expert recommendations and optimally combining them with data-driven causal discovery results. By adapting learning-to-defer (L2D) algorithms for pairwise causal discovery (CD), we learn a deferral function that selects whether to rely on classical causal discovery methods using numerical data or expert recommendations based on textual meta-data. We evaluate L2D-CD on the canonical Tübingen pairs dataset and demonstrate its superior performance compared to both the causal discovery method and the expert used in isolation. Moreover, our approach identifies domains where the expert's performance is strong or weak. Finally, we outline a strategy for generalizing this approach to causal discovery on graphs with more than two variables, paving the way for further research in this area.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。