用大模型提取概率知识,自动构建贝叶斯网络
Extracting Probabilistic Knowledge from Large Language Models for Bayesian Network Parameterization
- 从大模型中提取事件概率,用于贝叶斯网络参数化
- 在80个领域数据上验证效果,优于随机与均匀分布
- 适合数据少时的专家先验建模,首次建立评估基准
本文评估大型语言模型(LLMs)在构建贝叶斯网络(BNs)中近似领域专家先验的潜力。尽管LLMs在事实知识方面表现良好,但其生成真实世界事件概率知识的能力仍待研究。我们探索利用LLMs中内嵌的概率知识,推导关于事件及其关系的条件概率估计。通过该方法可实现特定领域的概率建模。在涵盖医疗到金融等80个公开贝叶斯网络上的实验表明,向LLMs查询事件条件概率所得结果显著优于随机分布、均匀分布以及基于下一词生成概率的方法。我们进一步验证了这些由LLM生成的概率分布可作为专家先验,用于优化从稀缺数据中提取的分布。本工作提出一种结合大模型概率知识与真实数据的贝叶斯网络自动构建策略,并建立了首个评估大模型提取概率知识能力的综合性基线。
原文摘要 · Abstract (English)
In this work, we evaluate the potential of Large Language Models (LLMs) in building Bayesian Networks (BNs) by approximating domain expert priors. LLMs have demonstrated potential as factual knowledge bases; however, their capability to generate probabilistic knowledge about real-world events remains understudied. We explore utilizing the probabilistic knowledge inherent in LLMs to derive probability estimates for statements regarding events and their relationships within a BN. Using LLMs in this context allows for the parameterization of BNs, enabling probabilistic modeling within specific domains. Our experiments on eighty publicly available Bayesian Networks, from healthcare to finance, demonstrate that querying LLMs about the conditional probabilities of events provides meaningful results when compared to baselines, including random and uniform distributions, as well as approaches based on next-token generation probabilities. We explore how these LLM-derived distributions can serve as expert priors to refine distributions extracted from data, especially when data is scarce. Overall, this work introduces a promising strategy for automatically constructing Bayesian Networks by combining probabilistic knowledge extracted from LLMs with real-world data. Additionally, we establish the first comprehensive baseline for assessing LLM performance in extracting probabilistic knowledge.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。