让大模型自动把模糊问题转为贝叶斯模型，无需人工写公式。
Large Language Bayes
- 用大模型生成多种可能的贝叶斯模型，再统一推断
- 仅凭描述和数据就能得到合理预测，不需手动建模
- 适合没时间或不会写贝叶斯模型的研究者
许多领域专家缺乏时间或能力编写正式的贝叶斯模型。本文将非正式问题描述作为输入，结合大语言模型与概率编程语言，定义了一个关于形式化模型、隐变量和数据的联合分布。通过观测数据条件化并积分模型，可得隐变量的后验分布，这带来一个复杂的推断问题。我们提出一种推断方法：从大语言模型生成多个形式化模型，对每个模型执行近似推断，最后进行加权平均。该方法被证明是自归一化重要性采样、马尔可夫链蒙特卡洛和重要性加权变分推断的组合。实验表明，仅凭数据和非正式问题描述即可产生合理预测，无需指定正式模型。
原文摘要 · Abstract (English)
Many domain experts do not have the time or expertise to write formal Bayesian models. This paper takes an informal problem description as input, and combines a large language model and a probabilistic programming language to define a joint distribution over formal models, latent variables, and data. A posterior over latent variables follows by conditioning on observed data and integrating over formal models. This presents a challenging inference problem. We suggest an inference recipe that amounts to generating many formal models from the large language model, performing approximate inference on each, and then doing a weighted average. This is justified and analyzed as a combination of self-normalized importance sampling, MCMC, and importance-weighted variational inference. Experimentally, this produces sensible predictions from only data and an informal problem description, without the need to specify a formal model.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。