发现大模型更新信念时并非始终符合贝叶斯原则，且非贝叶斯方法反而表现更好。
LLMs are not (consistently) Bayesian: Quantifying internal (in)consistencies of LLMs' probabilistic beliefs
- 将大模型视为信息处理规则，用信息处理差距分析其信念更新一致性。
- 部分方法接近贝叶斯更新，但多数采用学习到的启发式策略。
- 非贝叶斯启发式在下游任务中表现更优，暗示模型世界认知存在偏差。
现代AI系统正被部署于医疗、科学和法律等复杂领域，不仅需给出正确答案，还需能随新证据动态更新对世界的不确定信念。本文提出新方法，将大模型视为信息处理规则，利用信息处理差距研究其信念更新的内在一致性。通过大量实验评估多种信念融合策略，发现部分方法接近贝叶斯更新，而另一些则依赖学习到的启发式规则。令人意外的是，这些非贝叶斯启发式在下游任务中表现常优于精确贝叶斯计算，表明大模型对世界的概率建模存在显著偏差。最后，本文展示该度量可作为诊断工具，识别大模型推理系统中的潜在问题。
原文摘要 · Abstract (English)
Modern AI systems are being deployed in complex domains such as medicine, science, and law, where it is important that they not only produce correct answers, but also represent and update uncertain beliefs about the world as new evidence arrives. We introduce the novel technique of studying LLMs as information processing rules and utilize the information processing gap to study the internal (in)consistencies of how LLMs update their probabilistic beliefs from evidence. Our extensive experiments evaluate multiple approaches in which LLMs can incorporate evidence into their beliefs. Some of these approaches produce (nearly) Bayesian updates; others seem to use a learned heuristic. Surprisingly, the non-Bayesian heuristic updates often outperform exact Bayesian computation in terms of downstream task performance -- indicating the LLMs' probabilistic models of the world are misspecified. Lastly, we show how our measure can provide diagnostics to identify issues with LLM-powered inferential systems.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。