检验大模型说的信念是否真的影响了它的决策。
When Agents Say One Thing and Do Another: Validating Elicited Beliefs from LLMs
- 用决策理论框架同时收集模型的判断和行为，验证一致性。
- 最强模型的信念与行为差异小，但仍有不一致。
- 无需假设收益函数，就能测试模型是否理性。
大型语言模型（LLMs）在高风险场景中越来越重要，其决策依赖于对未知结果的概率判断。然而，尚不清楚这些模型在做决策时是否表现出一致的信念，以及如何验证其报告的信念。本文提出一种基于决策理论的框架，同时从智能体处获取概率判断和实际决策，并检验二者的一致性。形式上，该方法判断是否存在一个“近似理性”的决策者，其真实信念即为所报告的概率。我们发现，即使不假设智能体的效用函数，该形式化仍可导出可实证检验的条件。在模拟临床诊断任务中，模型报告的信念并不能完美反映其决策所揭示的信息，但最强模型的偏差较小。
原文摘要 · Abstract (English)
Large language models (LLMs) are increasingly deployed in high-stakes settings where good decisions require forming beliefs over the probability of unknown outcomes. However, it is unclear whether LLMs act as if they hold coherent beliefs when making decisions, or if so, how we could validate models' reports of such beliefs. We propose a decision-theoretic framework that elicits both probability judgments and decisions from an agent and tests their mutual consistency. Formally, our methods characterize whether it is possible for the actions to be produced by a ``near-rational" decision maker who holds the elicited probability as their true belief. We show that, perhaps surprisingly, this formalization implies empirically testable conditions even without any assumption about the agent's utility function. Applying our framework to stylized clinical diagnosis tasks, we find that models' reported beliefs are demonstrably imperfect summaries of the information revealed in their decisions, but that the discrepancies are small for the strongest models.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。