语言模型在数值上下文中概率不准确，常受选项顺序干扰。
Language Model Probabilities are Not Calibrated in Numeric Contexts
- 测试模型在等可能与非等可能场景下的输出概率是否匹配真实分布。
- GPT-4o-mini和Llama-3.1-8B均存在系统性偏差，概率分配不准确。
- 选项顺序、词频等文本特征显著影响模型判断，适合评估模型可靠性者关注。
某些陈述有唯一合理延续（如“埃菲尔铁塔位于[巴黎]”），而另一些则有多个自然选项分布（如“硬币投掷结果是[正面/反面]”）。我们认为语言模型应能捕捉这类自然分布。本文聚焦于模型输出概率在数值上下文中的校准性。若上下文涉及两个等可能事件（如公平硬币正反面），模型输出概率也应相等；若事件概率不均（如双骰子点数之和为7），模型应输出成比例的概率。然而，我们在简单设定下发现，当前最优模型（1）校准性差，（2）存在系统性偏差：词身份、词序、词频等文本特征显著影响概率分配。例如，gpt-4o-mini常优先选择提示中第一个选项，无论其真实可能性；而Llama-3.1-8B则倾向选第二个。模型未能以校准方式在有效选项间分配概率质量。
原文摘要 · Abstract (English)
Some statements have one well-defined continuation (e.g., "the Eiffel Tower is in [Paris]"), whereas others have a natural distribution over multiple options (e.g., "the weighted coin flip was [Heads/Tails].") We argue that language model (LM) outputs should capture these natural distributions. Our work specifically tests whether LM output probabilities are calibrated to numeric information within their textual contexts. For example, if the context (the prompt) concerns two equally likely options (e.g., heads or tails for a fair coin), the LM output probabilities should also be equal. Likewise, in a context with nonuniformly likely events (e.g., rolling a pair with two dice) an LM should output proportionate probabilities. However, we find that even in simple settings, the best LMs (1) are poorly calibrated and (2) have systematic biases: artifacts like word identity, word order, and word frequency all impact calibration. For example, gpt-4o-mini often picks the first of two options presented in the prompt regardless of the options' implied likelihoods, whereas Llama-3.1-8B picks the second. Models do not allocate probability mass among valid options in a calibrated manner.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。