arXiv:2507.12553cs.CLcs.AI2025-07被引 4

语言模型能准确判断事件是否合理,且与人类判断高度一致。

Is This Just Fantasy? Language Model Representations Reflect Human Judgments of Event Plausibility

  • 通过线性向量识别模型对事件合理性的判断机制。
  • 模型越强大,其判断事件合理性的能力越稳定且可预测。
  • 该方法可映射人类对事件合理性的细微判断,适用于认知研究。

语言模型(LMs)被广泛应用于问答、创作奇幻故事等任务,需具备区分句子模态类别(如可能、不可能、完全荒谬等)的能力。然而,近期研究质疑语言模型在模态分类上的可靠性(Michaelov et al., 2025;Kauf et al., 2023)。本文识别出多种语言模型中用于区分模态类别的线性表示,即模态差异向量。分析显示,语言模型具备比以往认为更可靠的模态分类能力。此外,模态差异向量在模型训练过程、层数和参数量增加时呈现一致的出现顺序。值得注意的是,模型激活中的模态差异向量可有效建模人类对事件合理性的精细判断。我们通过将投影沿模态差异向量与人类对可解释特征的评分进行相关性分析,探索了人类如何区分模态类别。本研究利用机械可解释性技术,为理解语言模型及人类的模态判断提供了新视角。

原文摘要 · Abstract (English)

Language models (LMs) are used for a diverse range of tasks, from question answering to writing fantastical stories. In order to reliably accomplish these tasks, LMs must be able to discern the modal category of a sentence (i.e., whether it describes something that is possible, impossible, completely nonsensical, etc.). However, recent studies have called into question the ability of LMs to categorize sentences according to modality (Michaelov et al., 2025; Kauf et al., 2023). In this work, we identify linear representations that discriminate between modal categories within a variety of LMs, or modal difference vectors. Analysis of modal difference vectors reveals that LMs have access to more reliable modal categorization judgments than previously reported. Furthermore, we find that modal difference vectors emerge in a consistent order as models become more competent (i.e., through training steps, layers, and parameter count). Notably, we find that modal difference vectors identified within LM activations can be used to model fine-grained human categorization behavior. This potentially provides a novel view into how human participants distinguish between modal categories, which we explore by correlating projections along modal difference vectors with human participants' ratings of interpretable features. In summary, we derive new insights into LM modal categorization using techniques from mechanistic interpretability, with the potential to inform our understanding of modal categorization in humans.

语言模型模态判断可解释性人类认知

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。