揭示大模型'真假认知'的三大误区，提出新方法识别模型内部知识状态。
The Trilemma of Truth in Large Language Models
- 用多实例学习+置信区间，从模型内部表征中判断陈述真伪或无定论。
- 16个开源模型测试显示现有方法不可靠，且真/假信号不对称。
- 发现模型还编码一种既非真也非假的第三类信息，适合可信度研究者。
公众常将大语言模型（LLM）视为具备人类认知能力，认为其‘知道’某些事实。实际上，LLM通过训练过程保留的信息形成内部概率知识。本研究分析现有探查方法，揭示三个根本性假设缺陷。为此，我们提出sAwMIL（稀疏感知多实例学习）框架，结合多实例学习与置信区间预测，利用模型内部表示对陈述分类为真、假或无关。我们在包含16个开源LLM（含默认与对话型）的基准上进行评估，使用三个新构建的数据集。结果表明：(1) 常规探查方法无法提供可靠且可迁移的真伪判断方向，某些情况下甚至劣于零样本提示；(2) 真实与虚假信号在模型中并非对称编码；(3) 模型还编码了一种独立于真/假的第三类信号。
原文摘要 · Abstract (English)
The public often attributes human-like qualities to large language models (LLMs), assuming that they "know" certain things. In reality, LLMs encode information retained during training as internal probabilistic knowledge. This study examines existing methods for probing the veracity of that knowledge and identifies three flawed underlying assumptions. To address these flaws, we introduce sAwMIL (Sparse-Aware Multiple-Instance Learning), a multiclass probing framework that combines multiple-instance learning with conformal prediction. sAwMIL leverages LLMs' internal representations to classify statements as true, false, or neither. We evaluate sAwMIL across 16 open-source LLMs, including default and chat-based variants, using three new curated datasets. Our results show that (1) common probing methods fail to provide a reliable and transferable veracity direction and, in some settings, perform worse than zero-shot prompting; (2) truth and falsehood are not encoded symmetrically; and (3) LLMs encode a third type of signal that is distinct from both true and false.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。