大模型用线性方式编码情境化真相,伙伴观点能影响其判断。
Language Models Encode the Contextual Truth of Propositions
- 在激活空间中用线性方向表示情境真相,跨不同输出策略保持稳定
- 伙伴陈述可显著影响模型对命题的真值判断,即使已有足够证据
- 显式重复错误主张时,真值表征被改变的概率是隐式同意的2.59倍
以往研究发现,大型语言模型在激活空间中沿线性方向编码事实性命题的真值。但这些表征如何延伸到情境真值——即依赖上下文证据而非世界知识决定真伪的命题——尚不明确。本文证明,大模型维持了情境真值的线性表征,且该表征在结构不同的输出策略下依然存在,即使输出无需判断命题真伪。通过协作视觉-语言任务中双模型共享共同认知的对话转录,我们发现模型对某命题的真值表征会显著受合作方陈述的影响,即便模型自身已有充分证据判断真伪。我们还发现,接近决策边界的命题更易被合作方陈述所改变。将表征与输出分离后,可区分两种服从行为:模型可能在输出上附和错误命题,但仍将其表征为假;或跨越边界改变表征。当模型显式重述错误主张时,后者发生的概率是隐式同意的2.59倍。
原文摘要 · Abstract (English)
Prior work has shown that LLMs encode the truth of factual propositions along linear directions in activation space. It's unclear how these representations extend to contextual truth: propositions whose truth is determined by in-context evidence rather than world knowledge. We show that LLMs maintain a linear representation of contextual truth that persists across structurally different output policies, even when the output doesn't require the model to determine a proposition's truth, and show causal evidence via steering experiments. Using the transcripts from a collaborative vision-language task that requires two LLMs to maintain a shared common ground, we show that truth representations of a proposition are significantly swayed by partner assertions about that proposition, even when the LLM has enough evidence to determine its truth. We find evidence that propositions near the decision boundary are more susceptible to having their truth shifted through partner assertions. Separating representation from output distinguish two forms of sycophancy that output behavior alone cannot: the model may accommodate a false proposition while continuing to represent it as false, or shift its representation across the boundary. The latter is 2.59x more common when the model agrees by restating the false claim explicitly than when it agrees implicitly.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。