arXiv:2602.04918cs.LGcs.CL2026-02

发现大模型处理冲突信息时,靠几何偏移而非压缩记忆来假装服从。

Simulated Adoption: Decoupling Magnitude and Direction in LLM In-Context Conflict Resolution

  • 通过分解残差流的大小和方向,分析模型如何应对上下文冲突。
  • 三款模型中两例在性能下降时残差范数稳定,否定信号稀释假说。
  • 发现模型用近正交向量干扰原知识方向,实现表面服从但不真遗忘。

大型语言模型(LLMs)常优先采纳上下文中的矛盾信息,而非已有参数化记忆,这一现象被称为奉承或合规。然而其机制尚不明确:模型如何通过合规解决知识冲突?这种抑制是源于信号幅度稀释,还是残差流中方向性的几何改变?为此,我们对Qwen-3-4B、Llama-3.1-8B和GLM-4-9B进行了逐层几何分析,将反事实上下文引起的残差流更新分解为径向(范数)与角向(余弦)分量。实证结果否定了“流形稀释”假设的普适性:三款模型中有两款在事实类查询上性能显著下降,但残差范数保持稳定。相反,我们观察到合规始终表现为“正交干扰”——冲突上下文注入一个近似正交于真实方向的引导向量,有效旋转隐藏状态表示。这表明模型并非‘遗忘’或抑制内部真理的幅度,而是通过几何位移机制绕过正确的解码向量,实现模拟采纳,同时保留原有结构幅度。该发现挑战了基于标量置信度检测幻觉的有效性,强调需采用矢量监控来区分真正的知识融合与表面的上下文模仿。

原文摘要 · Abstract (English)

Large Language Models (LLMs) frequently prioritize conflicting in-context information over pre-existing parametric memory, a phenomenon often termed sycophancy or compliance. However, the mechanistic realization of this behavior remains obscure, specifically how the model resolves these knowledge conflicts through compliance, and whether this suppression arises from signal magnitude dilution or directional geometric alteration within the residual stream. To resolve this, we conducted a layer-wise geometric analysis across Qwen-3-4B, Llama-3.1-8B, and GLM-4-9B, decomposing the residual stream updates induced by counter-factual contexts into radial (norm-based) and angular (cosine-based) components. Our empirical results reject the universality of the "Manifold Dilution" hypothesis, as two of the three architectures maintained stable residual norms despite exhibiting significant performance degradation on factual queries. Instead, we observed that compliance is consistently characterized by "Orthogonal Interference," where the conflicting context injects a steering vector that is quasi-orthogonal to the ground-truth direction, effectively rotating the hidden state representation. This suggests that models do not "unlearn" or suppress the magnitude of internal truths but rather employ a mechanism of geometric displacement to bypass the correct unembedding vector, effectively simulating adoption while preserving the original structural magnitude. These findings challenge scalar confidence metrics for detecting hallucinations and underscore the necessity of vectorial monitoring to distinguish between genuine knowledge integration and superficial in-context mimicry.

大模型知识冲突几何机制幻觉检测

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。