arXiv:2605.09314cs.AI2026-05被引 1

少数注意力头决定大模型是否被说服,且可精准干预。

How LLMs Are Persuaded: A Few Attention Heads, Rerouted

论文配图:How LLMs Are Persuaded: A Few Attention Heads, Rerouted
图 1 · 摘自论文原文
  • 仅少数中间层注意力头控制答案选择,将选项映射到特定顶点。
  • 说服本质是离散的隐空间跳跃,非信心降低或模糊信念。
  • 可定位并操控关键路由特征,适用于安全防御与对抗攻击研究。

语言模型可能被说服而放弃事实知识,这一安全隐患核心机制尚不明确。我们发现一种紧凑的因果机制:少数中层注意力头几乎完全决定模型回答。这些头将答案选项写入低维多面体,不同选项占据独立顶点。说服并非模糊信念或降低信心,而是导致从正确答案顶点到目标顶点的离散隐空间跃迁。决策头并非基于证据推理,而是复制其注意力选中的选项标记。说服通过重定向注意力实现,我们识别出一个秩一的证据路由特征,该特征控制选择路径。直接修改此特征可引导模型决策,移除则阻断说服。进一步追踪发现,该特征由浅层注意力头从输入中的说服性关键词构建而成。每一步均经干预验证。该机制在开源LLM及生成引擎优化等真实中毒场景中普遍存在,揭示说服为窄而可监控的电路。

原文摘要 · Abstract (English)

Language models can be persuaded to abandon factual knowledge. This vulnerability is central to AI safety, but its internal mechanism remains poorly understood. We uncover a compact causal mechanism for persuasion-induced factual errors. A small set of mid-layer attention heads almost entirely determines the model's answer. These heads write answer options into a low-dimensional polyhedron, with options occupying distinct vertices. Persuasion does not blur belief or merely reduce confidence; it causes a discrete latent jump from the correct-answer vertex to the persuasion-target vertex. We show that decision heads are not reasoning over evidence. Instead, they copy whichever option token their attention selects. Persuasion works by redirecting attention. We isolate a rank-one evidence-routing feature that controls the route. Directly modifying this feature steers the model's choice, and removing it blocks persuasion. We then trace the feature back to a band of shallower attention heads that build it from persuasive keywords in the input. Every step is validated by intervention. This mechanism appears across open-source LLMs and realistic poisoning scenarios such as Generative Engine Optimization, revealing persuasion as a narrow, monitorable circuit.

大模型安全注意力机制说服攻击可解释性

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。