arXiv:2605.12922cs.AIcs.CL2026-05被引 3

揭示大模型多轮对话中丢掉任务主线的内在机制。

When Attention Closes: How LLMs Lose the Thread in Multi-Turn Interaction

论文配图:When Attention Closes: How LLMs Lose the Thread in Multi-Turn Interaction
图 1 · 摘自论文原文
  • 提出注意力通道关闭理论,解释任务目标如何在对话中逐渐失联。
  • 发现残差表示能保留目标信息,但注意力已失效时行为仍会崩溃。
  • 适用于调试各类大模型,尤其对长对话应用有指导意义。

大型语言模型在单轮对话中可精准遵循复杂指令,但在长时间多轮交互中常丢失任务主线、角色设定与规则。这种退化现象虽有行为层面的测量,却缺乏机制解释。本文提出‘通道转换’假说:定义目标的标记在注意力中变得不可及,而目标信息可能仍保留在残差表示中。我们引入目标可访问性比(GAR),衡量生成标记对任务定义标记的注意力程度,并结合滑动窗口消融与残差流探测。结果显示,当注意力关闭时,不同架构表现出截然不同的失败模式:部分模型即使注意力归零仍保持目标引导行为,另一些则在残差信息可解码时仍失败,且编码出现层位从第2层到第27层不等。对Mistral模型的因果消融实验显示,在强制关闭注意力通道后,20条事实保留任务召回率从接近完美骤降至11%,角色约束违规率超过对抗压力基线,且两效应均在预测的交叉轮次出现。线性探测器从残差表示中恢复每轮回忆结果的准确率高达AUC 0.99,而输入嵌入始终处于随机水平。跨架构与模型规模下,注意力损失与残差可解码性的差距可预测目标行为是否存活。本文贡献了GAR诊断工具、通道转换框架及窗口注意力关闭下的失败时间参数化预测。

原文摘要 · Abstract (English)

Large language models can follow complex instructions in a single turn, yet over long multi-turn interactions they often lose the thread of instructions, persona, and rules. This degradation has been measured behaviorally but not mechanistically explained. We propose a channel-transition account: goal-defining tokens become less accessible through attention, while goal-related information may persist in residual representations. We introduce the Goal Accessibility Ratio (GAR), measuring attention from generated tokens to task-defining goal tokens, and combine it with sliding-window ablations and residual-stream probes. When attention to instructions closes, what survives reveals architecture. Across architectures, the transition yields qualitatively distinct failure modes: some models preserve goal-conditioned behavior at vanishing attention, others fail despite decodable residual goal information, and the layer at which this encoding emerges varies from 2 to 27. A within-model causal ablation that force-closes the attention channel in Mistral collapses recall from near-perfect to 11% on a 20-fact retention task and raises persona-constraint violations above an adversarial-pressure baseline without user pressure, with both effects emerging at the predictable crossover turn. Linear probes recover per-episode recall outcomes from residual representations with AUC up to 0.99 across all four primary architectures, while input embeddings remain at chance. Across architectures and model scales, the gap between attention loss and residual decodability predicts whether goal-conditioned behavior survives channel closure. We contribute GAR as a diagnostic, the channel-transition framework as a controlled mechanistic account, and a parametric prediction of failure timing under windowed attention closure.

大模型注意力机制对话系统可解释性

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。