发现大模型顶层注意力作用有限，信息处理主要靠内部机制。
Attend First, Consolidate Later: On the Importance of Attention in Different LLM Layers
- 顶层隐藏状态被替换后性能影响小，说明注意力依赖低
- 早期层状态替换导致性能暴跌，显示前期处理关键
- 顶层忽略输入词替换，适合快速推理场景优化
在基于解码器的大型语言模型中，某一层的表示既作为下一层的输入，也作为后续词注意力机制的输入。本文发现后者的作用可能被高估。通过在四款模型和四项任务上操作前序词的隐藏状态（如用随机向量替换第k层状态），我们发现若此操作位于模型顶层（最后30%-50%层），性能下降通常很小甚至可忽略；而早期层的操作则会导致接近随机水平的表现。进一步实验中，将提示词中的'Italy'替换为'France'，若在顶层执行该替换，模型仍答'罗马'；若在底层执行，则正确回答'巴黎'。结果表明，Transformer类大模型存在两阶段过程：第一阶段汇聚前序信息，第二阶段主要进行内部处理。
原文摘要 · Abstract (English)
In decoder-based LLMs, the representation of a given layer serves two purposes: as input to the next layer during the computation of the current token; and as input to the attention mechanism of future tokens. In this work, we show that the importance of the latter role might be overestimated. To show that, we start by manipulating the representations of previous tokens; e.g. by replacing the hidden states at some layer k with random vectors. Our experimenting with four LLMs and four tasks show that this operation often leads to small to negligible drop in performance. Importantly, this happens if the manipulation occurs in the top part of the model-k is in the final 30-50% of the layers. In contrast, doing the same manipulation in earlier layers might lead to chance level performance. We continue by switching the hidden state of certain tokens with hidden states of other tokens from another prompt; e.g., replacing the word "Italy" with "France" in "What is the capital of Italy?". We find that when applying this switch in the top 1/3 of the model, the model ignores it (answering "Rome"). However if we apply it before, the model conforms to the switch ("Paris"). Our results hint at a two stage process in transformer-based LLMs: the first part gathers input from previous tokens, while the second mainly processes that information internally.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。